A Comparison of the Effectiveness of Model Selection Criteria for Regression Model
Abstract
The aim of this study is to compare the effectiveness of the ten model selection criteria for regression model, namely, AIC, BIC, HQIC, AICc, AICu, HQICc, KIC, KICcC, KICcSB, and KICcHM. The conditions for simulation were the differences in sample size, number of parameters in the model, regression coefficient, error variance, and distribution of independent variable. The results of the study showed that a small sample case and the true model is somewhat difficult to identify, the appropriate criteria are AIC, HQIC. Medium sample cases and the true model is somewhat difficult to identify, the appropriate criterion is AIC. If the true model can be specified more easily for both small and medium samples, the appropriate criteria are AICu, KICc. For the large sample cases, and the true model is somewhat difficult to identify, the appropriate criteria are AIC, BIC. If the true model can be specified more easily, the appropriate criterion is BIC.Keywords : model selection criterion, regression model, efficiencyReferences
Akaike, H. (1974). A New Look at the Statistical Model Identification. IEEE Transactions on Automatic Control, 19(6), 716-723.
Cavanaugh, J.E. (1999). A Large-Sample Model Selection Criterion Based on Kullback’s Symmetric Divergence. Statistics and Probability Letters, 42(4), 333-343.
Cavanaugh, J.E. (2004). Criteria for Linear Model Selection Based on Kullback’s Symmetric Divergence. Australian and New Zealand Journal of Statistics, 46(2), 257-274.
Hafidi, B. & Mkhadri, A. (2006). A Corrected Akaike Criterion Based on Kullback’s Symmetric Divergence: Applications in Time Series, Multiple and Multivariate Regression. Computational Statistics and Data Analysis, 50(6), 1524-1550.
Hannan, E.J. & Quinn, B.G. (1979). The Determination of the Order of an Autoregression. Journal of the Royal Statistical Society Series B, 41(2), 190-195.
Hurvich, C.M. & Tsai, C.L. (1989). Regression and Time Series Model Selection in Small Samples. Biometrika, 76(2), 297-307.
Keerativibool, W. (2014a). Unifying the Derivations of Kullback Information Criterion and Corrected Versions. Thailand Statistician Journal of Thai Statistical Association, 12(1), 37-53.
Keerativibool, W. (2014b). Study on the Penalty Functions of Model Selection Criteria. Thailand Statistician Journal of Thai Statistical Association, 12(2), 161-178.
Keerativibool, W. & Siripanich, P. (2017). Comparison of the Model Selection Criteria for Multiple Regression Based on Kullback-Leibler’s Information. Chiang Mai Journal of Science, 44(2), 699-714.
McQuarrie, A.D.R. & Tsai, C.L. (1998). Regression and Time Series Model Selection. World Scientific, Singapore.
McQuarrie, A.D.R., Shumway, R.H. & Tsai, C.L. (1997). The Model Selection Criterion AICu. Statistics and Probability Letters, 34(3), 285-292.
Montgomery, D.C., Peck, E.A. & Vining, G.G. (2006). Introduction to Linear Regression Analysis. (4th ed.). New York: John Wiley & Sons.
Neath, A. & Cavanaugh, J.E. (1997). Regression and Time Series Model Selection Using Variants of the Schwarz Information Criterion. Communication in Statistic-Theory and Method, 26(3), 559-580.
Sangthong, M. (2019). A Study of the Effectiveness of Model Selection Criteria for Multilevel Analysis. Burapha Science Journal, 24(1), 156-169. (in Thai)
Schwarz, G. (1978). Estimating the Dimension of a Model. The Annals of Statistics 6(2), 461-464.
Seghouane, A.K. & Bekara, M. (2004). A Small Sample Model Selection Criterion Based on Kullback’s Symmetric Divergence. IEEE Transactions on Signal Processing, 52(12), 3314-3323.
Cavanaugh, J.E. (1999). A Large-Sample Model Selection Criterion Based on Kullback’s Symmetric Divergence. Statistics and Probability Letters, 42(4), 333-343.
Cavanaugh, J.E. (2004). Criteria for Linear Model Selection Based on Kullback’s Symmetric Divergence. Australian and New Zealand Journal of Statistics, 46(2), 257-274.
Hafidi, B. & Mkhadri, A. (2006). A Corrected Akaike Criterion Based on Kullback’s Symmetric Divergence: Applications in Time Series, Multiple and Multivariate Regression. Computational Statistics and Data Analysis, 50(6), 1524-1550.
Hannan, E.J. & Quinn, B.G. (1979). The Determination of the Order of an Autoregression. Journal of the Royal Statistical Society Series B, 41(2), 190-195.
Hurvich, C.M. & Tsai, C.L. (1989). Regression and Time Series Model Selection in Small Samples. Biometrika, 76(2), 297-307.
Keerativibool, W. (2014a). Unifying the Derivations of Kullback Information Criterion and Corrected Versions. Thailand Statistician Journal of Thai Statistical Association, 12(1), 37-53.
Keerativibool, W. (2014b). Study on the Penalty Functions of Model Selection Criteria. Thailand Statistician Journal of Thai Statistical Association, 12(2), 161-178.
Keerativibool, W. & Siripanich, P. (2017). Comparison of the Model Selection Criteria for Multiple Regression Based on Kullback-Leibler’s Information. Chiang Mai Journal of Science, 44(2), 699-714.
McQuarrie, A.D.R. & Tsai, C.L. (1998). Regression and Time Series Model Selection. World Scientific, Singapore.
McQuarrie, A.D.R., Shumway, R.H. & Tsai, C.L. (1997). The Model Selection Criterion AICu. Statistics and Probability Letters, 34(3), 285-292.
Montgomery, D.C., Peck, E.A. & Vining, G.G. (2006). Introduction to Linear Regression Analysis. (4th ed.). New York: John Wiley & Sons.
Neath, A. & Cavanaugh, J.E. (1997). Regression and Time Series Model Selection Using Variants of the Schwarz Information Criterion. Communication in Statistic-Theory and Method, 26(3), 559-580.
Sangthong, M. (2019). A Study of the Effectiveness of Model Selection Criteria for Multilevel Analysis. Burapha Science Journal, 24(1), 156-169. (in Thai)
Schwarz, G. (1978). Estimating the Dimension of a Model. The Annals of Statistics 6(2), 461-464.
Seghouane, A.K. & Bekara, M. (2004). A Small Sample Model Selection Criterion Based on Kullback’s Symmetric Divergence. IEEE Transactions on Signal Processing, 52(12), 3314-3323.
Downloads
Published
2022-05-20
Issue
Section
Research Article
