Sparse maximum likelihood estimation of regression models

Abstract For regression model selection and estimation, we study a small set of candidate models of maximum likelihood from which all information criteria such as the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) choose their models. We show that asymptotically sufficiently large models in this set are all correct models in that they contain all active variables. Such a concentration of correct models in this small set encourages selecting models from this set. The constrained minimum criterion (CMC) of Tsao is well suited for selecting models from this set. As a by‐product of this study, we obtain selection consistency of the CMC. We also compare CMC with AIC and BIC through a simulation study.