Model selection uncertainty and the problem of averaging model coefficients

Model selection uncertainty and the problem of averaging model coefficientsYou may recall the adage that "wildlife science is not rocket science, it's much harder," which reflects that the great majority of our research takes place in uncontrolled circumstances.We never quite have the data we need, nor can we exactly observe the processes we aim to effectively manage.We are instead hamstrung by suites of correlated, surrogate variables from which we attempt to draw useful inferences.From the first issue of The Journal of Wildlife Management (JWM), our authors have endeavored to publish the most robust and reliable science to inform conservation and management actions.Since the late 1990s and early aughts, statistical approaches in JWM have become progressively dominated by information-theoretic methods, or model selection approaches, built on the foundations laid by Burnham and Anderson (1998, 2002) using Akaike's Information Criterion (AIC) and similar criteria.One clear value of such an approach is the ability to talk in terms of the weight of evidence behind alternative models, with each model representing a specific hypothesis of how you think the system works.Model selection uncertainty occurs when statistical measures, such as AIC, cannot differentiate substantive informational differences between models that contain either different sets of covariates or the same covariates in different forms.Thus, a second strength of model selection approaches is the ability to transparently discuss uncertainty with respect to different hypotheses.Yet dealing with model selection uncertainty, when your candidate model set lacks one clearly best supported model, has vexed every practitioner in some way or another.Given a set of competing models, when drawing inferences, one might opt to select the most parsimonious model, while another might desire to combine inferences and uncertainties across the suite of competing models.Those wishing to combine inferences across competing models are the target audience for this editorial, yet even those favoring the parsimony path will have something to gain (so please read on!).Burnham and Anderson (1998, 2002) outlined model averaging, i.e., averaging either predictions or coefficient values across competing models, as 2 approaches for combining inferences under model selection uncertainty.The popularity of AIC, availability of software that conducts model averaging, and desire to include uncertainty as part of the study conclusions likely explain the prevalence of model averaging today (Banner and Higgs 2017).However, there is a poorly recognized reality that although it is generally appropriate mathematically to average model predictions, averaging of model coefficients is statistically valid only under an extremely narrow set of conditions that effectively precludes its utility for wildlife biologists (Posada and Buckley 2004, Cade 2015, Banner and Higgs 2017, Dormann et al. 2018).Later I will outline the specific conditions under which it is appropriate to average model coefficients, but first let's understand the problem. THE PROBLEM WITH AVERAGING MODEL COEFFICIENTSModel parameters are estimated conditional on the given model structure.More specifically, each estimated coefficient represents a rate of change in the response variable given a unit change in the covariate, conditional on other covariates in the model (Cade 2015).Change the model structure, say by leaving out a single variable, and parameter estimates for the same covariate in these 2 different models may become incommensurable (Banner and Higgs 2017)-meaning they cannot be compared on the same scale.Dormann et al. (2018 [Appendix S1.1]) illustrated this problem with the case of best subsets regression, where the averaged parameter estimates of the resulting model (which often include the same set of covariates as the full model) differ from the maximum likelihood estimates of the structurally identical full model-Journal of Wildlife Management 2026;e70186.wileyonlinelibrary.