Post-reduction inference for confidence sets of models

Summary Sparsity in a regression context makes the model itself an object of interest, pointing to a confidence set of models as the appropriate presentation of evidence. A difficulty in areas such as genomics, where the number of candidate variables is vast, arises from the need for preliminary reduction prior to the assessment of models. The present paper considers a resolution using inferential separations fundamental to the Fisherian approach to conditional inference, namely, the sufficiency/co-sufficiency separation,and the ancillary/co-ancillary separation. Tests of model adequacy based on such separations do not involve specifying a direction for departure from any postulated model, avoiding issues of calibration that would arise in directed tests from using the same data for reduction and for model assessment.In idealised cases with no nuisance parameters, the separations extract the relevant information without loss or redundancy. The extent to which estimation of nuisance parameters affects this idealisation is illustrated in detail for the normal- theory linear regression model, extending immediately to a log-normal accelerated-life model for time-to-event outcomes. As part of the analysis, we introduce a modified version of the refitted cross-validation estimator of Fan et al. (2012), whose distribution theory is tractable in the appropriate conditional sense. The paper concludes, among other things, that in settings where reduction on a reduced sample is unproblematic, sample splitting has high efficiency relative to our conditional analysis, echoing Cox (1975); otherwise it gives miscalibrated confidence sets.