Agreement between heuristic shrinkage factor and optimal shrinkage factors in logistic regression for risk prediction: a simulation study across different sample sizes and settings

Abstract Introduction The heuristic shrinkage factor of Van Houwelingen and Le Cessie (𝑆 𝑉𝐻 ) is a commonly used closed-form solution to adjust for overfitting in unpenalised logistic regression models for risk prediction. It is also the basis of widely-adopted minimum sample size criteria for developing clinical prediction models. However, current evidence is lacking regarding the bias of 𝑆 𝑉𝐻 compared to the optimal shrinkage factor (𝑆 𝑜𝑝𝑡 ). Here, we examine this issue and also assess the bias of an alternative bootstrap-derived shrinkage factor (𝑆 𝑏𝑜𝑜𝑡 ). Methods We undertook two simulation studies. The first examined the bias of 𝑆 𝑉𝐻 and 𝑆 𝑏𝑜𝑜𝑡 as estimators of 𝑆 𝑜𝑝𝑡 across a range of different scenarios defined by 𝐶 𝑝𝑜𝑝 , the C-statistic of the model developed in a population sized dataset. The second examined the bias of 𝑆 𝑜𝑝𝑡 when using development sample sizes targeting a shrinkage of 0.9, based on a sample size calculation defined by 𝑆 𝑉𝐻 itself (𝑁 𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙 ) or by an adapted simulation-based approach (𝑁 𝑠𝑖𝑚 ). Results For high C-statistics, 𝑆 𝑉𝐻 overestimated 𝑆 𝑜𝑝𝑡 , whereas for low C-statistics 𝑆 𝑉𝐻 underestimates 𝑆 𝑜𝑝𝑡 . For example, across scenarios when 0.8≤𝐶 𝑝𝑜𝑝