Separation-like irregularity and sample size optimism in high-discrimination logistic prediction models

Closed-form minimum sample size criteria for developing logistic prediction models, such as the Riley framework implemented in pmsampsize, are widely used but may become optimistic when anticipated discrimination is high. We conducted a Monte Carlo simulation study to compare the formula-based recommended development sample size, nRiley, with an empirical required sample size, nreq, defined by out-of-sample calibration-slope stability under repeated development sampling. Scenarios fixed the candidate parameter dimension at p = 10 and crossed predictor distribution (normal, standardized skewed continuous, binary), signal density (dense versus sparse), prevalence (ϕ∈{0.05,0.10,0.20}), and target discrimination (AUCtarget∈{0.70,0.75,0.80,0.85,0.90}), with intercept and signal strength calibrated to match targets. We defined nreq as the smallest n such that 𝔼(bn)≥0.90 and Pr(bn