Decoupled Transferability of Calibration and Explainability in Cross-Cohort Breast Cancer Survival Prediction

Background. Clinical machine-learning models are increasingly evaluated for trustworthiness as well as predictive accuracy. Two pillars of trust — the reliability of predicted probabilities (calibration) and the reliability of feature-level explanations — are usually studied in isolation. How these two pillars behave together under cross-cohort distribution shift remains an open question, with direct implications for clinical deployment.Methods. Seven classifiers — Logistic Regression, Random Forest, XGBoost, LightGBM, an MLP, and two soft-voting hybrid MLP-plus-tree ensembles — were trained on the SEER breast cancer cohort (n = 4,024; 84.7% five-year alive) and externally validated on the METABRIC cohort (n = 1,904; 42.1% alive) using six shared clinical features. Three calibration methods (temperature scaling, Platt scaling, isotonic regression) were compared, with the best selected per model by Expected Calibration Error (ECE) on the SEER validation partition. SHAP attribution was computed for every model using the family-appropriate exact or sampling-based explainer; importance rankings were validated through cross-validation stability, clinical-alignment scoring, and cross-model Spearman rank correlation. Cross-cohort transfer was assessed separately for calibration (ECE, MCE, Brier, NLL on METABRIC after applying the SEER-fit calibrator) and for feature importance (per-model Spearman rank correlation between SEER and METABRIC mean |SHAP| vectors).Results. Isotonic regression reduced SEER test ECE by 50–92% across all seven models. The same SEER-fit calibrators, applied unchanged to METABRIC, increased ECE by a factor of two to three for every model on both the imputed and complete-case external cohorts, a direct consequence of the base-rate shift from 84.7% to 42.1% alive. By contrast, SHAP feature-importance rankings transferred strongly across cohorts in the five-model base-model analysis: tree-based models (RF, XGB, LGBM) achieved SEER–METABRIC Spearman rank correlations of ρ ≈ 0.94 (p < 0.01), with Age and Nodes Positive top-ranked in both cohorts; LR and MLP showed positive but non-significant agreement (ρ = 0.66 and 0.71, respectively) in the six-feature setting. Cross-model agreement was very high among the tree-based and hybrid models (ρ > 0.91), and the top-ranked features for every model lay within the established set of breast-cancer prognostic factors.Conclusions. Calibration and feature importance dissociate under prevalence shift: probability scale is cohort-specific, but feature ranking is cohort-stable. This two-layer view supports a concrete deployment rule — feature-level explanations can be reported with confidence across cohorts, but predicted probabilities require per-cohort recalibration on a labeled sample of the target population. Treating calibration and explanation as separate transferability questions clarifies what can and cannot be carried across cohorts in clinical machine-learning practice.