A cumulative, pipeline-oriented framework for extreme imbalanced classification: Large-scale benchmarking of 4,100 rare-event models

Extreme class imbalance remains a long-standing challenge in supervised learning, particularly when the minority class corresponds to rare but high-impact events such as fraud, equipment failure, or retirement transitions. Although a wide range of imbalance-handling techniques has been proposed—including penalization, resampling, loss shaping, threshold calibration, optimization, and ensemble strategies—the field largely lacks an operational understanding of how these components should be combined, calibrated, and consolidated within end-to-end pipelines. This paper introduces a cumulative, pipeline-oriented framework for imbalanced classification under extreme skew. Rather than categorizing learning algorithms or paradigms, the framework reorganizes the design space of imbalanced learning around where and how imbalance-aware interventions act within the modeling pipeline. By progressively integrating data-level adjustments, objective and decision-rule shaping, and higher-level consolidation mechanisms, the framework exposes the structural logic underlying effective rare-event classifiers. We validate this framework through a large-scale benchmark comprising 4,109 pipeline configurations applied to administrative microdata from the Spanish Social Security system, where monthly retirement detection exhibits a positive-class prevalence of 2.34%. The results reveal clear and interpretable performance regimes as pipeline complexity increases. Explicit pipeline-level ensemble architectures achieve the strongest recall and balance-sensitive metrics, while well-calibrated and optimized intermediate pipelines frequently attain comparable or superior performance in selected metrics—such as F1-score and Matthews correlation coefficient—relative to ensemble baselines, at substantially lower computational cost and with improved interpretability. Distributional and robustness analyses further show that consolidation mechanisms act primarily by stabilizing the upper tail of the performance distribution rather than by improving isolated best-case outcomes. Beyond this application, the proposed framework provides a reproducible benchmarking protocol and a unifying operational lens for understanding when additional methodological sophistication yields robust, stable, and practically meaningful performance gains under extreme class imbalance.