Adaptive Machine Learning for Dynamic Environments: Evaluating Data Drift-Triggered Retraining in COVID-19 Severity Prediction

Machine learning (ML) models deployed in operational systems often face performance degradation due to data drift. Designing effective, scalable strategies for adaptive model maintenance remains an open challenge. This study evaluates four retraining strategies—naïve (static), periodic, clinically guided context-driven, and data drift-triggered—in a dynamic, high-stakes environment: predicting severe COVID-19 outcomes. Using a large-scale CDC dataset, we find that data drift-triggered retraining strikes an effective tradeoff between predictive performance and retraining cost. It matches the performance of periodic retraining while requiring far fewer retraining cycles and offers a fully automated mechanism for adaptive learning. In contrast, context-driven retraining performs well and requires fewer retraining cycles but depends on expert input and lacks automation. Our findings provide empirical insights and practical guidance for designing adaptive ML systems in dynamic environments, with implications for both researchers and practitioners deploying ML models in healthcare and other domains with evolving data landscapes.