Comparing Post-Training Threshold Selection Methods for Predicting Pediatric Asthma-Related Readmissions

Background: In binary classification for healthcare applications, choosing an appropriate decision threshold is critical, as classification outcomes can directly affect patient care. In particular, clinical models for predicting emergency department (ED) or hospital readmissions must carefully balance the risk of false positives and false negatives. Objectives: This project aims to compare four post-training threshold selection methods in the context of predicting asthma-related ED and hospital readmissions among pediatric patients, focusing on maximizing precision while maintaining a minimum recall level. Methods: We evaluated Grid Search, GHOST, Bootstrap, and Order Statistics methods using both simulated data and real-world pediatric data from the Children’s Hospital of Eastern Ontario (CHEO). The goal was to identify thresholds that maximize precision while ensuring recall ≥ 0.80. Model performance was assessed based on precision, recall, and threshold stability across repeated runs. Results: Grid and GHOST methods consistently achieved recall above 0.80, with Grid performing best. Order Statistics achieved the highest precision but often fell below the recall threshold. Bootstrap showed moderate performance in both precision and recall, with slightly less stability across repetitions. Conclusion: When high recall is the priority, Grid and GHOST are preferred. Order Statistics is suitable for maximizing precision but may compromise recall. Bootstrap provides a reasonable balance. These findings support more informed threshold strategy decisions for clinical risk prediction models.