A leakage-controlled and SHAP driven machine learning framework for paediatric respiratory disease classification using Indian hospital EHR data

BACKGROUND: Classification of paediatric respiratory diseases is important for timely diagnosis and decision-making. Real-world clinical data is often compounded by challenges of data leaking, class imbalance, and high dimensional data modalities, which might compromise the reliability, methodological rigour, and interpretability of the model. METHODS: This research work introduces a leakage-controlled and explainable machine learning model for multiclass classification of respiratory diseases in children namely Bronchitis, Bronchopneumonia, Upper Respiratory Tract Infection (URTI), and Wheeze using both structured numerical and unstructured free-text information obtained from practical Electronic Health Records (EHRs). The dataset includes 1,121 paediatric cases sourced from an Indian hospital. A multi-step pre-processing pipeline was applied, including data quality filtering prior to partitioning train-only imputation, and a two-stage data leakage prevention mechanism. Numerical variables were standardized, and clinical free-text narratives were vectorized using Term Frequency–Inverse Document Frequency (TF–IDF) encoding. Class imbalance was handled using the Synthetic Minority Oversampling Technique (SMOTE) applied strictly within cross-validation folds to avoid contamination. SHapley Additive exPlanations (SHAP) were used on a Random Forest baseline to audit for diagnostic leakage and guide feature selection. The top 100 SHAP-ranked features were then used to train Logistic Regression, Random Forest, XGBoost and Stacking Ensemble models. SHAP ranking was performed exclusively on the training partition following train-test splitting, ensuring that feature selection was not influenced by test set information. RESULTS: SHAP analysis confirmed that predictions were driven by clinically relevant features such as age, breathing difficulty days, Peripheral Oxygen Saturation (SpO₂), respiratory rate (RR), and serum bicarbonate. “Respiratory System” feature was identified as leakage-prone and was excluded from final model training. Random Forest achieved the highest hold-out test accuracy of 0.8578 (95% Confidence Interval (CI): 0.8089–0.9022), with XGBoost achieving the highest micro-averaged Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.9706 and Area Under the Precision-Recall Curve (AUPRC) of 0.9285, computed using a one-vs-rest strategy. The narrow Cross Validation (CV) test gap is consistent with limited overfitting within this single-centre internal validation setting. CONCLUSION: The proposed pipeline supports transparent and leakage-controlled validation within a single-centre internal setting, contributing to methodological rigour in paediatric EHR-based Machine Learning (ML research). External validation across diverse clinical settings would be required before consideration of clinical deployment. The study also contributes to global health goals by supporting the development of equitable and reliable diagnostic technologies aligned with Sustainable Development Goals, particularly SDG 3 (Good Health and Well-being) and SDG 9 (Industry, Innovation, and Infrastructure).