An interpretable two-stage prediction framework for fatal crashes: mitigating extreme data imbalance problem

Traffic crash analysis of mountainous freeways frequently relies on imbalanced data, hindering effective prediction of fatal crashes. Previous research has not fully explored performance improvement for fatal crash prediction, leading to insufficient model performance. This study proposes a two-stage prediction (TSP) framework based on Stacked Sparse Autoencoder (SSAE), which sequentially classifies each injury severity to enhance prediction performance. First, K-means is used for unsupervised clustering to improve data homogeneity. The Adaptive Synthetic Sampling Approach (ADASYN) balances the dataset by increasing sample size. Then, LightGBM with Partial Dependence Plot (PDP) analysis identifies key features and reveals their nonlinear relationships with injury severity. The TSP-SSAE model is compared with Support Vector Machine (SVM), LightGBM, and Deep Neural Network (DNN). Results show TSP-SSAE achieves higher accuracy, precision, recall, and F1-score. It effectively handles extreme data imbalance and improves predictive performance, particularly enhancing fatal crash prediction accuracy, thereby providing insights for traffic safety management.