Development and validation of a deep learning-based emergency triage model: a feasibility and effectiveness study

BACKGROUND: Due to insufficient triage accuracy, general purpose Large Language Models (LLMs) cannot be directly applied to clinical pre-triage. At the same time, the existing emergency pre-triage systems also have certain limitations. Dedicated self-trained models, with their advantages of high performance, easy scalability, and low cost, have become the primary development direction in the field of medical diagnosis. We aimed to propose a method for training a dedicated emergency triage model to improve the accuracy of triage level classification for emergency patients. METHODS: This study employed emergency triage data from December 2023 to December 2024 (raw data: 270,298 entries, cleaned data: 113,302 entries). The data was temporally partitioned into a training set (n = 91,743; 8:2 split for training and validation), a test set (n = 11,559), and an online diagnostic set (n = 10,000). The data was uploaded to the EasyDL platform to develop and deploy an emergency triage model, followed by the evaluation of model performance and online diagnosis. RESULTS: The emergency triage model developed on the EasyDL platform demonstrated reliable performance within defined parameters. The model achieved 97.2% accuracy (95% CI: 96.2–98.2%) on the training set and 97.7% (95% CI: 96.8–98.6%) on the test set. Post-deployment, the overall triage accuracy reached 89.45%, with > 90% accuracy across all disease severity levels (I–IV). The Matthews Correlation Coefficient (MCC) was 0.92. External validation achieved an accuracy of 78.3% (95% CI: 76.9–79.8%), representing a modest decrease compared to the Online Diagnostic set. The development process was simple and efficient; the model can be deployed to mobile terminals via an Application Programming Interface (API), enabling direct clinical application. CONCLUSION: The deep learning-based emergency triage model demonstrates potential for clinical translation, and its development plan is operationally feasible. However, to realize its practical value, subsequent research and efficacy validation in real-world clinical settings are necessary prior to implementation.