Inter-clinician diagnostic agreement of shock etiology: a multicenter observational study

Purpose: We sought to (1) quantify lack of inter-clinician diagnostic agreement of shock etiology and (2) predict patients without complete inter-clinician diagnostic agreement of shock etiology. Methods: This multicenter retrospective, cohort study identified patients evaluated by two or more clinicians who documented a shock diagnosis from 2018 to 2023 across intensive care units (ICU) at 9 acute care hospitals. Shock etiology was abstracted using regular expression from clinician notes in the electronic health record then was made into a 9-dimensional vector representing 9 different shock etiologies. Inter-clinician diagnostic agreement of these vectors was calculated for each patient using Cosine Similarity Scores. Measure of agreement was based on cosine similarity of etiology vectors, not clinical adjudication. Patients without complete inter-clinician diagnostic agreement (Cosine Similarity Score < 1) were compared to patients with diagnostic agreement. Machine learning models were tested to predict patients without complete inter-clinician diagnostic agreement. Results: < 0.001). Machine learning models that most accurately predicted patients without diagnostic agreement were logistic regression (Accuracy: 0.8597, F1-Score: 0.9117, AUC-ROC: 0.9257), random forest (Accuracy: 0.8658, F1-Score: 0.9201, AUC-ROC: 0.9255), and gradient boosting (Accuracy: 0.8515, F1-Score: 0.9138, AUC-ROC: 0.9227). Conclusion: Patients without complete inter-clinician diagnostic agreement of shock etiology can be successfully predicted. Supplementary Information: The online version contains supplementary material available at 10.1007/s13755-025-00423-w.