An external, contemporary evaluation of the Epic End of Life Care Index among hospitalized patients across two large health systems: A retrospective cohort study

ABSTRACT Background The Epic End of Life Care Index (EOLCI) predicts one-year mortality and was developed to improve serious illness care. However, prior external EOLCI evaluations had limited sample sizes, populations, and equity evaluations. In preparation for a multi-system pragmatic clinical trial, we sought to evaluate the EOLCI performance and equity in the trial’s two participating health systems. Objective Evaluate EOLCI model performance overall and across key subgroups. Design/Setting/Patients Retrospective cohort study of patients hospitalized for ≥36 hours in 2022 to 39 hospitals in the Trinity Health and Kaiser Permanente Southern California (KPSC) health systems. Measurements We predicted one-year mortality risk stratified by health system using the EOLCI, a logistic regression model including age, sex, race/ethnicity, ethnicity, insurance, and diagnoses. We evaluated model performance using Scaled Brier Scores (SBS; range -1 to 1; composite measures of calibration and discrimination), calibration plots, and c-statistics. Results Among 116,749 Trinity patients with 154,063 encounters, 12,054 (10.3%) patients died within one year. Among 94,489 KPSC patients with 133,043 encounters, 16,872 (17.9%) died within one year. The SBS was -0.007 at Trinity and 0.178 at KPSC. Calibration was poor for both. Trinity’s discrimination was acceptable/good (c-statistic 0.76, 95% CI 0.76-0.77), and KPSC’s was good/very good (c-statistic 0.81, 95% CI 0.81-0.81). Model performance across subgroups was similar to the overall cohort. Limitations Death data were collected exclusively within Trinity and KPSC, risking outcome misclassification; several subgroup evaluations were limited by small sample sizes. Conclusions An external evaluation of the widely available Epic EOLCI demonstrated adequate to very good discrimination, poor calibration, and equitable performance across sociodemographic characteristics and diagnoses in two of the nation’s largest health systems. Primary funding source PCORI PLACER-2022C3-30553.