Improving trust in AI diagnosis of pulmonary hypertension with patient-specific insight

Although AI diagnostic models can achieve high accuracy, their black-box nature limits clinical adoption due to poor interpretability and transparency. To address this, we employed Local Interpretable Model-Agnostic Explanations (LIME) to generate patient-specific insight into the results from the PH-Model, a machine-learned model for detecting pulmonary hypertension (PH). Our approach improves explanation fidelity using bootstrapped sampling, kernel width optimization, and local model performance analysis. We introduce PHysiological Local EXplanations (PHLEX), which aggregates feature importance into six physiologically meaningful categories, and PHysiological Global EXplanations (PHGEX), derived from true positive and negative subgroups. Together, PHLEX and PHGEX enhance interpretability and robustness, reducing complexity from 217 input features to 6 categories. Fidelity of explanations was confirmed through statistical testing, and PHGEX generalized well to unseen datasets. This framework provides reliable, patient-specific explanations without requiring prior knowledge of model inputs, supporting the white-boxing of diagnostic models and aiding clinical decision-making in cardiovascular care. • A framework for stable and physiologically interpretable explanations of AI diagnostics. • PHLEX aggregates feature-level explanations into cardiovascular physiological mechanisms. • PHGEX enables population-level analysis of explanation patterns across patient cohorts. • LIME instability and high-dimensional explanation challenges are systematically addressed. • Explanations support transparent non-invasive detection of pulmonary hypertension.