Distilling knowledge from AI models used to detect heart conditions in 12-lead ECG traces

Abstract Background/Introduction Deep learning matches or exceeds expert arrhythmia detection, but its black-box nature limits clinical use. Explainable AI (XAI) can accelerate cardiovascular research by converting accurate AI predictions into actionable human knowledge, especially in underexplored domains. However, ECG-focused XAI faces issues: analyses limited to single predictions, superficial heatmaps lacking causal depth, and vulnerability to cherry-picking. Concept Relevance Propagation (CRP) offers dataset-wide, concept-level explanations, but has not been adopted for ECG data in cardiovascular applications and lacks thorough medical data validation. Purpose In this work, we tested the hypothesis that CRP can translate AI insights into human-understandable knowledge in a quantitatively and qualitatively verifiable way. To evaluate this, we analyzed a high-performing model predicting atrial fibrillation (AF) and assessed how its patterns align with clinical knowledge. Methods To enable our analysis, we adapted a state of the art ECG model in PyTorch via knowledge distillation, training on PTB-XL (median age 62, IQR 22, 52% male) and validating on CPSC-2018 (median age 62, IQR 26, 54% male) using AUROC. A custom canonizer/composite enabled CRP through batch-norm-rich residual blocks. We validated implementation choices using Symmetric Relevance Gain (SRG) bit-flipping across layers. Concepts were ranked by mean relevance for AF prediction across both datasets and visualized with RelMax. To test faithfulness, we pruned the model to the top