Evidence over explanations: put medical AI to the test
Abstract
Medical AI often optimizes end-to-end performance while offering post-hoc explanations, widening an epistemic gap between behavior and justified clinical claims. Interpretable models and XAI help, but are insufficient. We advocate governance by testability: preregistered empirical trials of causal alignment (avoiding shortcuts) and invariance across scanners, sites, and subgroups, plus monitoring and rollback. We propose Institutional AI—auditable provenance, prospective validation, external audits, and analogous adversarial evaluations for LLMs.
Fields of Study & Subjects
Artificial Intelligence in Healthcare and Education
Explainable Artificial Intelligence (XAI)
Machine Learning in Healthcare
Publication details
- Journal / source
- npj Artificial Intelligence
- Publisher
- Springer Science and Business Media LLC
- Volume
- 2
- Issue
- 1
- Publication date
- 5 June 2026
- Language
- EN