MedExplainer: an interpretable ensemble parallel-tree framework for interpreting vision–language models in medical imaging

Large multimodal vision–language models (VLMs) have achieved remarkable success in image and video analysis. However, their inherent black-box nature limits their applicability in medical imaging, where interpretability is critical. Although several existing explainable machine learning techniques partially address this issue, they often suffer from model dependency, interpretability randomness, and high computational complexity. To overcome these challenges, this paper proposes MedExplainer, an innovative explainable ensemble parallel-tree framework. The method first generates a set of perturbed samples by applying fixed masking operations to individual images or video frames, and then obtains prediction scores through the VLM. Using Parallel bagging and boosting strategies, MedExplainer constructs parallel decision trees and visualizes feature importance for local interpretability, while aggregating all sample-level explanations to produce a comprehensive global interpretation. The experiment evaluated the method on multiple medical image and video datasets, and the results demonstrate that the proposed MedExplainer provides stronger and more consistent explanations than existing methods. Interestingly, this design also enables the enhancement of lightweight VLMs for medical image segmentation tasks. In particular, the MedExplainer-augmented medicalGemma-4B model outperforms Google’s recently released Gemini Nano Banana on segmentation benchmarks. Demonstration videos are available in the supplementary materials.