Selective multimodal deep learning for reliable breast cancer subtype classification from histopathology and genomic data

View at publisher
Authors: Nabil Hezil, Ahmed Bouridane, Rifat Hamoudi, Somaya Al-Maadeed, Younes Akbari, Faseela Abdullakutty
Cited by: 3
References: 27

Abstract

) with non-overlapping 256 × 256 pixel patches, while RNA-seq features were standardized and projected into a shared latent space. We evaluated multiple fusion techniques including gated attention, cross-attention, and concatenation. To enhance transparency, attention rollout was applied to visualize feature importance across WSIs. An uncertainty-aware smart routing mechanism was introduced to selectively use RNA-Only predictions for high-confidence cases or defer to multimodal inference for ambiguous samples, optimizing computational efficiency. Evaluated on the TCGA-BRCA dataset comprising 924 patients with PAM50 molecular subtypes, our routing-based multimodal model achieved 94.93% accuracy, outperforming both unimodal baselines and fixed fusion strategies. The routing mechanism required multimodal inference for only 38.2% of samples, achieving a 3.12× computational speedup while maintaining superior accuracy. Confusion matrices revealed significant improvements in distinguishing closely related subtypes such as Luminal A and Luminal B. Attention rollout provided interpretable heatmaps localizing discriminative histological regions associated with each subtype, aligning with established pathological criteria. This study demonstrates that integrating CTransPath-derived histology features with transcriptomic profiles through confidence-aware routing offers a practical, explainable, and computationally efficient approach for breast cancer subtype classification suitable for clinical decision support systems.

Fields of Study & Subjects

AI in cancer detection Single-cell and spatial transcriptomics Cell Image Analysis Techniques

Publication details

Journal / source
Medical Engineering & Physics
Publisher
IOP Publishing
Volume
147
Issue
3
Pages
035006–035006
Publication date
11 February 2026
Language
EN