LEUKA-FUSENET: Multimodal Acute Leukemia Detection with Hybrid vit-CNN, Uncertainty Quantification and SHAP-Enhanced Explainability

LEUKA-FuseNet is a multimodal deep learning framework that classifies white blood cells (WBCs) as benign or malignant. YOLOv11 is leveraged in the system to automatically detect and crop WBCs from peripheral blood smear images, thereby ensuring accurate localization of cell areas. The detection of the cells initiates dual backbone networks which include ResNet-50 and Hybrid Vision Transformer (DINO) to extract local and global visual features of the cells, resulting in a better representation of cell morphology. The deep learning model also incorporates complete blood count (CBC) data with image embeddings through multimodal fusion to further improve diagnostic accuracy. Predictive uncertainty is estimated using Monte Carlo Dropout, providing a confidence measure for each prediction to assist clinicians. Explainability is achieved with Grad-CAM visualization to highlight significant image regions, in addition to SHAP analysis to explain the contributions of clinical and visual features. The multimodal approach improves the reliability and trustworthiness for WBC classification. The complete framework is deployed using Streamlit to enable real-time predictions, uncertainty estimation and visual explanation. Thus, this framework provides an intelligent and interpretable tool to assist with leukemia diagnosis through multimodal analysis.