Seeing Isn't Always Believing: Analysis of Grad-CAM Faithfulness and Localization Reliability in Lung Cancer CT Classification

Explainable Artificial Intelligence (XAI) techniques, such as Gradient-weighted Class Activation Mapping (GradCAM), have become indispensable for visualizing the reasoning process of deep neural networks in medical image analysis. Despite their popularity, the faithfulness and reliability of these heatmap-based explanations remain under scrutiny. This study critically investigates whether Grad-CAM truly represents the internal decision-making of deep models trained for lung cancer image classification. Using the publicly available IQ-OTH/NCCD dataset, we evaluate five representative architectures - ResNet50, ResNet-101, DenseNet-161, EfficientNet-B0, and ViT-Base-Patch16-224 - to explore model-dependent variations in GradCAM interpretability. We introduce a quantitative evaluation framework that combines localization accuracy, perturbationbased faithfulness, and explanation consistency to assess GradCAM reliability across architectures. Experimental findings reveal that while Grad-CAM effectively highlights salient tumor regions in most convolutional networks, its interpretive fidelity significantly degrades for Vision Transformer models due to nonlocal attention behavior. Furthermore, cross-model comparison indicates substantial variability in saliency localization, implying that Grad-CAM explanations may not always correspond to the true diagnostic evidence used by the networks. This work exposes critical limitations of current saliency-based XAI approaches in medical imaging and emphasizes the need for model-aware interpretability methods that are both computationally sound and clinically meaningful. Our findings aim to inspire a more cautious and rigorous adoption of visual explanation tools in medical AI, urging the community to rethink what it truly means to “trust” a model's explanation.