Adversarial CAM Guidance for Chest X-Ray Classification: Reducing Framing Sensitivity with Mask Supervision

The deep classifiers used for chest X-ray diagnosis can be sensitive to the visual frame around the evidence, producing correct labels for the wrong reasons by relying on the background, borders, text marks, or acquisition artifacts. This framing sensitivity reduces their trustworthiness, and they can fail under a distribution shift. Inspired by biological vision, especially figure-ground segregation and selective attention, we propose a bio-inspired adversarial attention alignment training process that encourages evidence-centered decisions without changing the classifier structure. A classifier is first trained with image-level labels. Class activation mapping (CAM) is then used to produce a differentiable heatmap that indicates where the model attends. We treat this heatmap as a generated localization map and train a discriminator to distinguish the generated heatmaps from ground truth masks. The classifier is updated using a joint objective that preserves the classification performance while pushing its CAM heatmap toward a mask-like structure, reducing the reliance on background cues. We also introduce evaluation measures for the test phase, including augmentation inconsistency (prediction flip rate under angle-based augmentations) and framing sensitivity (CAM energy outside the mask). The experiments show improved lung-focused attention and robustness, while requiring masks only during training and no additional inputs at inference.