Saliency-Guided Transformer Attention With Pixel-Level Contrastive Learning for Weakly Supervised Defect Localization

Weakly supervised learning has emerged as a powerful paradigm for image segmentation, providing a practical solution to reduce the dependence on costly, pixel-level annotated datasets. By leveraging partial annotations, such as image-level tags, this approach significantly reduces the labeling burden while maintaining competitive performance. However, weakly supervised methods inherently face challenges, including overlapping activation regions and insufficient localization due to sparse and noisy signals from image-level tags, often leading to suboptimal segmentation performance. These issues are further exacerbated in industrial defect localization, where datasets present unique complexities, including low-contrast object boundaries, inconsistent shapes, interclass similarities, and substantial intraclass variations. To address these limitations, we introduce a novel saliency-guided transformer attention with contrastive learning framework. The proposed framework leverages Transformer attention to generate localization maps and enhances the learning of challenging foreground and background information through saliency-guided cues. In addition, a pixel-level contrastive learning module is employed to refine feature map representations by bringing positive pairs closer together and pushing negative pairs apart, effectively addressing challenges, such as overlapping activations and ambiguous boundaries. Through extensive experiments and ablation studies, the proposed method demonstrates superior performance compared to state-of-the-art approaches across three defect segmentation datasets. We also evaluated the generalization of our approach on the PASCAL VOC segmentation and MVTec anomaly detection datasets.