SEM-CLIP 2.0: Precise Zero-/Few-Shot Learning for Nanoscale Defect Detection in SEM Image

In integrated circuit manufacturing, detecting and classifying stochastic morphological wafer surface defects is crucial for root cause analysis and yield enhancement. Complex background patterns in scanning electron microscope (SEM) images and diverse defect textures pose significant challenges. Traditional methods usually suffer from insufficient data, labels, and poor transferability. In this paper, we propose a novel zero-/few-shot learning approach, SEM-CLIP 2.0, based on our SEM-CLIP framework [1], for accurate defect classification and segmentation. By leveraging learnable text prompts, SEM-CLIP 2.0 fully utilizes large-scale external data sources to facilitate robust cross-domain knowledge transfer. It also customizes the Contrastive Language-Image Pretraining (CLIP) model to better focus on defect areas and minimize background distractions, thereby enhancing segmentation accuracy. Furthermore, our approach employs discriminative feature learning for open-set classification, facilitating more effective defect classification and enabling the detection of unknown defect categories. By alleviating the need for manual prompt engineering and extensive data annotation, SEM-CLIP 2.0 significantly reduces labor costs. Extensive experimental results demonstrate that our model achieves impressive classification and segmentation results under zero-/few-shot learning scenarios.