Context-Aware Ultrasound Diagnosis for Robust Few-Shot Fine-Grained Anomaly Classification and Pixel-Level Localization

Ultrasound imaging plays a vital role in clinical diagnostics, yet its interpretation remains highly subjective and varies across observers, particularly when identifying subtle or finegrained anomalies. While automated systems have shown promise, they often face challenges in generalizing across domains and learning effectively from limited medical data. To address these issues, we introduce CAUD (Context-Aware Ultrasound Diagnosis), a CLIP-based few-shot adaptation framework that enhances context understanding for more reliable anomaly classification and precise lesion localization. CAUD integrates four key components: Dynamic Contextual Prompting (DCP) for adaptive text modulation, a Multi-Scale Feature Fusion Adapter (MSFA) for visual feature integration, Adaptive Memory Bank Enhancement (AMBE) for improved feature discriminability, and Context-Guided Patch-Text Interaction (CGPTI) for aligning local image cues with global semantics. The model is optimized through a multi-task learning strategy to promote synergy between classification and localization tasks. Experiments on multiple ultrasound datasets demonstrate that CAUD achieves robust generalization and superior accuracy in both image-level diagnosis and pixel-level localization. Human evaluations further confirm its potential to assist clinicians by improving diagnostic confidence and efficiency.