De-FuSE -Disentangled Frequency and Semantic Embeddings for Unified Anomaly Detection

Vision anomaly detection (VAD) is an open research problem due to the diversity of application areas and the magnitude of underlying challenges. Among the key solutions in literature, unified anomaly detection (UAD) aims to learn a single generalized model for heterogeneous anomalies. To realize this scalability, supervised learning approaches are constrained by limited data availability. Therefore, self-supervised training methods are used since they rely on frozen foundation models like DINOv2 and CLIP. However, due to limited anomalous samples in real-world datasets, training suffers from an identity shortcut, where powerful semantic priors trivially reconstruct the input regardless of anomalies. To address these challenges, we propose DeFuSE , a novel architecture that explicitly disentangles semantic representations from high-frequency residuals. Our method introduces a semanticguided cross-frequency attention mechanism in the hybrid decoder, which acts as a conditional gate to inject frequency priors with semantically aligned queries. We further enforce spectral fidelity through a composite loss function combining hard-mining strategies with focal frequency loss. Extensive experimentation on the MPDD, VisA, and MVTec-AD benchmarks demonstrates the superiority of our approach for VAD. In the challenging unified setting, DeFuSE outperforms all competing baselines by an average of 1.03% in I-AUROC across the complex VisA and MVTec-AD benchmarks. With just 59.1M trainable parameters, DeFuSE delivers a massive 31.75% reduction in model complexity compared to baseline unified frameworks. The code is available at: https://github.com/mariam-162/De-FuSE