Learning Local Semantic Signals and Inter-Class Discrepancy for Weakly Supervised Video Anomaly Detection

Weakly supervised video anomaly detection (WSVAD) aims to retrieve temporal intervals containing anomalous events within untrimmed videos, leveraging video-level annotations. The existing methodologies exhibit unsatisfactory performance, primarily attributed to the absence of frame-level annotations. To alleviate this issue, we argue that representations exhibiting high similarity within local regions can offer dependable semantic knowledge. With this insight, we introduce a novel pseudo-label optimization mechanism grounded in the similarity of local features, specifically designed to direct framelevel anomaly predictions. Besides, we have noticed that the majority of videos utilized for anomaly detection originate from surveillance scenarios and are captured by stationary cameras. Consequently, the frames within a given video share consistent background properties. Motivated by this observation, we propose an intra-frame background-foreground separation strategy for each frame to extract discriminative visual representations. Given the considerable similarity in the backgrounds of most frames, the distinctions among the features of diverse frames become subtly indistinct. As a result, to ensure the rationality of predictions, we encourage maximizing the inter-class variance between normal and abnormal frames. Extensive experiments and ablation studies, encompassing both coarse-grained and fine-grained, have been conducted on XD-Violence, UCF-Crime, and ShanghaiTech benchmark datasets. The results demonstrate that our proposed method achieves substantial improvements, outperforming the current state-of-the-art approaches.