Adaptive Trust Gated Fusion of RGB and Depth Modalities with Entangled Spatiotemporal Attention for Self-Supervised Video Anomaly Detection

Read full text Summarise with VIRAT
Authors: Dr. P. Krishna Priya Sindhu T
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Anomaly detection in surveillance video is a vital task for ensuring public safety, particularly in crowded and dynamic environments. This study introduces a self-supervised spatiotemporal framework that integrates cross-modal gated fusion and spatiotemporal attention to predict future frames in video sequences. By leveraging both RGB and depth modalities, the model learns normal temporal dynamics and identifies anomalies through reconstruction error. The gated fusion mechanism adaptively balances the reliability of RGB and depth features, while the spatiotemporal attention module highlights motion cues and suppresses background clutter. Structural similarity (SSIM) loss is employed to preserve human silhouettes and spatial structures, complemented by mean squared error (MSE) for depth reconstruction. The proposed framework was evaluated on three benchmark datasets — UCSD Pedestrian, CUHK Avenue, and ShanghaiTech — achieving superior performance compared to baseline autoencoder and prediction models. Results demonstrate that the model effectively detects diverse anomalies such as bicycles, running, and crowd disturbances, even under challenging conditions of occlusion and illumination variation. This work underscores the importance of multimodal fusion and attention mechanisms in advancing robust surveillance anomaly detection

Fields of Study & Subjects

Anomaly Detection Techniques and Applications Video Surveillance and Tracking Methods Human Pose and Action Recognition

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
25 March 2026
Language
EN