Not Only Detection But Also Explain: An Interpretable Multimodal System for Anomaly Detection

Anomaly detection is often formulated as a Multiple Instance Learning (MIL) problem, where an anomaly detector learns to generate frame-level anomaly scores under multi-modal supervision. However, most previous studies have two drawbacks: 1) the feature fusion between attributes overlooks the model’s interpretability; and 2) the training strategy based on pre-trained models leads to excessively long training times. To address these issues, this paper proposes anInterpretableMulti-modalAnomalyDetectionSystem called IMADS. The proposed IMADS abandons the traditional multi-modal fusion approach and adopts single-modal training with late-stage decision fusion optimized by Bayesian techniques to resolve the interpretability problem caused by traditional feature fusion. Moreover, the proposed IMADS introduces a transfer learning method from frame-level to video-level, which essentially adds a temporal dimension to the 2D-to-3D conversion by randomizing parameters along the time dimension during training. This approach not only improves training efficiency but also ensures the model’s interpretability. Experimental results demonstrate that the proposed IMADS significantly outperforms existing methods in terms of performance, proving its innovativeness and effectiveness.