Multi-Perceptual Framework for Anomaly Detection

Anomaly detection is an imperative but challenging task in intelligent surveillance due to the scarcity and irregularity. Most existing unsupervised methods mainly detect anomalies through frame prediction, but their discriminative distance is limited. Meanwhile, some issues such as lack of semantic correlation and less attention to inter-frame relations are also existed, which are important to anomaly detection. To solve these issues, a multi-perceptual framework (MPFork) using textual and visual information is proposed, which combines the ideas of fusion prediction, semantic perception, and temporal discrimination. Specifically, features fusion prediction module is devised to concentrate on representing the normality and repressing anomalies by fusing heterogeneous features. Meanwhile, an image-text semantic perception module is proposed for semantic consistency by correlating the visual with textual features, where the local and global inferences are adopted to perceive local anomalies, simultaneously measuring the belongingness of each object in global frame. Moreover, a temporal attention discrimination module is developed to capture the inter-frame relations and identify pseudo-abnormal sequences from normal ones in temporal direction. Extensive experiments show the proposed MPFork outshines most of state-of-the-art methods, which indicates the efficacy of our MPFork for anomaly detection.