MU-Net-optLSTM: Two-Stream Spatial–Temporal Feature Extraction and Classification Architecture for Automatic Monitoring of Crowded Art Museums

Abstract Networked cameras that continuously capture video data have generated a high demand for hybrid edge-to-cloud servers that can process live videos in real time. The environment of art museums is rarely studied, but visual analysis is an important factor in categorizing and distinguishing individuals and crowds through smart surveillance systems. This paper demonstrates how video surveillance data from art museums can be analyzed to identify abnormal behavior using an innovative deep learning framework. To enhance the extracted features, a spatial feature extraction method based on the U-Net architecture is applied, along with the encoder component of the proposed approach, MobileNetV2. Additionally, we propose an improved Long-Short-Term Memory (LSTM) algorithm for extracting temporal features. Optical flow enhances surveillance in art museums by tracking individuals and crowds. Our approach yields an average accuracy of 97.67±1.23% when applied to a collection of video datasets. Using U-Net, MobileNetV2, and optimized LSTM algorithms, the model recognizes patterns in video data, such as crowd motion in museums. Consequently, this methodology generates reliable results as well as being computationally efficient. Compared to the state-of-the-art, the proposed method is more comprehensive and generalizable for analyzing atypical museum visitor behavior.