Detecting Unusual Physical Activities Using LSTM

Abstract Timely detection of unusual activities in developmental disability support facilities is vital for preventing accidents and ensuring resident safety. However, conventional video-based surveillance systems raise privacy concerns and often require extensive labeled data. This paper proposes a privacy-preserving and data-efficient method using Long Short-Term Memory (LSTM) networks to recognize human activities from 2D skeletal pose sequences. Our approach employs an LSTM backbone to first recognize activity sequences and then classify them as normal or unusual using a densely connected neural network head. By relying solely on pose keypoints, the system ensures anonymity while retaining motion-related information necessary for temporal modeling. The model is evaluated using a Leave-One-Subject-Out (LOSO) strategy, demonstrating its ability to generalize with the help of a dropout system, across individuals. Experimental results indicate that the proposed method achieves promising performance in distinguishing unusual behaviors with limited data, making it suitable for sensitive environments such as care facilities. Our code is available here.