Recognition of Normal and Unusual Activities: Using a Two Stage Classification Model with Voting on 2D Keypoint Data

Abstract To address the challenge of distinguishing between normal and unusual behaviors especially in the context of labor shortages, workload reduction, and the need for more efficient alternatives to traditional observation methods, this study proposes an effective processing pipeline for Human Action Recognition (HAR) based on 2D keypoint data extracted from pose estimation models. A two stage hierarchical classification architecture is developed to enhance behavior recognition performance. The dataset consists of 30 fps videos of five individuals performing eight actions: four normal ( Walking, Using phone, Eating snacks, Sitting quietly ) and four unusual ( Head banging, Throwing things, Attacking, Biting nails ). Pose data was extracted using YOLOv7 to obtain 17 keypoints representing major body parts. Each subject performed all actions with uncontrolled durations, allowing for natural behavioral variations. We segmented the keypoint time series using a 3 second sliding window with a 1.5 second overlap. From each segment, over 700 features were extracted, including motion dynamics, posture information, frequency domain descriptors, and time domain statistics such as RMS, variance, and autocorrelation. In addition to training traditional machine learning models including Random Forest (RF), XGBoost (Xb), and k Nearest Neighbors (KNN), we implemented a two stage classification architecture with a voting ensemble strategy to improve recognition accuracy and robustness. We adopted Leave One Person Out (LOPO) cross validation to ensure model generalizability across subjects. In the first stage, we tested two classification strategies focused on distinguishing between sitting and standing postures. In the second stage, specific actions were classified using a majority voting ensemble combining RF, XGBoost, and KNN models. The system was evaluated using five LOPO repetitions to ensure stability. Results from the first stage showed high accuracy: 87.2% for standing behaviors and 70.3% for sitting behaviors. The second stage achieved an average accuracy of 75.6% and a macro F1 score of 76.25%, with a standard deviation of ±0.3%. These findings underscore the stability and effectiveness of our approach in distinguishing between normal and unusual behaviors using 2D pose data, demonstrating a promising, flexible, and easily deployable solution to enhance the efficiency of healthcare and social service systems.