Leave-One-Subject-Out (LOSO)-Validated Ensemble of Bi-LSTM and Transformer Models for Unusual Activity Detection

Abstract The accurate and timely detection of unusual behaviors in individuals with developmental disabilities is crucial for ensuring their safety and well-being, yet traditional human observation methods are labor-intensive and prone to error. This research addresses the coding challenge as a part of the International Symposium of Applied Science 2025 with the concept “Watch the pose, catch the risk: Unusual activity recognition for developmental disability support” by developing and evaluating deep learning models capable of identifying unusual activities solely from pose estimation data. We propose and compare three architectures: a Bidirectional Long Short-Term Memory (Bi-LSTM) network, a Hybrid model combining Bi-LSTM with a Transformer, and an Ensemble model that integrates the predictions of both using soft voting. To ensure a robust and realistic assessment of model performance, this study employs a rigorous Leave-One-Subject-Out (LOSO) cross-validation strategy. The models were trained on a specialized dataset of eight simulated activities, including four typical and four atypical behaviors, designed in collaboration with a caregiving facility to ensure clinical relevance. Initial results showed that the standalone Bi-LSTM model demonstrated the strongest individual performance, though analysis of the confusion matrices revealed difficulties in distinguishing between subtle, low-motion activities such as ‘Sitting quietly’ and ‘Using phone’. To improve upon these results, a detailed analysis of different weighting schemes for the soft-voting ensemble was conducted. This optimization revealed that an equal weighting (0.50/0.50) after tuning produced an F1 of 0.7323, which surpasses the untuned models. This study validates that while standalone deep learning models are effective, a carefully weighted ensemble strategy provides a significant improvement, offering a more accurate and reliable solution for real-world unusual activity detection.