Pose-Driven Anomaly Detection in Human Behavior with Machine Learning

Abstract Facilities offering care support to disabled individuals are often under-staffed, which results in a lack of appropriate care for the people involved. For the case of disabled individuals with developmental difficulties, it is important to perform visual observation of unusual behavior so as to perform remedial intervention before the individuals can put themselves or others in harm. However, this is not always possible due to limited staffing and resources. Technology can play a vital role in alleviating this situation; visual information can be used to detect unusual behaviour in people and so as to notify the caregiver immediately. This paper proposes a method for the automated detection of unusual behavior in videos by extracting spectral, statistical, time-domain, and fractal features from derived pose data combined with multiple machine learning classifiers for classification. Experiments were conducted using a leave-one-patient-out (LOPO) cross-validation setup. It was found that the XGBoost classifier combined with Synthetic Minority Oversampling Technique (SMOTE) achieved the highest F1 macro score of 0.6282 and balanced accuracy of 0.6264 for the multiactivity classification task. Moreover, statistical analysis using ANOVA revealed that the peak to peak distance of the left elbow left hip angle was the most relevant feature, along with parameters resulting from the Teager-Kaiser energy operator. These results suggest that combining state of the art signal processing approaches with machine learning methods is a powerful tool for unusual behaviour detection systems.