Human pose estimation based engagement detection in E-learning: a hybrid approach

In E-learning platforms, pose estimation offers a privacy-preserving approach for monitoring student engagement. This paper presents a hybrid pose-based engagement detection framework using deep architecture Movenet and machine learning techniques. The feature extraction phase uses skeletal features extracted using MobileNetV2 and feature pyramid network of Movenet. Binary and multi-level classification experiments demonstrate the effectiveness of ensemble learning methods, with Random Forest and XGBoost achieving superior performance. The study further highlights the suitability of the One-vs-Rest strategy for handling class imbalance in multi-level engagement classification. Multi-level engagement classification on the WACV2016 dataset, characterized by majority-to-minority class ratio of 5.45:1, demonstrated notable performance improvements with the One-vs-Rest strategy, achieving accuracies of 71.6% and 71.4% with XGBoost and Random Forest. We conducted binary classification on a balanced custom dataset comprising 1324 upper-body pose images, achieving accuracies of 97.8% and 97.7% using Random Forest and XGBoost, respectively. Model performance was evaluated using accuracy, precision, recall, F1-score, and ROC curve metrics. Comparative analysis on the WACV2016 dataset indicates that the proposed approach outperforms baseline models and established deep learning architectures. Overall, this work underscores the potential of integrating deep pose estimation with classical machine learning for real-time, privacy-aware student engagement monitoring in e-learning environments.