Head-mounted first-person view camera-based worker action recognition using visual SLAM

Since construction workforce management is crucial for productivity and safety, recognizing workers’ actions on site has become important. Researchers have explored various sensors, including cameras, for worker action recognition. To directly capture egocentric body and head movements from the worker’s perspective, this paper proposes an action recognition approach using a head-mounted first-person view (FPV) camera. The proposed approach defines and analyzes seven body movement types and five head movement types, supported by visual simultaneous localization and mapping (SLAM)-based FPV camera pose estimation and 3D head pose–based action recognition. It achieved accuracy of 94.52% (body: 97.08%, head: 92.83%) indoors and 89.31% (body: 90.62%, head: 87.99%) outdoors, with macro F1-scores of 87.58% (body: 95.04%, head: 80.13%) and 78.14% (body: 82.53%, head: 73.75%), respectively. These results indicate that head-mounted cameras can be effectively employed for action recognition by capturing the worker’s field of view and mitigating limitations inherent in third-person monitoring.