Automated worker action recognition in construction videos via domain-adapted vision-language fusion

Effective recognition of worker actions on construction sites is critical for monitoring unsafe behaviors, ensuring workflow compliance, and analyzing productivity. Although computer vision has surpassed traditional manual observations by leveraging spatiotemporal features in videos, persistent challenges include recognition accuracy, generalization capability, and semantic comprehension in complex environments. Therefore, this paper proposes a domain-adapted vision-language fusion (DA-VLF) method for worker action recognition in construction videos, which utilizes the spatiotemporal visual-semantic understanding capabilities of pre-trained vision-language models. Validation on the COA (centering on operational activities) and CMA (focusing on unsafe behaviors) datasets shows that DA-VLF achieves average precisions of 95.39% and 95.44%, outperforming existing leading vision-based methods by 8.73% and 8.84%, respectively. The approach also demonstrates robust few-shot learning performance. This paper offers a perspective for worker action recognition, indicating that domain-adapted large multimodal models provide advantages for construction management and potential for practical application in video-based recognition tasks.