Fusing enhanced YOLO and knowledge graph-based large language models for automatic risk perception in tower crane operations

Tower crane operations in construction are inherently hazardous due to complex and dynamic site environments. Enhancing operators' perceptual and cognitive capabilities is essential for ensuring safety and improving situational awareness. This paper presents an integrated framework that combines an improved YOLOv8 model with a Knowledge Graph (KG)-enhanced large language model to achieve proactive and intelligent safety management. The improved YOLOv8 incorporates attention-based optimization to improve detection accuracy for small targets in tower crane perspectives. A domain-specific safety KG is constructed to represent critical entities, relationships, and operational contexts, and is aligned with a fine-tuned GPT model, enabling semantic reasoning and context-aware hazard interpretation. The integrated system links visual perception with structured knowledge reasoning to provide real-time and interpretable safety feedback. This approach enhances the perception, understanding, and decision-making capabilities of tower crane operators, transforming safety management from reactive monitoring to proactive and intelligent control in complex construction environments.