Enhancing Vision-Language Model for Construction Safety Inspection via Visually Grounded Reasoning and Reinforcement Learning

Ensuring automatic safety inspection on dynamic construction sites requires reliable multi-entity visual interpretation and rule-based compliance checking, which conventional computer vision systems struggle to achieve due to limited semantic reasoning. Although modern vision-language models (VLMs) support end-to-end multimodal reasoning, generic VLMs lack fine-grained visual grounding for safety-critical inspection. This study proposes a transparent, object-centric VLM framework that (1) formalizes four visually grounded reasoning strategies-Enumerate Anchor Entities, Key-feature Identification, Sequential Inspection and Explicit Visual Grounding to guide VLMs toward evidence-based inspection; (2) auto-generates a self-instruct visual-grounding dataset for cold-start supervised fine-tuning, enabling internalization of inspection behaviors; (3) optimizes rule-level accuracy using Group Relative Policy Optimization with designed ComplianceReward function that reinforces reasoning trajectories leading to correct answers. Trained on Qwen3-VL-8B, the proposed agent achieved 0.916 violation recall and 0.904 overall accuracy in real-site evaluation, outperforming baseline models. The framework delivers scalable, interpretable, and domain-aligned multimodal reasoning for automated construction safety inspection.