From Digital Twins to Immersive Manufacturing: XR and Gesture‐Based Control for Enhancing Human–Robot Collaboration

ABSTRACT Digital twins (DTs) and immersive extended reality (XR) interfaces offer new opportunities for intuitive, human‐centred interaction in smart manufacturing. However, implementations often lack rigorous validation, quantitative performance analysis and assessment of scalability and robustness, especially in small‐scale or resource‐constrained manufacturing settings. This work proposes a modular metaverse framework integrating ROS2, Unity and Meta Quest devices to develop interactive, bidirectional DTs enhanced with gesture‐based and mixed reality (MR) control. The framework is demonstrated through a lab‐scale case study combining a robotic Wire Arc Additive Manufacturing (WAAM) system and a collaborative robot‐based laser‐cleaning station, showing broad applicability across industrial robotics. To evaluate usability, a preliminary user study with 17 participants was conducted, comparing a standard teach pendant with the proposed XR interface for a tool‐inspection task. Results show an 80% reduction in programming time and significant decreases in perceived workload. Measured end‐to‐end gesture‐to‐action latency ranged from 430 to 450 ms, representing suitable timeframe for high‐level interaction and task initiation. The study provides empirical user‐centred evidence for metaverse‐enabled interaction and discusses scalability, latency and industrial constraints. Aligned with the human‐centric intelligence principles of Industry 5.0, the proposed approach improves accessibility, operator well‐being and adaptability, contributing towards more inclusive and robust smart manufacturing systems.