A Multimodal Intelligent System for Human Digital Twin Simulation with Continuous Kinematic Data Tracking, Biometric Prognosis, and Cognitive State Feedback in Industrial Environments

The concept of human digital twin (HDT) offers transformative potential in dealing with the growing complexity of industrial tasks. By creating a virtual model of the human worker, the HDT system can help reduce occupational hazards, optimize the work environment, and make better informed decisions. Although the foundational technology has existed for years, the real‐world application of digital twins is limited to the asset, component, or system unit. State‐of‐the‐art research in the HDT technology remains largely theoretical, with little to no functional real‐world applications in industrial settings. To bridge the gap, this article introduces a comprehensive HDT framework capable of simulating industrial tasks, tracking multimodal kinetic and biometric data, and delivering real‐time cognitive state analysis. The system integrates a Physics‐Informed Neural Network for movement simulation, eXtreme Gradient Boosting and Random Forest ensembles for biometric prediction, and a Bidirectional Long–Short‐Term Memory network for fatigue detection and fine motor control of finger movements. An innovative hardware interface using dielectric elastomer actuators provides haptic and acoustic feedback to the user. Additionally, the system's portability is enhanced through a custom‐developed smart application. The system is evaluated for functionality, accuracy, and robustness via comprehensive quantitative‐qualitative assessments and lab experiments.