A comprehensive evaluation system for vocational skills supported by multimodal graph neural networks

Precision and adaptability are crucial for evaluating vocational skill development, enabling individuals to prepare for the evolving demands of the workforce. Most systems employ static measures and overlook multimodal learner data, thereby limiting their efficacy. This proposed methodology introduces SkillNet-MMG, a Multimodal Graph Neural Network (GNN)-based vocational skill acquisition evaluation system, is introduced in this study. To accurately classify learners into efficiency tiers using physiological, behavioral, and contextual indicators from 1,000 training sessions. Build a heterogeneous graph with over 3,000 nodes and 7,500 edges to represent users, tasks, feedback, and adjustments. Each node has 14 multimodal features, including hand movement precision, eye-tracking attention, stress, response time, and prior skill levels. To learn task-user interactions and attention weights across modalities, a bespoke MHAGNN is used.SkillNet-MMG predicts skill acquisition efficiency more accurately than traditional GNNs (– 8.4%) and classical ML models (– 12.1%), achieving a classification accuracy of 92.6%. Eye-tracking focus (24.3%) and biometric stress (21.7%) account for the majority of attention weights. Analysis reveals a 15.5% increase in the early identification of low-efficiency learners and a 17.2% increase in the effectiveness of interventions. Ultimately, SkillNet-MMG offers a robust, data-driven framework for vocational skill assessment.