Advancing Virtual Lab Immersion Through YOLO-Powered Gesture Recognition in Natural HCI
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
This study presents a gesture-based human-computer interaction (HCI) system for virtual engineering education, leveraging a modified YOLO-NAS Pose model for real-time hand gesture recognition. Unlike traditional input methods (mouse and keyboard) and commercially available virtual interfaces, this approach provides natural interaction within a screen-based electrical power lab simulation. The proposed system was benchmarked against MediaPipe, OpenPose, and CNN-based models, demonstrating superior inference speed (~2.35ms per frame on NVIDIA T4) with competitive accuracy (96.2%), making it highly suitable for real-time educational applications.
Fields of Study & Subjects
Hand Gesture Recognition Systems
Interactive and Immersive Displays
Gaze Tracking and Assistive Technology
Publication details
- Journal / source
- The ASEE Computers in Education (CoED) Journal
- Volume
- 15
- Issue
- 1
- Publication date
- 1 February 2026
- Language
- EN