Advancing Virtual Lab Immersion Through YOLO-Powered Gesture Recognition in Natural HCI

View at publisher Summarise with VIRAT
Authors: Zhou Zhang, Momina Liaqat Ali
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This study presents a gesture-based human-computer interaction (HCI) system for virtual engineering education, leveraging a modified YOLO-NAS Pose model for real-time hand gesture recognition. Unlike traditional input methods (mouse and keyboard) and commercially available virtual interfaces, this approach provides natural interaction within a screen-based electrical power lab simulation. The proposed system was benchmarked against MediaPipe, OpenPose, and CNN-based models, demonstrating superior inference speed (~2.35ms per frame on NVIDIA T4) with competitive accuracy (96.2%), making it highly suitable for real-time educational applications.

Fields of Study & Subjects

Hand Gesture Recognition Systems Interactive and Immersive Displays Gaze Tracking and Assistive Technology

Publication details

Journal / source
The ASEE Computers in Education (CoED) Journal
Volume
15
Issue
1
Publication date
1 February 2026
Language
EN