Visualizing Eye-Tracking Dynamics for Predicting Human Error in Virtual Reality

Human error remains a pervasive risk across safety-critical domains, motivating research on predictive approaches that enable selective automation and adaptive training. This paper presents a longitudinal Virtual Reality (VR) study for cognitive failure prediction, collected over 18 months from a participant solving mental arithmetic problems. We propose a spatio-temporal representation that transforms eye-tracking features into compact visualizations, and evaluates its effectiveness using convolutional, recurrent, and fusion neural models. These models leverage both spatial and temporal dynamics, achieving 87% accuracy in identifying missed responses. However, distinguishing wrong from correct answers proved more difficult, likely due to similar ocular behavior during confident but incorrect responses and the impact of class imbalance. Overall, the results highlight the feasibility of personalized AI systems trained on rich within-subject data, and position VR combined with deep learning as a platform for real-time monitoring and adaptive training to reduce human error.