An Integrated Open-Source Framework for Automated Analysis of Driver Activity, Glance, and Vehicle Automation Mode

OCCUPATIONAL APPLICATIONAn efficient software framework for analyzing driver behavior from videos is presented. The system automatically detects driver activities, such as phone use, identifies glance patterns, and determines the vehicle’s automation state, achieving over 96% accuracy for glance and mode detection. For transportation and safety researchers, vehicle engineers, and fleet safety managers, this tool drastically reduces the time and cost of manual video analysis by over 90%. This efficiency enables large-scale naturalistic driving studies, the objective quantification of driver distraction and inattention in commercial and passenger fleets, and the safety evaluation of new in-vehicle information systems and advanced driver-assistance systems. The framework provides an accessible method for understanding real-world driver behavior, ultimately helping to design safer vehicle systems and inform evidence-based road safety policies.TECHNICAL ABSTRACT Background The increasing prevalence of Level 2 (L2) automated driving systems introduces complex human-machine interactions, making the analysis of driver behavior critical for safety. Traditional research methods rely on manual video annotation, a time-consuming and resource-intensive process that limits the scale of naturalistic driving studies and creates a bottleneck for research.Purpose This study aimed to develop and validate a robust, open-source, and integrated framework for the efficient, multi-faceted analysis of driver behavior using standard in-vehicle video recordings, thereby making large-scale analysis more accessible.Methods The framework integrates several pre-trained computer vision models. It uses YOLOEv8 for object detection and MediaPipe for skeletal and facial landmark tracking to classify physical secondary tasks (e.g., phone use, consuming). Driver attentional state is determined by classifying glance zones based on head pose and eye gaze estimation. Driving mode (Manual vs. Automated) is detected using GPU-accelerated OpenCV to analyze dashboard iconography. The entire process is managed through a custom graphical user interface with comprehensive debugging tools.Results Validation against manually annotated data confirmed the framework’s high accuracy. Activity detection achieved F1-scores of 93.3% for consuming, 90.9% for browsing on a phone, and 87.5% for talking on a phone. Driving mode and glance allocation detection achieved mean accuracies of 97.4 and 96.5%, respectively.Conclusion The proposed framework is a validated, efficient, and replicable alternative to manual coding. By significantly reducing analytical workload, it provides an accessible tool for conducting scalable research into driver distraction, attention, and human-automation interaction.