LLMs in the Loop: A Survey of Language-Driven Driver Monitoring and Assistance Systems

Read full text
Authors: Vanchha Chandrayan, Ignacio Alvarez
Cited by: 1
References: 64

Abstract

In recent years we have seen large language models (LLMs) demonstrating robust reasoning capabilities comparable to human performance. This makes them increasingly appealing for driver assistance, where adaptation to dynamic human context is essential. Yet, research in this area remains fragmented, often focusing on isolated applications, lacking utilization of LLM's full potential to deliver integrated, context-specific support and action. This survey synthesizes recent advancements in LLM-driven occupant monitoring systems, focusing on their capabilities for interpreting driver states and acting appropriately, enabling a new generation of intelligent driver assistance. We critically examine pioneering frameworks, benchmarks, and foundational datasets that employ techniques like reasoning chains, multimodality, and human-in-the-loop feedback to create personalized and safe driving experiences. We lay out the current trends, limitations, and emerging patterns, in addition to a novel human-centered evaluation of the field, providing researchers with a roadmap towards transparent and trustworthy in-cabin systems that bridge safety with driver experience.

Fields of Study & Subjects

Human-Automation Interaction and Safety Multimodal Machine Learning Applications Sleep and Work-Related Fatigue

Publication details

Journal / source
Sensors
Publisher
MDPI AG
Volume
26
Issue
9
Pages
2870–2870
Publication date
4 May 2026
Language
EN