Your Ride, Your Rules:Psychology and Cognition Enabled Automated Driving Systems

Abstract Current autonomous vehicles (AVs) primarily respond to external traffic conditions and treat humans as passive occupants, limiting their ability to personalize driving behavior and handle ambiguous scenarios that could benefit from occupant input. This study proposes PACE-ADS (Psychology and Cognition Enabled Automated Driving Systems), a human-centered autonomy framework that enables AVs to interpret and respond to both external traffic conditions and internal occupant states. PACE-ADS employs three collaborating foundation model agents: the Driver Agent interprets the external environment; the Psychologist Agent infers occupant state from passive psychological signals (e.g., facial expressions) and interprets active cognitive inputs (e.g., verbal commands); and the Coordinator Agent integrates these inputs to generate high-level driving behavior decisions. Operating at the low-frequency semantic planning layer, PACE-ADS complements existing AV stacks and activates selectively in response to changes in occupant state, cognitive instructions, or vehicle immobilization. Closed-loop CARLA simulations under prescribed occupant-state trajectories show that PACE-ADS adapts driving behavior along kinematic dimensions associated with ride comfort and supports recovery from operational immobilization through self-reasoning and occupant-in-the-loop guidance.