Multimodal attention-guiding takeover alerts in autonomous driving: Comparing visual, auditory, and combined modalities

In the current phase of autonomous driving, drivers are still required to take over the vehicle when the driving environment exceeds the operating design domain of the system or when a failure occurs. This study focuses on the impact of visual-auditory attention-guiding takeover alerts on driver performance and subjective user experience during emergency takeovers. Twenty-eight young participants with advanced driver assistance systems (ADAS) experience were recruited for the experiment. This study involved four combinations of multimodal attention-guiding stimuli, and participants were asked to perform non-driving-related tasks (NDRTs) and takeover tasks in simulated scenarios. Takeover reaction times, eye-tracking data, situational awareness, and subjective evaluations were recorded under four stimulus conditions (Baseline, V-AG, A-AG, and VA-AG). This study found that attention-guiding takeover alerts significantly improved driver performance, situational awareness, comfort, and satisfaction. Multimodal visual-auditory attention-guiding takeover alerts outperformed unimodal alerts in enhancing situational awareness and the overall subjective takeover experience. However, auditory-only attention-guiding alerts were more effective in significantly shortening the time required for drivers to complete the takeover task.