Longitudinal and Lateral Control for Discretionary Lane Change via Safe Multi-Agent Reinforcement Learning

Lane-change automation is a core technology for autonomous driving in highway environments. To address the challenges of enhancing lateral lane-keeping ability, enforcing safety constraints, and ensuring control smoothness during the lane-change process, this study introduces a safe multi-agent reinforcement learning (MARL) framework for lane change. First, to enhance lateral lane-keeping ability, the solution employs a decoupled lateral and longitudinal control framework, constructing a distributed multi-agent system where two agents separately handle lateral and longitudinal control. Second, to guarantee safety, the framework integrates safety constraints into the multi-objective optimization through the Lagrangian method. Finally, the framework is built upon a soft actor-critic (SAC) algorithm to ensure control smoothness. The simulation results demonstrate that, compared with single-agent reinforcement learning, our safe MARL solution reduces lateral lane-change overshoot by 70% and offset by 68%, and achieves zero collision. Additionally, the results reveal that SAC-based algorithms provide smoother control than twin delayed deep deterministic policy gradient-based algorithms.