Deep reinforcement learning for CAV control in signalized intersections: a DDPG approach

Intersections serve as critical nodes in the urban road traffic network. However, their inefficient operation has emerged as a significant challenge to traffic system that requires urgent solutions. In this regard, this paper proposes a vehicle control approach based on Deep Deterministic Policy Gradient (DDPG) algorithm for urban signalized intersections in connected and autonomous environment. Firstly, the passage of Connected and Autonomous Vehicles (CAV) through intersections is described as a Markov Decision Process (MDP). The state space comprises four dimensional features like vehicle position, speed and so on. The action space consists of continuous acceleration and steering angle. The reward function is designed from three aspects of safety, traffic efficiency and the degree of passage at the intersection. Secondly, the DDPG algorithm is implemented with an optimized Actor-Critic network structure to enhance convergence. Finally, the INTERACTION dataset is accessed to verify the proposed framework. Experimental results show that the proposed framework achieves stable convergence. As the number of training steps increases, the algorithm can effectively explore the penalty minimization strategy and accurately evaluate the strategy reward. This study raises a Deep Reinforcement Learning (DRL) framework for CAV motion control, providing references for related researches such as CAV cooperative driving and intelligent traffic management.