Research on vehicle platoon keeping based on double deep Q-network

Multi-vehicle formation control holds significant application value in intelligent transportation and unmanned system collaboration. However, due to factors such as inter-vehicle coupling, nonlinear dynamics, and environmental uncertainties, traditional methods based on precise models often require cumbersome modeling and tuning under complex working conditions, with limited robustness and generalization. To address this, this paper constructs a learning-based formation control framework based on the Double Deep Q Network (DDQN).With relative poses and formation errors as the core observations, a discrete action set is used to output longitudinal and lateral control commands. Experience replay and target networks are introduced to stabilize the training, and the decoupling of action selection and evaluation is adopted to mitigate overestimation bias. Under the task-driven reward design (considering both error convergence and control smoothness), the proposed method achieves stable formation maintenance in a reproducible simulation environment. This paper establishes a multi-dimensional index system, including the root mean square error of planar position (RMSEXY), the time to reach the error threshold (Tε), and the control cost (Ju),and provides trajectory, error, and reward curves, as well as formation evolution, to support the analysis. Experimental results show that the learned strategy can converge to the desired relative poses under various initial conditions, demonstrating good steady-state accuracy and control smoothness.