Learning Decision-Making Model for Autonomous Overtaking via Forward-Backward View Pretraining Policy Optimization

Autonomous overtaking is a critical and challenging maneuver in self-driving, especially when both longitudinal (speed and acceleration) and lateral (steering) controls operate in continuous action spaces. The main difficulty arises from the inability to accurately assess the delayed positive impacts of early actions, which often leads to such actions being misclassified as suboptimal. This issue poses a significant obstacle to learning effective overtaking policies in continuous control domains. To tackle these issues, we propose a novel deep reinforcement learning algorithm called the Forward-Backward View Pretraining Policy Optimization Algorithm (FBPO). FBPO first utilizes the Decision Transformer to learn high-quality trajectory knowledge and use its Causal Transformer structure to identify critical state importance, leading to a new loss function that guides the agent toward meaningful actions and accurate value estimation. We compare FBPO with several state-of-the-art methods. Simulation results demonstrate that our approach significantly improves both training efficiency and overall performance, achieving a 16.4% to 23.8% higher overtaking success rate and reducing autonomous driving task completion time by 4.5% to 44.7% compared to other methods.