Autonomous Collision Avoidance Decision-Making Method for Multiple Marine Surface Ships Based on Deep Reinforcement Learning

In the field of maritime transportation, autonomous collision avoidance is extraordinarily significant in ensuring the safety of navigation. This study proposes a multiple marine autonomous surface ships (MASSs) collision avoidance decision-making method based on improved rollback restriction and gate recurrent unit authentic boundary proximal policy optimization (RG-ABPPO) algorithm. The novelty of the study is: 1) Using the new rollback restriction solves the problem of excessive clipping of PPO algorithm, thereby improving the learning performance; 2) developing an improved gate recurrent unit (GRU) network structure based on auxiliary loss function, which is able to improve the computational efficiency; 3) designing a group of complete reward functions that can ensure the navigation safety of MASSs in the face of extremely complex encounter situations. In the simulation, the analysis of the loss function, average reward curve, collision avoidance success rate, computational efficiency, and navigation time demonstrates that the RG-ABPPO trained network model exhibits superior performance in collision avoidance. Finally, a physical experiment in the real world demonstrates the potential of the proposed method for engineering applications.