Autonomous navigation and collision avoidance for unmanned sailboats in complex environments based on deep reinforcement learning

Abstract Autonomous navigation of unmanned sailboats in real-world oceans faces significant challenges due to the strong coupling among stochastic winds, irregular waves, and time-varying currents. Existing approaches often rely on simplified kinematic models or idealized environmental assumptions and therefore fail to capture the substantial drift forces and hydrodynamic disturbances encountered in rough sea states. To address this issue, we propose a robust deep reinforcement learning framework, termed difficulty-adaptive phasic policy gradient (DA-PPG), integrated with a high-fidelity fluid-structure interaction model that explicitly accounts for relative flow velocities and second-order wave-drift forces. To mitigate sparse rewards and training instability in such chaotic environments, DA-PPG introduces a local A*-guided reward-shaping mechanism and a difficulty-adaptive curriculum learning strategy. Comprehensive simulations in high-fidelity scenarios show that the proposed method outperforms traditional geometric planners and baseline DRL algorithms. In particular, the learned policy maintains smoother and more goal-directed navigation under coupled wind-wave-current disturbances and demonstrates zero-shot generalization by proactively avoiding unseen moving vessels, thereby achieving superior trajectory smoothness, maneuvering efficiency, and navigational safety in complex marine environments.