Autonomous Surface Vessels Path Following and Static Obstacle Collision Avoidance using Deep Reinforcement Learning

Autonomous Surface Vessels (ASVs) require reliable path following and collision avoidance for safe maritime operations, particularly in cluttered or uncertain environments. This paper proposes a Deep Reinforcement Learning (DRL) framework that integrates a 3-DOF dynamic ship model with a simulated LiDAR sensor to enable realistic perception and manoeuvring. Two algorithms, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC), are trained and evaluated in randomized scenarios with static obstacles. The framework employs a continuous rudder control action space and a multiobjective reward function with a weighting coefficient to balance path following and obstacle avoidance. Extensive experiments across 1000 test episodes show that SAC consistently outperforms PPO, achieving over 90% success rates, smoother trajectories, and better robustness to varied obstacle layouts. The weighting coefficient is shown to significantly influence agent behaviour, with lambda value of 0.5 providing the best trade-off between safety and efficiency. These results demonstrate the effectiveness of entropy-regularised off-policy methods for ASV navigation and highlight design considerations for future real-world deployment.