Reinforcement Learning-Based Sliding Mode Control of Underwater Vehicles with Bow Rudders and X-Stern Rudders

This paper addresses the motion control for an x-rudder underwater vehicle, which features a bow rudder and four independent x-shaped stern rudders. To achieve coordinated operation of bow and stern rudders of the x-rudder underwater vehicle, the motion controller is divided into two parts: dynamic controller and control distributor. A model-free sliding mode parameter optimization control algorithm for underwater vehicles based on reinforcement learning (RL) is proposed. The proposed algorithm integrates a fast terminal sliding mode controller based on prior model knowledge with a model-free, data-driven input derived from reinforcement learning, ensuring both efficiency and adaptability. The control allocator employs an improved sequential quadratic programming approach to tackle the mixed minimization problem, considering various evaluation criteria and constraints. The effectiveness of the proposed control method is validated through numerical simulations across different conditions, and its performance is compared in terms of accuracy, convergence, and computational complexity.