Distributed Formation Control for Underactuated Multi-ASVs Under DoS Attacks Using Value Decomposition Reinforcement Learning

This article addresses the formation control problem of underactuated multi-autonomous surface vehicles (ASVs) under denial-of-service (DoS) attacks on the communication network and complex uncertainties, including unknown ASV dynamics, external disturbances, and obstacles. First, a novel distributed target estimator (DTE) using a first-order low-pass filter is designed to estimate the state of the target based on partial observability under target information constraints and DoS attacks. Second, a safe guidance law for the ASVs is developed using the estimated target state and a control barrier function. Third, a “dual-adaptation control and learning” bidirectional fusion model is constructed. Specifically, on one hand, a distributed adaptive formation controller based on value decomposition reinforcement learning (VDRL) is designed. By decomposing the global value function, this controller addresses the credit allocation issue in cooperative formation and allows the ASVs to adjust their strategies autonomously based on the environment. On the other hand, an adaptive mechanism is used to improve the computation strategy of the global value function in VDRL, enhancing the learning efficiency and stability of the reinforcement learning (RL) algorithm in complex environments. The proposed VDRL-based formation control algorithm of the underactuated multi-ASVs ensures accurate target state estimation and convergence of formation errors under DoS conditions. A rigorous theoretical analysis is further used to ensure the closed-loop stability of the multi-ASV systems. Finally, simulation results validate the effectiveness of the proposed distributed formation control algorithm.