Deep Recurrent Q-Network Based Communication and Coordination for Collaborative UUV-USV Swarms

This paper introduces a joint communication and coordination strategy for a mixed team of unmanned surface vehicles (USVs) and unmanned underwater vehicles (UUVs) executing distributed underwater tasks. We propose a Deep Recurrent Q-Network (DRQN) strategy to enable UUVs to select actions that maximize global task completion. It deals with the challenge that acoustic communication is inherently stochastic and constrained. We formulate the problem of underwater collaborative task execution as a partially observable Markov decision process (POMDP) whose solution jointly determines the optimal plan for communication, movement, and recharging. The proposed DRQN model learns to balance task handling, data exchange, and energy efficiency. By reasonably exchanging data with the USV, UUVs can approximate globally optimal decisions for coordination. To ensure a realistic evaluation, we incorporate a high-fidelity acoustic channel model that accounts for geometric spreading, Thorp absorption, salinity-dependent sound speed profiles, current-induced Doppler shifts, and wind-driven ambient noise. Extensive simulations demonstrate the efficacy of the proposed approach in optimizing task assignment and navigation for connected UUV swarms.