A Reward Cooperative Distribution and Tracing Mechanism-Enabled MARL Algorithm for Adaptive Routing in SDN-Enabled UASNs

Underwater Acoustic Sensor Networks (UASNs) have garnered considerable attention in recent years due to their widespread applications in both industrial and civilian domains, such as ocean exploration and environmental monitoring. This paper introduces an intelligent Multi-Agent Reinforcement Learning (MARL) algorithm to determine routing in UASNs with dynamic underwater environments adaptively. Initially, we model ocean currents and acoustic signal loss in underwater communication to perform the real-world characteristics of underwater routing. Based on software-defined networking (SDN) principles, we redefine the architecture of UASNs and propose an Adaptive Routing scheme for Software-Defined UASNs (ARSDU). Leveraging ARSDU, we propose the Reward Cooperative Distribution and Tracing Mechanism-enabled Multi-Agent Reinforcement Learning (RCDTM-MARL) algorithm to optimize routing decisions. The proposed RCDTM-MARL algorithm enhances the convergence speed of MARL by decomposing the reward function into independent-reward and interactive-reward, while incorporating an experience replay buffer to further accelerate convergence. Ultimately, the adaptive routing decision algorithm, based on RCDTM-MARL, adaptively determines the optimal routing path for UASNs. The evaluation results demonstrate that the proposed routing scheme outperforms recent research approaches, achieving superior underwater data routing decisions on multiple key performance metrics.