Modeling and Operation of Underwater Intelligent Communication Systems: A Survey of Multi-Agent Reinforcement Learning-Based Approaches

With the continuous evolution of the 6G sea-land-air integrated communication paradigm, underwater intelligent communication systems have emerged as a prominent research focus in recent years. Due to harsh environmental conditions, limited communication bandwidth, high propagation delays, and severe energy constraints, the modelling and operation of underwater intelligent communication systems face significant challenges in achieving intelligence, coordination, and sustainability. To meet the above challenges, multi-agent reinforcement learning (MARL), characterized by its decentralized and autonomous decision-making capabilities, has been regarded as a promising framework for intelligent, distributed, and adaptive coordination in such environments. Nevertheless, there is still a lack of comprehensive surveys on using MARL to optimize underwater communication networks. Therefore, this survey provides a comprehensive overview of recent advancements in applying MARL techniques to optimize underwater communication networks and bridges this gap. Specifically, we review the fundamental components of MARL and explain why it is particularly well suited for optimizing underwater networks. Then, we review MARL-based underwater applications in both static communication systems and dynamic mobile networks, including routing strategies, network security, resource allocation, MAC layer optimization, cooperative multi-robot formation, and target tracking. Furthermore, we summarize algorithmic innovations tailored to underwater communication optimization, focusing on improvements in training efficiency, robustness, and scalability. Building on these analyses, we further analyze open challenges and outline future research directions for advancing MARL-enabled underwater communication networks.