DT-MAEI: A Dual Timescale MARL-Embedded Embodied Intelligence Architecture to Enable AUV Cluster Network-Based Multi-Target Tracking

With the rapid development of underwater robotics and multi-agent reinforcement learning (MARL) technologies, autonomous underwater vehicle (AUV) cluster networks have emerged as a promising paradigm for smart multi-target tracking. However, mainstream intelligent algorithms are constrained by the training distribution and cannot effectively handle oceanic disturbances, particularly in MARL-based underwater target tracking. In this paper, we focus on constructing an MARL-embedded embodied intelligence architecture to enable smart and disturbance-robust underwater target tracking. Specifically, we introduce the concept of underwater multi-agent embodied intelligence and propose the Dual Timescale MARL-embedded Multi-Agent Embodied Intelligence (DT-MAEI) architecture, improving the intelligence of the AUV cluster network. For the dedicated MARL algorithm in the proposed DT-MAEI, we propose the dual timescale-based excellence influencing MARL algorithm, to enhance the convergence speed and adaptability of MARL under disturbed underwater environments. Moreover, to enhance the inter-AUV influence in the DT-MAEI, we propose an excellence-alignment reward function and provide a detailed theoretical derivation. Furthermore, we propose the DT-MAEI-based underwater target tracking scheme, to enable AUV cluster network-based smart, efficient, and disturbance-robust multi-target tracking. Extensive evaluation results showcase that our proposed approaches can perform accurate underwater target tracking and outperform some recent research products in terms of convergence speed, tracking performance.