Optimal Cluster Formation Control for Air-Sea Heterogeneous Multi-Robot Systems: A Predefined-Time Estimator-Based Reinforcement Learning Approach

This paper investigates the problem of optimal cluster formation control for air–sea heterogeneous multi-robot systems (HMRSs) subject to external disturbances and energy constraints. To address this challenge, a hierarchical control framework is proposed, which decomposes the problem into two sub-tasks: distributed sliding-mode-based predefined-time formation estimation and reinforcement learning (RL)-based local optimal tracking control. First, a distributed nonsingular predefined-time sliding mode estimation algorithm is designed, which not only effectively eliminates system dimensional heterogeneity but also ensures that each follower robot can accurately estimate the state of its corresponding virtual leader within a predefined time, thereby generating the desired formation con figuration. Subsequently, based on the estimated information, a neural network-driven critic–actor RL framework is constructed to optimize the control inputs. By incorporating backstepping, intermediate control signals are designed to achieve local optimal tracking control of the heterogeneous system. The proposed hierarchical framework effectively decouples predefined-time cluster formation estimation from local optimal tracking control, significantly reducing the complexity of applying RL in formation control. Moreover, compared with existing simplified models, the established air–sea HMRS model achieves higher accuracy. Finally, rigorous Lyapunov stability analysis and numerical simulations are conducted to verify the effectiveness of the proposed algorithm.