Reinforcement Learning Neural Network Observer-Based Adaptive Optimal Time-Varying Formation Control for Uncertain Multi-Agent Systems With External Disturbance

The traditional formation control methods of multi-agent systems (MASs) often rely on restrictive linear matrix inequalities and strong assumptions. To overcome this limitation, this work proposes an adaptive optimal time-varying formation control protocol for uncertain MASs, integrating reinforcement learning(RL)-based neural network observer. First, a radial basis function neural network(RBFNN)with adaptive law is designed to approximate the unknown nonlinear dynamics. An RBFNN-based state estimator and a novel disturbance estimator are developed to reconstruct the unmeasurable system state and the disturbance generated by an exosystem, respectively. Second, using the state estimate, a distributed formation control error system is established, and a performance index based on a Hamilton–Jacobi–Bellman(HJB)equation is introduced to achieve the objective of optimal formation control. Third, leveraging the state and disturbance estimates, an actor-critic RL-based optimal time-varying formation control protocol with adaptive update laws is proposed. Actor and critic networks are constructed to approximate unknown nonlinear terms in HJB equation. It is proved that estimator error systems and RBFNN approximation errors are semi-globally uniformly ultimately bounded(SGUUB). Simulation examples are provided to verify the effectiveness of the proposed estimators and optimal formation control protocol. Moreover, the method is successfully applied to unmanned aerial vehicles (UAVs) formation control in the Isaac Sim simulation environment, demonstrating the method’s practical feasibility.