Distributed Optimal Control Using Interval-Excited Relative-Output Reinforcement Learning Algorithm

This paper addresses the distributed adaptive optimal output tracking control problem for unknown heterogeneous multi-agent systems. Unlike existing studies that typically assume the leader's state is either globally or distributively available for information exchange, this paper proposes a dynamic state feedback control policy driven by an internal model to achieve output regulation. This policy utilizes only relative output measurements between neighboring agents to achieve output tracking, effectively relaxing the need for leader state information. Second, we design an online adaptive dynamic programming algorithm that operates under interval excitation (IE) conditions, replacing the traditional persistence of excitation (PE) requirement, to learn the optimal control policy. The uniqueness of the solution for the optimal gains of the linear regressor equation is obtained by verifying an invertible matrix under the online-verifiable IE condition. Compared with existing algorithms, our proposed online learning algorithm relaxes the requirements of the PE condition, the full-rank condition, and intelligent data storage. Finally, a numerical example is presented to demonstrate the effectiveness of the proposed algorithm.