Optimal Tracking Control of Uncertain Nonlinear Systems Using Simplified Reinforcement Learning

This article investigates the optimal tracking control problem for high-order uncertain nonlinear systems by developing a simplified reinforcement learning (RL) framework with minimal neural networks (NNs). In contrast to conventional RL-based schemes that rely on recursive backstepping and require $3n$ NNs (where $n$ is the system order), the proposed method leverages high-order fully actuated (HOFA) system theory to reformulate the dynamics into a compact normal form. This enables a unified, nonrecursive controller design that requires only three NNs regardless of the system order, thereby significantly reducing computational complexity and facilitating practical implementation. Furthermore, this work overcomes a critical theoretical deficiency in existing simplified RL strategies, where the vanishing minimum eigenvalue of the NN basis function correlation matrix often leads to invalid Lyapunov stability analysis. A novel critic-actor weight update law is designed to bypass this problematic matrix, rigorously guaranteeing the semiglobal uniform ultimate boundedness of the closed-loop system without requiring persistent excitation (PE) conditions. Simulation results on a representative example demonstrate the effectiveness and computational efficiency of the proposed approach compared with existing methods.