Reinforcement learning-based linear quadratic tracking design for linear time-invariant systems with unknown dynamics

The objective of this paper is to present the design of an on-policy Linear Quadratic Tracking (LQT) controller for Continuous-Time (CT) Linear Time-Invariant (LTI) systems whose dynamics are entirely unknown. The Q-learning technique, a popular algorithm of Reinforcement Learning, is utilised in the design of the LQT. An augmented Algebraic Riccati Equation (AARE), created especially for the LQT issue, is utilised to obtain a fixed-point equation in terms of the Q-function and Pontryagin's minimal principle in the context of Q-learning. The process of optimising the estimated parameters online is accomplished through the utilisation of gradient descent-based parameter update laws to minimise Bellman's error term. In contrast to the existing literature, the proposed controller for the LQT problem is an on Q-policy method, which does not require a restrictive initial stabilising control policy for estimated parameter convergence, and is free from the cumbersome process of intelligent storage of previous data for satisfying the rank condition. The usefulness of the employed Q-learning approach has been confirmed by displaying the simulation results.