Reinforcement Learning Based Optimal Control: A Survey on Adaptive Dynamic Programming in Robotics

Adaptive Dynamic Programming (ADP) aims to solve nonlinear control problems without explicitly solving complex equations, while still providing Lyapunov-based stability guarantees. Moreover, ADP offers a systematic framework for approximating near-optimal control policies through online learning and function approximation. This survey reviews the foundations of ADP architectures, including actor-critic structures and their variants, and examines representative applications in robotic manipulators and Mobile Wheeled Robots (MWR). By reviewing recent progress and pinpointing core limitations, this paper underscores the potential of ADP as an effective learning-based control framework for safety-critical robotic systems and outlines promising directions for future research.