Cart-Pole System: Practical Deployment of Model-Based Control Versus Reinforcement Learning

This paper compares the classical model-based approach and reinforcement learning (RL) approach for nonlinear systems with known structures but unknown parameters, which represents a wide class of engineered control systems. The study is based on a benchmark cart-pole system involving both swing-up and balance control. Although the model-based approach for the cart-pole system has been studied extensively in the literature, we propose a new pseudo-energy function with a logarithmic barrier function (LBF) for smoother swing-up action. A simple feedforward compensation is used to reduce the cart friction. The subsequent balance control employs a linear quadratic regulator. The model-based approach is complemented with a real-time parameter identification scheme. For the RL approach, we adopt the deep deterministic policy gradient (DDPG) algorithm in conjunction with several simulation techniques to mitigate the simulation-to-reality (sim-to-real) gap. Experimental results on the Googoltech platform demonstrate that both design approaches are effective. The model-based approach does not require training and has higher computational efficiency, but it requires substantial system modeling and controller design knowledge. In contrast, the RL approach has higher flexibility and adaptability, but requires massive training. The results reflect the trade-offs and complementarity of the two methods in practical control tasks.