Two-Dimensional Model-Free Off-Policy Optimal Iterative Learning Control for Time-Varying Batch Systems

Although iterative learning control (ILC) has been widely used in batch processes, designing an optimal iterative learning control scheme for batch systems with unknown dynamics and time-varying parameters remains an open problem. In this paper, we propose a novel two-dimensional model-free off-policy optimal iterative learning control to achieve optimal control performance for linear time-varying batch systems. First, the one-dimensional state space is expanded to the two-dimensional state space by integrating time and batch information. Then, based on dynamic programming and a recursive algorithm, the framework of two-dimensional model-based optimal iterative learning control is established. Based on this framework, two-dimensional model-free optimal iterative learning control is further developed using model-free Q-learning reinforcement learning. The optimal iterative learning control policy is obtained through online off-policy iteration using historical and online operation data. Meanwhile, a rigorous convergence proof of the model-free optimal iterative learning control law is presented. Finally, the simulation results in the injection molding batch process demonstrate the proposed control scheme's effectiveness, feasibility, and significant improvement in control performance.