Incremental Q -Learning for Data-Driven Adaptive Critic Control of Wastewater Treatment Processes With State Constraints

With the widespread adoption of massive multiple-input multiple-output (MIMO) technology, itThe wastewater treatment process (WWTP) can effectively convert wastewater into reusable water, with dissolved oxygen (DO) concentration playing a crucial role in treatment efficiency. To achieve online control of DO concentration, a data-driven incrementalQ-learning (IQL) method with utility reshaping is proposed. The core idea is to integrate the incremental information of control inputs, the norm value of tracking errors, and a control barrier function into the utility function. First, the IQL method designs an online controller by utilizing tracking errors and the incremental information of control inputs, which reduces the approximation pressure on neural networks caused by excessively large control inputs. Second, the IQL leverages the norm value of tracking errors to influence the magnitude of the incremental control inputs, thereby enhancing the adaptive capability and online stability. Third, an adjustable control barrier function is employed to ensure that the tracking errors do not exceed the predefined constraints, thus meeting the specified tracking accuracy requirements. Fourth, a stability criterion related to learning rates is established to demonstrate the convergence of neural network weights. Finally, the superior control performance of the IQL method is validated via simulation results under two distinct weather conditions.