A Data-driven Multi-Step Temporal Difference Pretraining Framework of DeepQ-Network for Boiler Combustion Control

Deep Q-network (DQN) has shown significant potential in industrial control, yet its high exploration costs and slow convergence during early deployment hinder practical adoption.To address this, this study proposes a data-driven pretraining framework for DQN, based on a multi-step temporal difference (TD) algorithm.Historical operation data and well-designed reward functions are employed to pretrain the agent, significantly reducing post-deployment exploration.The proposed framework is demonstrated through a case study on boiler combustion control system.The agent built on an enhanced recurrent neural network (RNN) architecture regulates coal feeding and air supply to meet dynamic steam load demands.The effects of different reward formulations and TD steps on pretraining efficacy are analyzed.Results show that agents pretrained with linear and nonlinear rewards both outperform historical control strategies.The 3-step TD pretraining strategy outperforms single-step TD, achieving 83.4% consistency with expert actions and realizing a 5.8% improvement in energy efficiency compared to manual operation.