Temperature Control for Cyber-Physical Thermal Systems over Wireless Networks: A Model-Assisted Deep Reinforcement Learning Approach

The integration of cyber-physical systems (CPS) with Industrial Internet of Things (IIoT) requires reliable and efficient thermal management, yet existing control methods often degrade under nonlinear thermal dynamics and unreliable wireless links. This paper investigates cyber-physical control of a nonlinear thermal system, where a remote controller regulates the temperature of a thermal plant over a wireless communication network. We first establish a cyber-physical thermal system (CPTS) model that explicitly incorporates nonlinear heat-transfer mechanisms (conduction, convection, and radiation) together with wireless transmission impairments (fading and noise). Based on this model, we formulate an optimal temperature tracking control problem and characterize the structural properties of the optimal solution using the homotopy perturbation method. To enable practical implementation under real-time constraints, we develop a model-assisted structured deep reinforcement learning (DRL) framework, in which a deep neural network (DNN) approximates only the residual high-order terms of the control law while structured update rules guide the effective learning process. The almost sure convergence of the proposed learning scheme is established using Lyapunov stability analysis. Numerical evaluations are performed under a furnace temperature control setup using simulation data generated from the proposed CPTS model parameterized by typical furnace settings, which accurately capture the underlying furnace dynamics. The proposed scheme achieves a best tracking error of about 0.01 in terms of mean square error (MSE) and converges within 50 iterations. This corresponds to a 20 dB reduction in MSE and a twofold improvement in convergence speed compared with state-of-theart control schemes, thereby demonstrating both the effectiveness and robustness of the proposed approach for wireless CPTS.