From Bellman to Real-Time: Graph Compression, DCRNN, and MARL for Scalable Energy System Control—Methodology and Initial Validation

The optimal control of complex energy systems via Bellman’s principle of optimality quickly becomes difficult for high-dimensional, path-dependent dynamics because the state space grows exponentially with each time step. We propose a computationally tractable framework based on hierarchical path-dependency decomposition: (i) graph compression that reduces multi-layer topologies to a single directed flow network; (ii) a diffusion convolutional recurrent neural network (DCRNN) that maps historical trajectories into a finite-dimensional hidden state, approximating the transport of past states without storing full trajectories; and (iii) multi-agent reinforcement learning (MARL) for decentralized local control. By restricting full-path online optimization to a rolling one-step horizon and using temperature and flow rate as sufficient statistics for thermal dynamics, the framework preserves physical fidelity while enabling real-time execution. This reduction is justified because thermal constraints constitute the primary active failure mode in energy systems. We provide a proof sketch showing that the diffusion convolution operation in the DCRNN approximates the Green’s function of the underlying transport equation. Using weather data from Palm Springs, California (a region with moderate path dependency), initial numerical experiments achieve 98.4% correlation with reference solutions, a temperature forecasting mean absolute error (MAE) of 1.23 °C, and mostly subsecond (