Foreseeable Implicit Training for ESS Operation Based on Deep Reinforcement Learning

The recent increase in behind-the-meter renewable energy (RE) has accelerated the deployment of residential energy storage systems (ESS). However, uncertainty in RE generation presents significant challenges in ESS operation, such as maintaining power balance and minimizing costs. To address these issues, we propose a deep reinforcement learning (DRL) framework. Unlike conventional DRL approaches, the proposed method reformulates the control problem as areference trackingtask, where the agent learns to follow reference trajectories generated offline by an optimization solver. This design enables the agent to implicitly consider future uncertainties while adapting to Time-of-Use (TOU) price signals without relying on any forecasting model. Additionally, we introduce a knowledge peering structure, in which peer actors guide each other during training to improve generalization and prevent overfitting. Our method achieves up to 82% improvement in optimality gap compared to existing baselines and demonstrates near-optimal performance without requiring explicit future information. Extensive experiments confirm that the proposed algorithm, named FITNESS, is a robust and scalable solution for real-time ESS control under uncertainty.