Optimal scheduling of active distribution network with ice storage system based on deep reinforcement learning

With the increase of renewable energy sources (RESs) and peak loads, the active distribution network (ADN) is faced with huge burdens in active power balance. The ice storage system (ISS) can improve the match degree between the cooling load and distributed RESs. This paper proposes an optimal scheduling scheme of ADN with ISS based on deep reinforcement learning (DRL). First, taking into account the operational flexibility of the ISS, an optimal scheduling model is established for the ADN to minimize the operation cost and RES curtailment. Then, the scheduling problem is modelled as a Markov decision process. And an actor-critic based DRL framework is developed to solve the optimization model, where the state space, action set and reward function are designed. Furthermore, an improved twin delayed deep deterministic policy gradient (TD3) algorithm with multi-experience replay buffers is employed to improve training stability and efficiency. The case study demonstrates the effectiveness of the proposed scheme, which guarantees reliable power supply-demand balance and reduce the curtailment of distributed RESs.