Federated reinforcement learning based dual-level voltage regulation for PV-rich distribution grids

• A FRL method is used for unbalanced MV-LV distribution grid operational safety, which preserves energy privacy of users. • A decentralized and hierarchical training framework is developed to ensure MV voltage security and LV three-phase balance. • The proposed method can be robust to communication delay and readily scaled to large-scale systems. In order to eliminate medium voltage (MV) voltage violations and low voltage (LV) three-phase voltage unbalance in PV-rich dual-level distribution networks simultaneously, a novel federated reinforcement learning (FRL)-based voltage regulation method is proposed. First, voltage regulation is formulated as a Markov Game, and each LV station is constructed as an agent. The rewards of MV-LV control goals are decomposed to hierarchically train agents, enabling simultaneous mitigation of MV voltage violations and LV three-phase voltage unbalance. The federated learning framework is employed on agent training for learning MV-LV voltage regulation policies by interacting with partial real data and policy rewards to achieve better privacy preservation and scalability. Moreover, to enhance robustness against imperfect communication environments, we implement weighted data filling for imputing missing data. Simulation results on MV-LV distribution systems demonstrate the effectiveness and advantages of the proposed method.