Reinforcement Learning‐Based Fast Frequency Response Using Energy Storage for Remote Microgrids

ABSTRACT The power system is undergoing a significant shift from fossil fuel‐based electricity generation to inverter‐based renewable energy resources (IBRs), accelerating the transition towards cleaner energy. This transition, however, introduces new challenges for system stability and control. One of the most critical issues is the decline in frequency stability due to reduced system inertia and damping, particularly in isolated or weakly interconnected power systems such as microgrids. Therefore, novel ancillary services capable of delivering fast and effective frequency support that accounts for the dynamic nature of the modern power system are crucial. In this study, we develop a reinforcement learning (RL)‐based control framework to provide fast frequency response (FFR) in a microgrid. The RL‐based controller is trained through continuous interaction with a simulated microgrid environment using the soft actor‐critic (SAC) algorithm, an advanced off‐policy RL technique. To enable efficient RL training, a scalable co‐simulation framework with a real‐time digital environment is employed, allowing a parallel execution of online RL training and microgrid model simulation. The RL training configuration is deployed on the Cordova, Alaska, benchmark microgrid. A detailed evaluation of the trained RL‐based controller demonstrates its ability to deliver efficient and timely frequency support to the microgrid, reducing frequency nadirs by 55.03% and 61.78% in cases with and without under‐frequency load shedding (UFLS). Load impact assessments confirm the controller's robustness under varying loading scenarios, and the computational times during training and testing validate its real‐time applicability for use in microgrids. Additionally, a practical evaluation of energy storage system (ESS) sizing under a 2‐day load profile provides valuable insights into resource considerations for real‐world FFR implementation.