A Reinforcement Learning Engine With Reduced Action and State Space for Scalable Cyber‐Physical Optimal Response

ABSTRACT Numerous research studies have been conducted to enhance the resilience of cyber‐physical systems (CPSs) by detecting potential cyber or physical disturbances. However, the development of scalable and optimal response measures under power system contingencies based on fusing cyber‐physical data is still in an early stage. To address this research gap, this paper introduces a power system response engine based on reinforcement learning (RL) and role and interaction discovery (RID) techniques. RL‐RID‐GridResponder is designed to automatically detect the contingency and assist with the decision‐making process to ensure optimal power system operation. The RL‐RID‐GridResponder learns via an RL‐based structure and achieves enhanced scalability by integrating an RID module with reduced action and state spaces. The applicability of RL‐RID‐GridResponder in providing scalable and optimal responses for CPSs is demonstrated on power systems during cyber disturbances. Moreover, simulations are conducted on a Volt‐Var regulation problem using the augmented WSCC 9‐bus, augmented IEEE 24‐bus and IEEE 8500‐node systems based on fused cyber and physical datasets. The results show that the proposed RL‐RID‐GridResponder can provide fast and accurate responses to ensure optimal power system operation under cyber intrusions, such as DoS, and can extend to other system contingencies, such as line outages and load losses.