Cooperative reinforcement learning for non-convex multi-area economic dispatch

As power systems continue to expand and electricity demand grows rapidly, efficient multi-area economic dispatch (MAED) becomes increasingly important for reducing operating costs. However, practical considerations such as transmission losses lead to non-convex coupling constraints in MAED. This substantially increases the computational difficulty of the problem. To address these challenges, this paper proposes a stepwise cooperative reinforcement learning (RL) framework for MAED, consisting of a power-exchange module and a power-distribution module operating at different algorithmic time scales. The power-exchange module optimizes inter-area power exchanges using deep hysteretic RL combined with a finite-time average consensus protocol. The power-distribution module determines intra-area generation dispatch via communication-assisted RL with quadratic function approximation and distributed projection optimization. By first determining inter-area exchanges and then performing local dispatch, the proposed framework decomposes the original non-convex MAED problem into a sequence of tractable subproblems and enforces operational constraints by construction. Simulation studies demonstrate that the proposed method achieves lower total operating cost than conventional optimization baselines and existing RL-based approaches, while maintaining strict feasibility under non-convex coupling constraints. • A hierarchical RL framework is proposed for multi-area economic dispatch, explicitly considering valve-point effects and transmission losses. • An algorithm is developed to address non-convex optimization problems while consistently satisfying non-convex coupling constraints. • Complex constraints are projected to efficiently guarantee feasibility.