Deep Reinforcement Learning-Based Bidding Strategies for Virtual Power Plants in Electricity Spot Markets: A Review

Electricity spot markets increasingly require flexible, fast, and risk-aware decisions as renewable penetration grows and price volatility intensifies. Virtual power plants (VPPs) aggregate distributed energy resources, storage, and flexible loads to participate in day-ahead and real-time markets, but their bidding decisions must balance profit, operational constraints, and uncertainty. Recent VPP reviews emphasize coordination and market integration challenges, while surveys on reinforcement learning in energy markets highlight the rapid adoption of DRL for decision-making under uncertainty. This review focuses on DRL-based bidding strategies for VPPs in electricity spot markets and related competitive market settings. We synthesize market structures and bidding rules, VPP modeling assumptions, uncertainty and risk treatments, and DRL algorithm design for single-agent and multi-agent settings. We also examine common simulators, datasets, and evaluation metrics, and we discuss limitations such as simplified market rules, weak adversarial modeling, and limited generalization. Finally, we outline future research directions, including safe and distributionally robust RL, multi-timescale coordination across energy and ancillary services, and cross-market transfer using digital twins and data-driven validation.