Digital twin-driven Proximal Policy Optimization for smart community energy management and peer-to-peer trading in the Smart Grid 4.0 framework

The integration of renewable energy and smart home technologies necessitates advanced Home Energy Management Systems (HEMSs) to optimize energy consumption, reduce costs, and maintain occupant comfort in the Smart Grid 4.0 framework. Peer-to-peer (P2P) energy trading enhances efficiency by enabling homes to share surplus energy, addressing challenges in dynamic real-time pricing (RTP) and variable renewable generation. This study introduces a novel Home Energy Management System based on Proximal Policy Optimization (PPO) an artificial intelligence (AI) method from deep reinforcement learning (DRL) integrated with digital twin technology, offering a transformative solution for smart community energy management and P2P energy trading. Applied to a community of eight homes over 96 15-min intervals, the model leverages Real time pricing (RTP), photovoltaic (PV) systems, wind turbines, energy storage systems (ESSs), and electric vehicles (EVs). Digital twins provide precise real-time tracking of indoor temperatures (22–26 degrees Celsius (°C)), hot water temperatures (40–46 °C), and battery state of charge (SOC), enabling adaptive control of appliances, heating, ventilation, and air conditioning (HVAC) systems, water heaters, and P2P trading at favorable rates. Enhanced by Generalized Advantage Estimation (GAE), PPO achieves rapid convergence within approximately 320 episodes, reducing costs by 32 % (from €324.63 to €220.74) and ensuring over 95 % comfort compliance. This approach surpasses traditional heuristic methods, demonstrates robustness against pricing and generation uncertainties, and establishes a scalable, cost-effective framework for sustainable energy systems.