Cooperative Multi-Agent Reinforcement Learning for Energy-Harvesting Aware Dynamic Clustering and Mobile Sink Path Co-Optimization in Wireless Sensor Networks

Energy efficiency and network lifetime are two critical issues in Wireless Sensor Networks (WSNs). This paper introduces a cooperative multi-agent reinforcement learning solution for energy-harvesting driven, joint dynamic clustering and mobile sink routing. There are two cooperative Q-learning agents; the clustering agent selects the best cluster heads based upon residual energy, geographical location, and energy harvesting rates, whereas the routing agent selects the best routing directions for the sink to maximize nodes under the clustered sink for data collection. A real-world experiment based upon the Intel Berkeley dataset features a validation protocol of 55 nodes and 2.3M+ readings which indicate 99 network lifetime steps (1 episode) with 100% of nodes alive, 2,308 collected packets per episode, and an 82% decrease in reward deviation substantiating stable convergence, as well as a 5.6% improvement over the best comparative (DRL-Routing) baseline from total reward 193,944 to 204,750 from step 1 to 200, confirming effective cooperative multi-agent training for energy-harvesting WSNs.