Adaptive Traffic Signal Control Based on Multiagent Soft Actor-Critic

In IoT-centric adaptive traffic signal control, existing multi-agent deep reinforcement learning methods are constrained by insufficient communication mechanisms. These limitations become particularly evident when handling the dynamic traffic networks and high-dimensional action spaces, leading to unstable environmental observability and diminished control effectiveness. Motivated by the idea of policy entropy maximization and the paradigm of centralized training and decentralized execution, the MASAC (Multi-Agent Soft Actor-Critic) algorithm was developed. However, the algorithm exhibited issues such as unstable training and low sample efficiency during its application. To enhance MASAC’s training speed and stability, the prioritized experience replay was introduced, leading to the development of a Multi-Agent Soft Actor-Critic algorithm based on prioritized experience replay(MPER-SAC), which improves the model’s learning efficiency. Joint simulation experiments using SUMO and Carla demonstrated that the proposed algorithm increased the average reward in the traffic environment by 9.55%, boosted average vehicle speed by 14.52%, and reduced average queue length and average intersection delay by 15.35% and 11.26%, respectively. Additionally, the ablation study displayed significant performance improvements, with an 11.36% increase in average reward, a 11.65% reduction in average queue length, and a 8.12% decrease in total travel delay. Additional performance metrics also exhibited varying degrees of enhancement.