ASMARL: Optimizing Traffic Signal Coordination Through Multi-Agent Reinforcement Learning With Attention Mechanisms and Sequential Rollout

In cooperative multi-agent reinforcement learning tasks, partial observability and non-stationarity from agents updating their parameters are significant challenges. A common solution is sharing all observations and actions among agents directly, but this can lead to excessive communication costs, especially with many agents, making it impractical. Hence, it’s crucial to selectively share important information to reduce redundancy and effectively represent and integrate the shared information. This paper proposes an Attention-based Shared Multi-Agent Reinforcement Learning (ASMARL) method for multi-intersection traffic signal coordination, introducing an optimal baseline to reduce variance. The approach includes constructing communication channels using attention mechanisms to integrate shared information, designing a sequential rollout mechanism to incorporate agents’ actions within the communication range into the Critic network’s input, and introducing an optimal baseline to mitigate high variance in multi-agent policy gradient. Simulations in the urban traffic software (SUMO) show that ASMARL outperforms current mainstream methods in reducing average traffic delay.