Twin-Timescale Traffic Signal Timing Method With Joint Communication Resource Allocation

In multi-intersection traffic signal timing (TST) the use of a uniform cycle time reduces efficiency at low-volume intersections and signal changes which can alter vehicle flows and degrade communication quality. This paper presents a solution to this problem introducing a Twin-Timescale model whereby a multi-intersection TST method is jointly considered with communication resource allocation (CRA) strategies. For each intersection, a Twin-Timescale Reinforcement Learning (RL) algorithm is proposed, by considering both CRA and TST as the Short- and the Long- Timescale models, respectively, and jointly formulating their rewards. Based on the characteristics of this Twin-Timescale model, the reward function is constructed as a singularly perturbed model (SPM), and a state feedback controller that can ensure asymptotic stability is designed. Furthermore, a novel asynchronous time Multi-Agent Coordinated RL algorithm is proposed, which allows each intersection to generate each own joint parameters for its decision strategy which are based upon information obtained from neighboring intersections at asynchronous time. In addition, the use of Federated Learning (FL) in conjunction with the proposed algorithms is investigated, and its advantages in enhancing the resistance to malicious attacks are discussed. Using real traffic data, various performance evaluation results obtained by means of extensive computer simulation experiments have confirmed that the proposed TST-CRA joint optimization approach significantly improves traffic indicators and maintains the quality of the overall communication performance. Through numerical calculations, the state feedback controller coefficients have been determined.