Discovery of Bus Loop Scheduling strategies with reinforcement learning to minimize commuters’ waiting and travel times

In this paper, we investigate the application of two reinforcement learning methods, known as the Dueling Double Deep Q-Network and Soft Actor-Critic to discover bus scheduling strategies and compare them against conventional approaches. In particular, we look into real-time control strategies where buses may choose to stay or leave at bus stops. We explore both waiting time and travel time as the optimization objectives. The results for uniform bus frequency show that average waiting time can be reduced by allowing buses to stay longer at stops with higher passengers’ arrival rate but at the cost of increased average travel time. This is also supported by our analytical calculation on a theoretical bus loop model. We then apply our method to a model based on a real world bus loop in Nanyang Technological University. The results highlight the potential benefit of reinforcement learning methods to find novel strategies that can be better than conventional approaches. The similar performance of the two distinct reinforcement learning methods also serves as independent verification of the validity of the strategies obtained. This is an extended version of our ICCS 2025 conference paper “Bus Loop Scheduling with Dueling Double Deep Q Network” Pradana and Chew (2025), with the main addition of the application of the Soft Actor-Critic method which has to be modified to handle the optimization problem described in this paper.