Aircraft push-back planning with stochastic requests and service times: a multi-agent reinforcement learning approach

Efficient planning of aircraft towing tractors is critical for aircraft push-back and flight on-time performances. However, various uncertainties exist (e.g., the arrival of push-back requests from aircraft and the corresponding service time are stochastic), making the scheduling of aircraft towing tractors challenging. This paper addresses a stochastic aircraft push-back scheduling problem which minimises towing service delays and tractor travel distances by developing an intelligent towing tractor scheduling approach. Specifically, the ready-to-push-back time of each aircraft becomes known only after the request is issued, while the real service time becomes known only after the push-back operation is completed. A Decentralised Markov Decision Process is developed to characterise the inherent stochasticity, in which each towing tractor acts as an agent that makes independent decisions based on the global state, while global optimisation is achieved through centralised training and a decentralised execution paradigm. Furthermore, a multi-agent reinforcement learning framework is proposed for policy training with two convergence-acceleration strategies: an action masking scheme and behavioural cloning pre-training. Computational experiments reveal that our proposed approach can achieve a maximum of 80.24% cost reduction compared to traditional approaches. We further evaluate and compare the performances of various tractor fleet operational models to make recommendations for airports.