Dynamic Demand Management for Parcel Lockers

In pursuit of a more environmentally sustainable and cost-efficient last mile, parcel lockers have gained a firm foothold in the parcel delivery landscape. To fully exploit their potential and ensure customer satisfaction, successful management of the locker’s limited capacity is crucial. This is challenging as future delivery requests and pickup times are stochastic from the provider’s perspective. In response, we propose to dynamically control whether the locker is presented as an available delivery option to each incoming customer with the goal of maximizing the number of served requests. We take compartment sizes into account, which entails a second type of decision as parcels scheduled for delivery must be allocated. As an extension, we balance rejected requests and failed deliveries through a careful use of overbooking. We formalize the problem as an infinite-horizon sequential decision problem and find that exact methods are intractable for realistic instances. In light of this, we develop a solution framework that orchestrates multiple algorithmic techniques from sequential decision analytics and reinforcement learning in an innovative way, namely cost function approximation and value function approximation (VFA). As a general methodological contribution, we enhance the training of our VFA with a modified version of experience replay. Our computational study shows that our method outperforms a myopic benchmark by 3.4% and an industry-inspired policy by 2.8%. Supplemental Material: The online appendices are available at https://doi.org/10.1287/trsc.2024.0871 .