Simulation-based deep reinforcement learning for autonomous loading control of underground load–haul–dump machines

Abstract Autonomous loading operations in underground mining present complex engineering challenges due to the unpredictable interactions between load–haul–dump (LHD) machines and fragmented ore, as well as the dynamic and heterogeneous nature of muck piles. To address these challenges, this study presents a simulation-based deep reinforcement learning (DRL) autonomous loading controller tailored for LHD performing drawpoint mucking in block and panel caving and sublevel caving mines. The proposed method leverages the proximal policy optimization algorithm, trained within a high-fidelity simulation environment developed using the MuJoCo physics engine and the Gymnasium interface, and enhanced by domain randomization techniques to improve robustness across diverse loading scenarios. A novel reward function incorporating thrust, loading volume (LV), cycle-time-related efficiency, and action penalties, together with an optimized observation space integrating key sensor data, is designed to enhance learning effectiveness. Comprehensive simulation experiments show that the proposed controller outperforms a finite-state-machine-based controller, skilled human operators, and standard model-free RL baselines (twin delayed deep deterministic policy gradient, soft actor–critic, deep deterministic policy gradient) in terms of LV and volumetric loading rate, while maintaining a high success rate of loading under the tested conditions. Ablation studies further validate the critical role of individual reward components and observation parameters in achieving stable and efficient loading behavior. In addition, we explicitly model sensor errors (measurement noise and drift) for all onboard sensors and evaluate the robustness of the learned controller under moderate and severe sensor degradation and single-sensor failures, quantifying their impact on loading performance. This simulation-based study thus indicates that DRL is a promising and effective approach for autonomous LHD loading, with the potential to enhance operational efficiency and safety in underground mining environments as the controller is progressively transferred from simulation to real machines.