Deep Reinforcement Learning-Based Beam Hopping Optimization in LEO Satellite Networks

In beam hopping-based low earth orbit (LEO) satellite systems, spatially imbalanced and temporally varying ground traffic requires dynamic wireless resource allocation. This study proposes a long short-term memory (LSTM)-based approach that leverages real-world mobile network data to predict percell traffic demand. Based on these predictions, two-types of deep Q-network (DQN) agents are assigned to each beam to jointly optimize cell selection and bandwidth allocation. The proposed dual-agent DQN architecture decouples the complex action space, thereby enhancing learning efficiency. In addition, a unified reward function is designed to simultaneously capture traffic service efficiency, average queueing delay, and the number of beam switchings. Simulation results show that the proposed algorithm significantly outperforms existing benchmarks in terms of overall resource utilization efficiency.