Hybrid Graph-RL for Beam Management in 6G Satellite Networks With Heterogeneous Service Requirements

To meet heterogeneous IoT service requirements in the era of ubiquitous connectivity, flexible and real-time beam management and power allocation have become a critical challenge for sixth-generation (6G) non-geostationary orbit (NGSO) satellite networks. Conventional model-driven approaches, although analytically tractable, exhibit limited adaptability and scalability in large-scale, dynamic satellite environments. Reinforcement learning (RL)-based solutions offer great flexibility, but still face scalability and robustness challenges due to the massive decision space, complex system dynamics and interference conditions. Moreover, most existing RL frameworks overlook model-based priors essential for facilitating policy training and improving generalization. To address these limitations, this paper proposes a Graph-embedded Deep Deterministic Policy Gradient (Graph-DDPG) beam management framework that jointly optimizes the Quality of Service (QoS) for delay-sensitive (DS) and throughput-sensitive (TS) services while ensuring satellite load balancing. By incorporating model-informed guidance into the learning process, the proposed method substantially enhances scalability and convergence. Simulations verify that the proposed framework achieves up to 56.12% throughput satisfaction increase and at least 19.47% queuing latency reduction over baseline approaches, underscoring its potential for practical and effective beam management in large-scale NGSO satellite networks.