Proportional Fair Resource Scheduling for Dynamic Beyond 5G Networks: A Distributed Hierarchical DRL Approach

In beyond 5G multi-cell networks, cell-edge users generally experience poor communication quality due to their greater distance from the base station (BS), and increased interference from neighboring cells, which severely impacts their user experience. To address this issue, achieving fair and efficient resource scheduling is key to ensuring the quality of service for edge users. Therefore, this paper formulates a joint optimization problem of spectral subband selection and power control, aiming to maximize the proportional fair sum rate of the multi-cell network. However, most existing algorithms require instantaneous global channel state information, which results in poor scalability and is impractical, especially in highly dynamic wireless network environments with user mobility. Noting that the considered problem can be modeled as a decentralized partially observable Markov decision process, we propose a multi-agent deep reinforcement learning (MADRL) scheme based on a hierar chical centralized training and distributed execution framework (MADRL-HE), enabling agents to make spectral subband and transmit power selections using only local information and some outdated non-local information. Simulation results demonstrate that the proposed scheme features excellent scalability and fast convergence. Moreover, its proportional fair sum rate performance consistently outperforms the existing MADRL scheme in dynamic environments and surpasses centralized iterative optimization schemes in most dynamic scenarios.