Deep Reinforcement Learning-Based eMBB and URLLC Service Multiplexing in Wireless NOMA Networks

In 5G/6G wireless networks, enhanced mobile broadband (eMBB) services provide gigabit-per-second high data rates, while ultra-reliable and low-latency communication (URLLC) services offer extremely low latency and high reliability. Simultaneously supporting these two types of services in the same wireless environment presents a significant challenge due to their fundamentally different service requirements. In this paper, we formulate the bandwidth and power resource allocation problem of multiplexing eMBB and URLLC services in a wireless non-orthogonal multiple access (NOMA) network as a nonlinear programming problem. The objective is to maximize the average supply-demand ratio (SDR) of eMBB users while minimizing the SDR variance among eMBB users, subject to the reliability constraint of URLLC users. We propose a deep reinforcement learning (DRL)-based NOMA resource allocation scheme (DRL-NRAS), which employs two DRL agents at the base station to jointly optimize bandwidth and power allocation for eMBB services, as well as the puncturing and superposition ratios for URLLC transmissions. The proposed DRL-NRAS fully utilizes a DRL technique, twin delayed deep deterministic policy gradient, to effectively handle the time-varying data rate requirements of eMBB users and the stochastic request arrivals of URLLC users. Simulation results demonstrate that the proposed DRL-NRAS reduces the SDR variance by 78.4%, 87.2%, and 72.7% compared to the CVX, B-greedy, and random schemes, respectively, while achieving a 0% delay violation rate under the considered simulation settings. These results indicate that the proposed DRL-NRAS can effectively improve the fairness among eMBB users while ensuring the reliability of URLLC services.