Enhancing A2G Robustness in Energy-Constrained Multi-UAV Networks: MADRL for Trajectory Control and Resource Allocation

In this paper, we investigate an air-to-ground (A2G) wireless network system where multiple uncrewed aerial vehicles (UAVs) provide downlink communication coverage for mobile ground users (GUs). This system accounts for UAVs progressively depleting their energy during coverage provision, ceasing operations when their energy reserves fall below a predefined threshold. We aim to maximize cumulative system throughput over the task period while satisfying the minimum fairness requirement through joint trajectory control and resource allocation (JTCRA) optimization. To meet the fairness requirement, enhancing system robustness is critical; energy-sufficient UAVs must autonomously assist GUs that lose connectivity when their serving UAVs terminate operations. Therefore, we propose a multi-agent deep reinforcement learning (MADRL) framework with a parameter-sharing architecture to solve this problem. As conventional parameter sharing is restricted to homogeneous agents with identical observation-action spaces, we design a dual-agent structure: a trajectory agent (Traj-agent) and a communication agent (Comm-agent) are deployed for each UAV. This separation organizes the heterogeneous tasks of trajectory control and resource allocation into distinct homogeneous agent groups, facilitating effective parameter sharing within each type. Based on this framework, we apply two alternative algorithms: an MAPPO-based JTCRA algorithm and a QMIX-based JTCRA algorithm. Simulation results demonstrate the superiority and effectiveness of our proposed JTCRA algorithms, which maintain service continuity for GUs through intelligent trajectory control, thereby minimizing the adverse impact of coverage gaps.