Two-timescale hierarchical DRL for resource allocation in UAV-assisted 6G edge networks

Integrating unmanned aerial vehicles (UAVs) into mobile edge computing (MEC) systems offers a promising solution to enhance coverage and flexibility in 6G edge networks. However, the dynamic and resource-limited nature of UAVs introduces strong cross-layer coupling between UAV–access point (AP) coordination and AP–user scheduling, making efficient resource allocation highly challenging. To address this issue, this paper proposes a unified two-timescale hierarchical deep reinforcement learning (DRL) framework that jointly optimizes channel utilization and transmission latency. Unlike conventional hierarchical decompositions that separate large- and small-timescale decisions, the proposed method models UAV–AP interactions as an integrated optimization process to preserve global consistency and avoid local optima. A dual-layer architecture is designed, where the UAV-level agent performs large-timescale bandwidth allocation and the AP-level agents conduct fine-grained small-timescale scheduling. To mitigate the exponential complexity caused by nested DRL updates, an asynchronous cross-timescale coordination mechanism is introduced, enabling stable convergence and efficient learning. Simulation results demonstrate that the proposed framework achieves superior performance in channel utilization, delay reduction, and robustness under dynamic traffic conditions compared with single-agent, prediction-based, and heuristic baselines.