An Energy-Efficient Deep Reinforcement Learning Framework for Joint UAV Trajectory Optimization in Logistics-Assisted 5G/6G Cellular Networks

Unmanned Aerial Vehicles (UAVs) offer significant potential for dual-purpose operations, simultaneously enhancing 5/6 G cellular network capacity and providing logistics services such as parcel delivery. However, existing trajectory optimization approaches rely on idealized disk coverage models that ignore realistic channel effects, leading to inefficient energy consumption and poor communication reliability. This paper proposes a novel Energy Consumption, Throughput, and Delay (ECTD) optimization framework that leverages Deep Reinforcement Learning (DRL) with attention mechanisms to address the NP-hard combinatorial-continuous optimization challenge of jointly determining UAV visiting sequences (discrete) and contact locations (continuous) for dual-purpose logistics-cellular missions. The DRL approach enables learning generalizable policies across variable cluster configurations while handling the joint discrete-continuous decision space that traditional optimization methods cannot efficiently solve. The ECTD algorithm employs an attention-based graph neural network architecture with greedy rollout baseline and incorporates a realistic Packet Reception Ratio-based coverage model with Nakagami-$m$fading. Extensive simulation results demonstrate that the proposed framework achieves up to 12% energy savings over state-of-the-art DRL methods (DDPG, PPO, A3C, SAC) and up to 37% compared to heuristic approaches (TSP, CARLO, ENERGENT, ECAE). The algorithm achieves network throughput improvements of up to 37%, maintains communication reliability above 90% under fading conditions.