Energy Efficiency Optimization Based on Multi-Agent DRL for Cognitive UAV-Assisted MEC HetNets

The future sixth generation (6G) mobile network will be a highly heterogeneous system that integrates diverse technologies and communication paradigms, encompassing diverse consumer electronics (CE) devices, including human-type user devices and various internet of things (IoT) devices utilizing different protocols. In this paper, a novel cognitive unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) heterogeneous network (HetNet) is proposed, where multiple cognitive UAVs equipped with MEC servers assist human-type users within the same signal modulation scheme, establishing one communication link, while selectively collecting data transmitted by the constrained application protocol (CoAP)-based IoT devices within different signal modulations, forming another communication link. Moreover, automatic modulation classification (AMC) technologies enable cognitive UAVs to communicate efficiently with multiple CoAP-based IoT devices. In this work, the reparameterization causal convolutional network (RepCCNet) is innovatively utilized to enable each cognitive UAV to achieve AMC for IoT devices. To fulfill the demand for high uplink data rate while minimizing overall system energy consumption, a superior multi-agent algorithm (RepCCNet-D3QN) is proposed to solve the problem of maximizing average energy efficiency, dueling double deep Q-network (D3QN) is applied to optimize user association factors and the three-dimensional (3D) trajectory of UAVs. Compared with other recent deep reinforcement learning (DRL) algorithms, the RepCCNet-D3QN algorithm can provide 45% energy efficiency improvement.