Energy Efficient Multi-UAV Trajectory Planning in Multi-Hotspot Scenarios: A Large Language Model Enhanced MARL Approach

The adaptability and distributed lightweight decision-making capabilities of multi-agent reinforcement learning (MARL) has been widely leveraged in multiple UAV assisted communications for trajectory planning. However, in multihotspot scenarios, existing MARL based algorithms suffer from sparse rewards and inefficient credit assignment, leading to poor learning efficiency in UAVs and constraining the optimization of energy efficiency in multi-UAV trajectory planning. Notably, Large Language Models (LLMs), leveraging their powerful reasoning capabilities, have demonstrated great potential to optimize objectives of complex systems. However, it is not trivial to deploy LLMs on UAVs with low computational resources. To address these challenges, we propose a novel LLM-enhanced Multi-Agent Proximal Policy Optimization algorithm (LLM-MAPPO) for multi-UAV trajectory planning. In LLM-MAPPO, we first design a Chain-of-Thought (CoT) prompt to guide the LLM in reasoning to enhance the energy efficiency of multiple UAVs and generate high-quality expert trajectory policies with superior credit assignment capabilities. Then, we develop a knowledge distillation and policy fine-tuning integrated dualphase update algorithm. This algorithm enables MAPPO agents to inherit knowledge from expert policies, thereby enhancing credit assignment and addressing the sparse reward problem. LLM-MAPPO integrates the reasoning advantages of LLMs with the lightweight decision-making attributes of MAPPO agents, significantly enhancing the global energy efficiency of multi-UAV trajectories while ensuring deployability on UAVs. Empirically, we demonstrate that LLM-MAPPO reduces training time costs by 67%. Additionally, LLM-MAPPO significantly outperforms compared algorithms among various performance metrics and improves the total energy efficiency metric by 194%.