Hierarchical Route Planning Framework and MMDQN Agent-Based Intelligent Obstacle Avoidance for UAVs

Efficient route planning technology is the core support for ensuring the successful execution of unmanned aerial vehicle (UAV) flight missions. In this paper, the coordination issue of global route planning and local real-time obstacle avoidance in complex mountainous environments is studied. To deal with this issue, a hierarchical route planning framework is designed, including global route planning and AI-based local route re-planning using deep reinforcement learning, exhibiting both flexible versatility and practical coordination and deployment efficiency. Throughout the entire flight, the local route re-planning task triggered by dynamic threats can be executed in real time. Meanwhile, a multi-model DQN (MMDQN) agent with a Monte Carlo traversal iterative learning (MCTIL) strategy is designed for local route re-planning. Compared to existing methods, this agent can be directly used to generate local obstacle avoidance routes in various scenarios at any time during the flight, which simplifies the complicated structure and training process of conventional deep reinforcement learning (DRL) agents in dynamic, complex environments. Using the framework structure and MMDQN agent for local route re-planning ensures the safety and efficiency of the mission, as well as local obstacle avoidance during global flights. These performances are verified through simulations based on actual terrain data.