Multi-AAVs Collision-Free Flocking and Navigation in Dynamic Environments

Multiple autonomous aerial vehicles (AAVs) system is consisting of multi-AAVs through collaborative techniques to efficiently accomplish various tasks in complex scenarios. Collision free flocking and navigation strategies for multi-AAVs in complex dynamic environments are still challenging in several aspects: real-time collision avoidance of dynamic obstacles, complexity of collaborative multi-AAV path planning, and group stagnation problem due to local optimization in dynamic environments. Traditional flocking algorithms usually rely on strict rules with limited adaptability to different environments. While Reinforcement Learning (RL), as a flexible and model-free framework, is able to solve some of the problems through autonomous exploration, it still suffers from challenges such as sample inefficiency, unstable training process, easy to fall into local optimums of the policy, and difficult to converge in sparse reward scenarios. In order to improve the training efficiency, we propose an improved algorithm called Curriculum-based Heuristic Guidance TD3 (CHTD3) for solving collision-free flocking and navigation of multiple AAVs in dynamic environments. The core idea is to decompose the task of AAV collision avoidance and navigation in dynamic environments into multiple sub-tasks and gradually increase the number of sub-tasks in a staged manner, while combining the heuristic rules with the Twin Delayed Deep Deterministic policy gradient (TD3) to solve the problem of prolonged learning cycle commonly found in the traditional RL methods. In addition, to better cope with complex and dynamic environments, we customize a risk assessment criterion and introduce a Human-AAV Collaboration (HAC) mechanism to enable the human operator to intervene at necessary moments to assist the AAV's motion decision. The experimental results demonstrate the effectiveness of the CHTD3-HAC approach in multi-AAVs collision-free flocking and navigation.