League Training-Based Dual Optimization Method for UAV Maneuver Decision-Making

Currently, most existing learning-based autonomous maneuver decision-making methods for Uncrewed Aerial Vehicles (UAVs) focus on network weights optimization, overlooking the critical role of network architecture optimization in enhancing decision-making performance. To address this issue, we propose a dual optimization method that improves autonomous maneuver decision-making for high-performance UAVs through concurrently learning network weights and architecture. Specifically, we utilize the recurrent Soft Actor-Critic (SAC) algorithm and Differentiable Architecture Search (DARTS) to optimize the weights and discover the optimal architecture of decision-making policy, respectively. Furthermore, we designed a league training mechanism that regards the policies with the discovered architecture as league players to further optimize their weights. The maneuver performance of league players is evaluated to facilitate the architecture search efficiency. Lastly, we validate the stability and generalization of the proposed method through a series of simulations. The results demonstrate that our method exhibits superior maneuver decision-making capabilities, outperforming existing algorithms in terms of win rates.