Investigating Multi-UAV Territory Guarding Using Cooperative-Competitive Reinforcement Learning

This paper explores the use of Multi-Agent Reinforcement Learning (MARL) to address the UAV territory guarding problem. The use of MATD3 with a separate agent replay buffer and critic networks for intruders and defenders is determined to demonstrate significant computational and performance improvements over MATD3 with centralized training and experience pools. To assess the impact of imperfect observations on UAV agent training, a more realistic radar measurement and detection model, based on search radar theory, is formulated and compared to an idealized observation model. Additionally, a conditional defender reward function, which prioritizes tracking the midpoint between intruder and target base for better adaptation of interception, is found to consistently outperform a pure interception reward function for both risk-taking and risk-averse intelligent intruders in both idealized and radar model environments. To address the sensitivity of training to initial hyperparameters, evolutionary hyperparameter optimization (EHPO) is employed to tune both the intruder and the defender agent networks within the training loop.