A Systematic Review of Agentic AI for Threat Detection and Mitigation in 5G Networks

Fifth-generation (5G) networks face escalating security challenges driven by decentralised architectures, stringent ultra-low-latency requirements, and rapidly evolving threat landscapes. Agentic Artificial Intelligence (agentic AI) autonomous systems that perceive network conditions, decide on countermeasures, and act in real time offers a promising route toward adaptive defence. This systematic review examines how agentic AI is being applied to detect and mitigate threats within 5G networks. Following PRISMA 2009 guidelines, four databases (IEEE Xplore, ACM Digital Library, SpringerLink, and ScienceDirect) were searched, yielding 22 eligible peer-reviewed studies published between 2020 and 2025, selected for explicit 5G relevance and empirical evaluation. The reviewed evidence clusters into four primary security areas: anomaly detection, DDoS mitigation, network slicing security, and intrusion detection. Across these domains, approaches based on federated learning, deep reinforcement learning, and multi-agent systems generally report stronger detection performance and/or more adaptive response behaviour than conventional, reactive baselines, while supporting privacy-preserving intelligence at the edge. However, key deployment barriers remain: 86% of studies rely on simulation-based validation, scalability beyond 100 nodes is insufficiently characterised, and reported coordination delays (120–180 ms) may conflict with 5G latency constraints in time-critical settings. To consolidate findings, this review proposes a Perception–Decision–Action–Feedback conceptual framework and highlights priorities for real-world validation and deployment-oriented evaluation.