Real-time network security: Integrating ANN and dynamic graph-based clustering

The increasing sophistication and frequency of cyberattacks have made Network Intrusion Detection Systems (NIDS) a critical component of modern cybersecurity. This work presents D-MAGIC, a novel real-time NIDS that leverages zero-shot learning and graph-based dynamic clustering to detect both known and unknown threats. Unlike traditional systems that rely on labeled datasets and predefined attack signatures, D-MAGIC operates unsupervised, identifying anomalies by detecting deviations from normal network behavior. D-MAGIC employs dynamic clustering to identify coordinated attacks and emerging threats by embedding network flow relationships into a graph structure and clustering similar patterns. Additionally, a second anomaly detection method, called SAGA, is introduced. SAGA enhances detection by using approximate nearest neighbor (ANN) analysis to evaluate isolated nodes, which clustering alone cannot classify. By leveraging information from a node’s neighbors, SAGA improves the identification of subtle or emerging threats. This hybrid approach ensures more comprehensive detection, capturing both clustered anomalies and isolated outliers. Experimental results on the CIC-IDS-2017 and CSE-CIC-IDS-2018 datasets demonstrate that D-MAGIC achieves up to 12% improvement in F1 score over state-of-the-art methods, significantly reduces false positives, and ensures rapid, real-time detection with minimal latency. Furthermore, on the CIC-IDS-2017 dataset, SAGA achieves up to a 6% improvement in accuracy, showcasing its effectiveness in handling diverse attack scenarios.