Shield-X: Vectorization and Machine Learning-Based Pipeline for Network Traffic Threat Detection

This paper presents an integrative methodology combining advanced network packet vectorization techniques, parallel processing with Dask, GPU-optimized machine learning models, and the Qdrant vector database. Our approach achieves a 99.9% detection rate for malicious traffic with only a 1% false-positive rate, setting new performance benchmarks for cybersecurity systems. The methodology establishes an average detection time limit not exceeding 10% of the total response time, maintaining high precision even for sophisticated attacks. The system processes 56 GB of PCAP files from Malware-Traffic-Analysis.net (2020–2024) through a five-stage pipeline: distributed packet processing, feature extraction, vectorization, vector database storage, and GPU-accelerated classification using XGBoost, Random Forest, and K-Nearest Neighbors models.