Malware Detection Using RNA Encoding and Convolutional Neural Networks on the Malicious Network Dataset

Background The detection of malware in network traffic remains a critical cybersecurity challenge. Traditional signature-based intrusion detection demonstrates a high level of familiarity with issues that have been recorded in the database; but show significantly lower effectiveness when it comes to polymorphic or zero-day attacks. Conversely, anomaly-based approaches are also endowed with the ability to detect new incursions, but often have a high false-positive rate. Methods This study proposes a combined malware-detection framework which makes use of RNA encoding network-flow attributes alongside Convolutional Neural Network (CNN) classifiers. The framework has three functionalities: a Signature-CNN, which is trained on RNA-encoded representation of known malicious flows; an Anomaly-CNN, which is developed to distinguish between benign and malicious traffic without any signature prior knowledge; and a Hybrid-CNN, which combines both paradigms in a two-stage detection pipeline. Results Experiments conducted on the Malicious Network Dataset show that the Signature-CNN achieves 91% accuracy with strong precision on known threats, the Anomaly-CNN achieves 93% detection rate on unknown malware, and the Hybrid-CNN achieves the best overall performance with 95% detection rate and 94.5% F1 score. Conclusions The results demonstrate that RNA encoding combined with CNN classifiers offers a robust and scalable solution for malware detection in networked environments.