A Hybrid Imbalanced DDoS Detection Framework Utilizing CNN, LSTM, and K-Means SMOTE

Cyberattacks remain a highly disruptive threat to modern networks. However, the imbalanced nature of real-world network traffic, where attack data constitute only a small fraction, poses significant challenges for accurate detection. This study proposes a hybrid deep learning framework that combines Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) models with a K-means Synthetic Minority Oversampling Technique (SMOTE) to address class imbalance in penetration testing data. A total of 1,532,029 network flow records were collected during penetration testing, comprising 1,230,487 benign flows (80.4%) and 301,542 malicious flows (19.6%), which represented Distributed Denial of Service (DDoS) attacks, including SYN floods, UDP floods, and ICMP floods. The CNN component extracts spatial features from network flows, while the LSTM captures their temporal dependencies. K-means SMOTE enhances detection by generating realistic synthetic samples for minority attack classes. The experimental results show that the CNN-LSTM model with K-means SMOTE achieves a DDoS detection recall of 94.59% and an F1-score of 89.45%, significantly outperforming the imbalanced baseline, with a recall of 64.35% and an F1-score of 73.05%, as well as other classifiers such as Support Vector Machine (SVM) and Random Forest (RF). These findings demonstrate the model's robustness and practicality in detecting minority-class attacks under real-world conditions.