Evolutionary Feature Engineering Meets AutoML: A Feature-Efficient Framework for Malware Detection

The rapid evolution of malicious software continues to challenge modern cybersecurity systems, driven by increasingly sophisticated evasion techniques such as polymorphism, packing, and code obfuscation. While machine learning has proven effective in detecting previously unseen malware, its practical adoption is often constrained by high-dimensional feature spaces and the complexity of model and hyperparameter selection. This paper presents an evolutionary AutoML-based framework that integrates Genetic Algorithm (GA) feature selection with Automated Machine Learning (AutoML) optimization for feature-efficient malware detection. The proposed approach employs a GA-driven search to identify a compact and informative subset of features from the high-dimensional EMBER dataset, effectively eliminating redundant attributes while preserving discriminative capability. The resulting reduced feature representation is then utilized by an AutoML pipeline to automatically discover an optimized classification model and hyperparameter configuration. Experimental results demonstrate that the proposed framework reduces the original feature space from 2381 to approximately 400 features while maintaining strong predictive performance. The optimized GA + AutoML pipeline achieves an accuracy of 96.9% with an ROC-AUC approaching 0.99. Although marginally lower than full-feature baselines, the performance remains competitive given the substantial reduction in dimensionality. Furthermore, ablation analysis confirms that the combination of evolutionary feature selection and automated model optimization provides complementary benefits, offering a robust trade-off between accuracy and efficiency. These findings highlight the effectiveness of the proposed framework in achieving a strong balance between predictive performance and feature-space reduction under benchmark conditions. However, further evaluation under temporal and cross-domain settings is required to fully assess real-world generalization.