Interpretable by Design: MH-AutoML for Transparent and Efficient Android Malware Detection without Compromising Performance

Malware detection in Android systems requires both cybersecurity domain knowledge and expertise in machine learning (ML). Automated Machine Learning (AutoML) aims to simplify ML development by automating key stages of the pipeline, including pipeline construction, algorithm selection, and hyperparameter optimisation, thereby reducing the need for specialised expertise. However, most general-purpose AutoML frameworks operate largely as black-box systems with limited transparency, interpretability, and experiment traceability. These limitations are particularly problematic in security-sensitive applications, where understanding model behaviour and ensuring reproducibility are essential. To address these challenges, we present MH-AutoML, a meta-heuristic-based AutoML framework designed specifically for Android malware detection. MH-AutoML automates the complete ML pipeline, covering data preprocessing, feature engineering, model selection, and hyperparameter optimisation. In addition, the framework incorporates built-in capabilities for interpretability, debugging, and experiment tracking, which are often absent or only partially supported in existing AutoML solutions. We perform a large-scale empirical evaluation comparing MH-AutoML with seven established AutoML frameworks (Auto-sklearn, AutoGluon, TPOT, HyperGBM, AutoPyTorch, LightAutoML, and MLJAR). The comparison is conducted across nine Android malware benchmark datasets evaluated under both original and class-balanced distributions using 10 random seeds, resulting in a total of 1,440 experimental runs. Using the Matthews Correlation Coefficient (MCC) as the primary evaluation metric, together with Friedman–Nemenyi statistical tests and pairwise Win/Tie/Loss analysis, the results reveal several key findings. First, MLJAR and LightAutoML form a consistently top-performing tier, with MLJAR achieving the highest aggregate dominance score across the analyses. Second, LightAutoML offers the most favourable efficiency–performance trade-off, reaching comparable predictive quality in approximately 9–17 minutes compared with MLJAR’s execution time of about one hour. Third, MH-AutoML achieves competitive recall performance, ranking first on three datasets including Adroit, Androcrawl, and MH-100, although this performance is accompanied by reduced MCC values due to elevated false-positive rates. Fourth, class balancing substantially increases the statistical separation among frameworks, doubling the Friedman test statistic and revealing fragile behaviour in certain tools. AutoPyTorch is the most notable case, collapsing from third position in the Original setting to the last rank under balanced distributions. Finally, no single framework dominates across all datasets, which reinforces the importance of selecting AutoML frameworks according to the specific characteristics of each dataset.