Greedy Feature Selection with Iterative Hierarchical Binning

This paper proposes a Greedy Feature Selection using Hierarchical Binning (GFS-HB) algorithm to address the computational inefficiency and feature redundancy problems of conventional greedy feature selection (GFS) methods. The proposed method groups features into multiple bins based on their importance scores and progressively refines the optimal subset through an iterative re-binning process in which bins are sequentially evaluated and selected. This hierarchical binning structure significantly reduces the number of evaluations while maintaining or improving classification performance. The performance of the proposed GFS-HB algorithm was validated using four high-dimensional datasets — Madelon, CLL_SUB_111, Lung, and Tox-171 — and three classifiers: Decision Tree, Random Forest, and XGBoost. Experimental results show that GFS-HB achieves higher classification accuracy than both the original classifiers and the conventional GFS method, while reducing the number of selected features and shortening the feature selection time by up to 100×. These findings demonstrate that GFS-HB provides an efficient and generalizable (feature-agnostic) framework for feature selection in high-dimensional data environments.