SimGroup-FS: Multi-Way Redundancy Modeling for Interpretable Feature Selection in Supervised Learning

Feature selection is a cornerstone of high-dimensional supervised learning, critically influencing model accuracy, computational efficiency, and interpretability. Conventional filter-based methods, however, predominantly rely on univariate relevance or pairwise redundancy measures, which are inherently limited in capturing complex, multi-way dependencies among three or more features. To address this gap, we propose SimGroup-FS, a graph-theoretic feature selection framework that explicitly models structural redundancy through connected component analysis. The method constructs a cosine-similarity graph over the feature set, applies a similarity threshold ϵ to define edges, and identifies redundancy groups as connected components. From each group, a single representative feature is selected based on its discriminative relevance to the target variable, using flexible metrics such as Spearman correlation, mutual information, or ANOVA F-values. Evaluated on six diverse UCI benchmark datasets—including hybrid and purely numerical types—SimGroup-FS achieves an average dimensionality reduction of 88.8%, with reductions exceeding 96% on datasets likewdbcandionosphere. Crucially, classification performance is consistently preserved or significantly enhanced across multiple classifiers (SVM, Random Forest, and k-NN). Most notably, on thewdbcdataset, the SVM F1-score improves from a near-failure level of 0.019 to a robust 0.887, demonstrating the method’s ability to mitigate noise-induced model degradation. SimGroup-FS operates inO(m2n)time, is model-agnostic, fully reproducible, and yields interpretable feature groupings.