Supervised Model-Based Feature Selection on Diverse Dataset

Feature selection plays a vital role in enhancing the performance of machine learning models by identifying the most informative attributes and removing irrelevant or redundant ones. This paper introduces a supervised feature selection framework that integrates three major approaches Filter, Wrapper, and Embedded methods to improve model accuracy and interpretability. The filter-based stage utilizes Minimum Redundancy Maximum Relevance (MRMR) and Relief techniques, while the wrapper stage employs forward feature selection and random forest strategies. The embedded phase combines feature importance ensemble methods with LASSO regularization to identify optimal subsets. The proposed framework is validated on two benchmark datasets Breast Cancer and Wine using standard evaluation metrics. Results demonstrate that the LASSO-based embedded model achieves the highest performance, attaining 99% accuracy on the Breast Cancer dataset, outperforming other models.