A Hybridized Feature Construction Method Based on Symbolic Transformers and Evolutionary Forest for Regression

Deriving meaningful and discriminative features from raw data is a critical step in improving model performance in regression problems. However, existing evolutionary approaches, such as Genetic Programming and Symbolic Transformers, often suffer from bloat and early convergence due to the complexity of the search space. These issues can lead to high multicollinearity among the generated features and reduced model stability. To address these limitations, this study proposes a hybrid feature construction architecture that combines the global mathematical expression capability of Symbolic Transformers with the local decision structure of Evolutionary Forests. The proposed method is comprehensively evaluated on 18 benchmark datasets using eight regression algorithms, including Random Forest, XGBoost, and LightGBM. Experimental results show that the hybrid approach, particularly when paired with ensemble models incorporating internal feature selection mechanisms, yields statistically significant improvements in prediction accuracy and model stability compared to the individual methods.