A Comparison of Machine Learning Methods for Missing Value Imputation and Their Impact on Statistical Distribution

Missing data presents a tough problem in data analysis, with the potential to throw off statistical distributions and lead to incorrect conclusions. Recently, machine learning (ML) methods have become popular for dealing with these gaps. This paper takes a look at different ML approaches for filling in missing values, putting a special focus on the Lazy Learner like K-Nearest Neighbors (KNN) as well as ensemble learning strategies like Bagging using Random Forest and Extra Trees, in addition boosting learners such as Gradient Boosting, Extreme Gradient Boosting, Light Gradient Boosting Machine, and Categorical Boosting. Two datasets from UCI utilized in this paper then we artificially simulate missing value by Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR) with varying proportions of missing values (5% to 25%) and then imputed by several ML methods, therefore the models were evaluated by using root mean squared error (RMSE) and R2, therefore to compare the distribution of original dataset and imputed dataset we use sum squared error (SSE). The experimental results for both datasets indicate that, tree-based methods such as Random Forest and Extra Trees followed by Light GBM demonstrated superior performance for imputing dataset. In contrast, KNN exhibited a marked decline in performance as the proportion of missing data increased. In our experiments, MCAR generally showed itself to be the most reliable, maintaining a resemblance to the original data and minimizing imputation errors. MNAR, however, displayed considerabledistortions in distribution and SSE, underlining its complexity and diminishing the efficacy of even sophisticated models.