Impact of Feature Scaling and Regularization on the Performance of Machine Learning Models

This research paper investigates the effect of feature scaling and regularization techniques on the performance of machine learning classification models. Using the publicly available UCI Diabetes Dataset, models including Logistic Regression, Support Vector Machines (SVM), and Random Forests were trained and evaluated under different preprocessing and regularization settings. The study focuses on numerical stability, convergence behavior, and predictive performance, emphasizing the mathematical foundations behind optimization and regularization in modern machine learning systems. Experimental results demonstrate that proper feature scaling and regularization significantly improve model accuracy, F1-score, and overall stability, highlighting best practices for applied data science and machine learning tasks. The dataset contains medical diagnostic features such as glucose level, blood pressure, BMI, insulin level, and age, with the target variable indicating whether a patient is diabetic or not. The paper includes tables, bar charts, and Python code to reproduce all experiments, making the study fully reproducible. This preprint is intended for researchers, students, and practitioners in Data Science, Machine Learning, and Applied Mathematics, and serves as a reference for scholarship applications, academic projects, and educational purposes.