A Comprehensive Study of Multiple Linear Regression with Ridge and LASSO Regularization Theory, Diagnostics, and Applied Analysis

Read full text Summarise with VIRAT
Authors: Heyam A. A. Hayawi
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This study presents an extended analysis of the multiple linear regression model by integrating classical estimation techniques with modern regularization methods. While Ordinary Least Squares (OLS) remains a fundamental approach for parameter estimation, its performance deteriorates in the presence of multicollinearity, outliers, and influential observations. To address these limitations, this research incorporates Ridge and LASSO regression methods as robust alternatives to OLS. Ridge regression stabilizes parameter estimates by introducing an penalty, whereas LASSO regression enhances model interpretability through variable selection using an penalty. The study begins with the theoretical formulation of the multiple linear regression model and the matrix derivation of OLS estimators. It then examines the classical assumptions and highlights the impact of multicollinearity on model stability. Diagnostic tools, including residual analysis, leverage statistics, and Cook’s distance, are employed to detect outliers and influential observations. An empirical dataset is analyzed to compare the performance of OLS, Ridge, and LASSO models. The results demonstrate that while OLS is sensitive to irregular observations, Ridge reduces variance and improves stability, and LASSO provides a more parsimonious model by eliminating irrelevant predictors. The findings emphasize the importance of combining diagnostic techniques with regularization methods to achieve reliable estimation and improved predictive performance in regression analysis.

Fields of Study & Subjects

Advanced Statistical Methods and Models Statistical Methods and Applications Statistical Methods and Inference

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
22 March 2026
Language
EN