Multiple Linear Regression: Theory, Matrix Derivation, Diagnostics, and Application

Multiple linear regression is one of the most fundamental and widely applied statistical methods for analyzing relationships between a dependent variable and several explanatory variables. The technique provides efficient estimation of model parameters through the ordinary least squares (OLS) method under a set of classical assumptions. However, the presence of outliers, high leverage points, or influential observations may significantly distort regression estimates and lead to misleading conclusions. This study presents a comprehensive theoretical and practical overview of multiple linear regression. The paper begins with the mathematical formulation of the regression model and its matrix representation, followed by the derivation of the ordinary least squares estimator. Diagnostic techniques are then introduced to detect unusual observations and evaluate model assumptions. These include standardized residuals, leverage statistics, Cook’s distance, and graphical diagnostic tools such as residual plots and Q–Q plots. A practical dataset is analyzed to demonstrate the effect of outliers on regression results and the improvement obtained after removing influential observations. The study highlights the importance of regression diagnostics and recommends careful examination of data prior to model interpretation.