Regression Analysis

This chapter discusses bivariate regression and approaches to multivariate regression analysis. We begin by introducing the concept of cause and effect, predictor and outcome, independent and dependent variables. We then explain the assumptions of linear regression analysis and hypotheses in the context of bivariate regression. Then we proceed to explain the concept of multivariate regression and its additional assumption of absence of multicollinearity. We then explain approaches to examining the absence of multicollinearity. In the context of the term “linear”, we explain the difference between linearity in parameters and linearity in variables, and that “linear” means linearity in parameters. We focus our discussion in this chapter on linearity in parameters and, though, we explain polynomial regression (non-linearity in variables). Since we saw that our bivariate regression model could be linear or non-linear in variables, we explain the methods to compare the regression models and choose the one based on parsimony/complexity. We then discuss a stepwise approach to multivariate regression and explain the model diagnostics to ensure the validity and reliability of the model. We end the chapter with a brief discussion on logarithmic transformation of variables and their implications for the interpretation of the results.