5.2 Multiple Linear Regression

Multiple linear regression (MLR) extends simple linear regression to model the relationship between one dependent variable and multiple independent variables. It is widely used in econometrics, business analytics, and social sciences to identify trends and make predictions, helping to interpret complex data and inform decision-making.

Application of Multiple Linear Regression

MLR can be used to examine how various factors such as education level, years of work experience, and industry type collectively impact an individual's salary. Alternatively, it can assess how variables like advertising expenditure, pricing strategies, and market competition affect a company's sales revenue. This ability to incorporate multiple predictors enables researchers to capture the complexity of real-world scenarios more effectively.

Below is a graph showing how the data is collected using a 3D graph, with three different variables:

**Figure 1.** 3D plot illustrating the relationship between miles per gallon (MPG), weight, and horsepower of vehicles. The surface plot highlights the trend that, as weight and horsepower increase, MPG tends to decrease, showing the combined effect of these two variables on fuel efficiency.

Model Specification

In MLR, the relationship is expressed as:

Y = β_{0} + β_{1} X_{1} + β_{2} X_{2} + \dots + β_{k} X_{k} + ε

Equation 1. The multiple linear regression model, where

Y

is the dependent variable,

β_{0}

is the intercept,

β_{1}

β_{2}

, ...,

β_{k}

are the coefficients of the independent variables,

X_{1}

X_{2}

, ...,

X_{k}

are the independent variables and

ε

is the error term.

Estimation of Coefficients

Estimation of coefficients in multiple linear regression (MLR) is typically done using the Ordinary Least Squares (OLS) method, which minimises the sum of the squared residuals (the differences between observed and predicted values).

The estimated coefficients provide insight into the strength and direction of the relationship between each independent variable and the dependent variable. Each coefficient, denoted $β_{i}$ , indicates how much the dependent variable $Y$ is expected to increase (if $β_{i}$ is positive) or decrease (if $β_{i}$ is negative) when the corresponding independent variable $X_{i}$ increases by one unit, assuming all other variables remain constant.

Measuring the Goodness of Fit of an MLR Model

The goodness-of-fit of an MLR model is often measured using R-squared ( $R^{2}$ ), which shows how much of the variation in the dependent variable can be explained by the independent variables. An $R^{2}$ value closer to 1 means the model fits well, while a value closer to 0 means the model doesn’t explain much of the variation.

Statistical Significance

Statistical significance is important in MLR because it helps determine whether the independent variables actually contribute to explaining the dependent variable. T-tests check the significance of individual coefficients, with a low p-value (usually < 0.05) showing a strong relationship between an independent variable and the dependent variable. The F-test looks at the overall model, indicating that at least one independent variable is important if its p-value is low. These tests help researchers identify which factors truly affect the dependent variable, guiding their analysis and decisions.

MLR Assumptions

For MLR to provide valid results, certain assumptions must be met. The relationship between the independent and dependent variables should be linear (linearity), observations should be independent of one another (independence), the variance of the residuals should be constant across all levels of the independent variables (homoscedasticity), and the residuals should be approximately normally distributed (normality).

Limitations of MLR

Despite its usefulness, MLR has several limitations. It is sensitive to outliers, which can skew results and lead to misleading coefficient estimates and interpretations. Multicollinearity, which occurs when independent variables are highly correlated with one another, can make it difficult to determine the individual effect of each variable on the dependent variable and inflate standard errors, resulting in less reliable statistical significance. If the assumptions of linearity, independence, homoscedasticity, and normality are violated, MLR can produce biassed estimates and invalid conclusions.

Overfitting is another issue, where including too many independent variables captures noise in the data rather than the underlying relationship, reducing the model's ability to generalise to new data. Lastly, MLR assumes a linear relationship between the dependent and independent variables, and if the true relationship is nonlinear, the model may fail to capture the dynamics accurately, leading to inaccurate predictions.

GCSE

University Taster