5.1 Simple Linear Regression Analysis

Simple linear regression is a foundational statistical method used to explore the relationship between two variables: a dependent variable and an independent variable. The primary goal is to determine how changes in the independent variable affect the dependent variable. By establishing this relationship, we can make informed predictions about the dependent variable based on the values of the independent variable.

The relationship in simple linear regression is encapsulated by the regression equation:

More specifically:

• \(y\) represents the dependent variable: This is the outcome we want to predict. For instance, it could be sales revenue, test scores, or any measurable factor we are interested in.
• \(x\) is the independent variable: This is the predictor that we manipulate or observe, such as advertising spend, hours of study, or temperature.
• \(\beta_0\) (the intercept): This coefficient indicates the expected value of \(y\) when \(x\) is zero. It represents the baseline level of the dependent variable without the influence of the independent variable. In practical terms, it gives us a starting point from which to understand changes in \(y\).
• \(\beta_1\) (the slope): This coefficient shows the change in \(y\) for each one-unit change in \(x\). A positive value indicates that as \(x\) increases, \(y\) also increases, while a negative value suggests that an increase in \(x\) leads to a decrease in \(y\). Understanding the slope helps us gauge the strength and direction of the relationship.
• \(\varepsilon\) (the error term): This component accounts for the variation in \(y\) that cannot be explained by \(x\). It represents all other factors affecting \(y\) not included in the model and captures random fluctuations or measurement errors.

In essence, simple linear regression helps us answer questions like: "How does an increase in hours studied impact exam scores?" or "What effect does temperature have on ice cream sales?"