58 Simple Linear Regression
Simple linear regression fits a straight line through a set of paired observations to predict a continuous outcome from a single predictor variable. This chapter works through the least-squares calculation by hand, then reproduces the same result using R and Python.
58.1 The Least-Squares Method
The fitted regression line is: \[ \hat{Y} = b_0 + b_1 X \]
The least-squares slope and intercept are calculated as: \[ b_1 = \frac{\sum (X_i - \bar{X})(Y_i - \bar{Y})}{\sum (X_i - \bar{X})^2} \qquad\qquad b_0 = \bar{Y} - b_1 \bar{X} \]
58.2 Assessing Model Fit: R-squared
The coefficient of determination, \(R^2\), measures the proportion of variance in \(Y\) explained by \(X\): \[ R^2 = 1 - \frac{\sum (Y_i - \hat{Y}_i)^2}{\sum (Y_i - \bar{Y})^2} \] \(R^2\) ranges from 0 to 1; an \(R^2\) of 0.80 means that 80% of the variation in the outcome is explained by the predictor. In simple linear regression, \(R^2\) is also equal to the square of the Pearson correlation coefficient between \(X\) and \(Y\).
58.3 Worked Example
A company records monthly advertising spend (in ₹ thousands) and the resulting sales revenue (in ₹ lakhs) over 6 months:
| Month | Advertising Spend (X) | Sales Revenue (Y) |
|---|---|---|
| 1 | 10 | 25 |
| 2 | 15 | 32 |
| 3 | 20 | 40 |
| 4 | 25 | 41 |
| 5 | 30 | 50 |
| 6 | 35 | 55 |
Calculate the Means
\[ \bar{X} = \frac{10+15+20+25+30+35}{6} = 22.5 \qquad \bar{Y} = \frac{25+32+40+41+50+55}{6} = 40.5 \]
Calculate the Slope
| \(X_i - \bar{X}\) | \(Y_i - \bar{Y}\) | Product | \((X_i-\bar{X})^2\) |
|---|---|---|---|
| -12.5 | -15.5 | 193.75 | 156.25 |
| -7.5 | -8.5 | 63.75 | 56.25 |
| -2.5 | -0.5 | 1.25 | 6.25 |
| 2.5 | 0.5 | 1.25 | 6.25 |
| 7.5 | 9.5 | 71.25 | 56.25 |
| 12.5 | 14.5 | 181.25 | 156.25 |
\[ \sum (X_i-\bar{X})(Y_i-\bar{Y}) = 512.5 \qquad \sum (X_i-\bar{X})^2 = 437.5 \]
\[ b_1 = \frac{512.5}{437.5} \approx 1.17 \]
Calculate the Intercept and Final Equation
\[ b_0 = 40.5 - (1.17 \times 22.5) \approx 40.5 - 26.36 = 14.14 \]
The fitted regression equation is: \[ \hat{Y} = 14.14 + 1.17\,X \]
Interpretation: For every additional ₹1,000 spent on advertising, sales revenue is predicted to increase by about ₹1.17 lakh. If advertising spend is ₹28,000, predicted sales revenue is \(14.14 + 1.17 \times 28 \approx 46.9\) lakh.
58.4 Simple Linear Regression in R and Python
Transition to Multiple Linear Regression
Simple linear regression uses a single predictor, but business outcomes are rarely driven by just one factor. The next chapter extends this model to multiple linear regression, where several predictors are used together to explain and forecast an outcome.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Simple Linear Regression | Fits a straight line predicting a continuous outcome from a single predictor variable |
| Least-Squares Slope Formula | b1 equals the sum of cross-products of deviations divided by the sum of squared X deviations |
| Least-Squares Intercept Formula | b0 equals the mean of Y minus b1 times the mean of X |
| Fitted Regression Equation | Y-hat equals b0 plus b1 times X, used to predict Y for any value of X |
| Model Fit | |
| R-squared (Coefficient of Determination) | The proportion of variance in Y explained by X, ranging from 0 to 1 |
| R-squared and Correlation | In simple regression, R-squared equals the square of the Pearson correlation coefficient |
| Interpretation | |
| Interpreting the Slope | The predicted change in Y for each one-unit increase in X |
| Interpreting the Intercept | The predicted value of Y when X equals zero |
| Prediction from the Model | Substituting a new X value into the fitted equation to forecast Y |
| In R and Python | |
| R lm() | Base R function used to fit linear regression models and extract coefficients and R-squared |
| Python scipy.stats.linregress() | SciPy function that fits a simple linear regression and returns slope, intercept, and r-value |