Linear Regression by Least Squares
Fit a straight relationship by choosing coefficients that make the sum of squared residuals as small as possible. With several features the same idea gives a plane or hyperplane, and each coefficient reads as the effect of one feature with the others held fixed.
What a learner can do afterwards
- Fit a line by least squares and interpret its slope in the units of the data
- Extend to two features and say what each coefficient means
- Plot residuals and say what a pattern in them reveals about the fit
1 · Read
For each point, the vertical gap to your line is called a residual. Squaring each gap keeps pluses and minuses from canceling, and the least squares line is the one with the smallest total of squared gaps.
Write the line as y hat equals a plus b x. The slope b says how much predicted y moves when x rises by one, in the units of your data: with s equals 52 plus 6.4 h, each extra study hour links to about 6.4 more exam points, and at 5 hours the prediction is 84. The intercept a is the prediction when x is zero.
With two features the model is y hat equals b0 plus b1 x1 plus b2 x2, and each coefficient is the effect of its feature while the others stay fixed. Plot the residuals to check the shape: random scatter around zero means the line fits, while a curve or fan means the line misses something and you should rethink.
Always attach units before explaining a coefficient, and remember the least squares line always passes through the point of the two means. Refitting on resampled data shows the wobble: a coefficient far from zero across resamples likely carries a real signal.
Minimize squared gaps, read each slope per unit with the rest fixed, and let residuals judge the fit.
2 · Watch
Take it off screen
Where it sits
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.