--- title: "Linear Regression by Least Squares" description: "Fit a straight relationship by choosing coefficients that make the sum of squared residuals as small as possible. With several features the same idea gives a plane or hyperplane, and each coefficient " canonical: https://lightmysky.com/learn/computing/linear-regression-by-least-squares-mt_QRRkpIrwWs source: https://lightmysky.com/learn/computing/linear-regression-by-least-squares-mt_QRRkpIrwWs.md retrieved: 2026-09-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # Linear Regression by Least Squares Fit a straight relationship by choosing coefficients that make the sum of squared residuals as small as possible. With several features the same idea gives a plane or hyperplane, and each coefficient reads as the effect of one feature with the others held fixed. Subject: Computing · Area: Machine Learning · Ages 19 to 20 Page: https://lightmysky.com/learn/computing/linear-regression-by-least-squares-mt_QRRkpIrwWs ## Ready when they can - Fit a line by least squares and interpret its slope in the units of the data - Extend to two features and say what each coefficient means - Plot residuals and say what a pattern in them reveals about the fit ## Lesson: Drawing the line that misses least For each point, the vertical gap to your line is called a residual. Squaring each gap keeps pluses and minuses from canceling, and the least squares line is the one with the smallest total of squared gaps. **Example.** Write the line as y hat equals a plus b x. The slope b says how much predicted y moves when x rises by one, in the units of your data: with s equals 52 plus 6.4 h, each extra study hour links to about 6.4 more exam points, and at 5 hours the prediction is 84. The intercept a is the prediction when x is zero. With two features the model is y hat equals b0 plus b1 x1 plus b2 x2, and each coefficient is the effect of its feature while the others stay fixed. Plot the residuals to check the shape: random scatter around zero means the line fits, while a curve or fan means the line misses something and you should rethink. **Tip.** Always attach units before explaining a coefficient, and remember the least squares line always passes through the point of the two means. Refitting on resampled data shows the wobble: a coefficient far from zero across resamples likely carries a real signal. **Recap.** Minimize squared gaps, read each slope per unit with the rest fixed, and let residuals judge the fit. ## Practice 12 questions on this page, each with its working shown. ## Needs first - [Data as a Matrix: Rows, Features and the Target](https://lightmysky.com/learn/computing/data-as-a-matrix-rows-features-and-the-target-mt_KEN-TU1Gpn) - [Gaussian Elimination and Row Echelon Form](https://lightmysky.com/learn/mathematics/gaussian-elimination-and-row-echelon-form-mt_PZBce3UvRm) - [Correlation and the Product-Moment Coefficient](https://lightmysky.com/learn/mathematics/correlation-and-the-product-moment-coefficient-mt_Rs_Z0P7ELJ) - [The Regression Line and Making Predictions](https://lightmysky.com/learn/mathematics/the-regression-line-and-making-predictions-mt_tsjctRmkyE) ## Opens up - [Gradient Descent: Following the Slope Downhill](https://lightmysky.com/learn/computing/gradient-descent-following-the-slope-downhill-mt_EjfIT1WgMX)