--- title: "Fitting a Model to Data: Least Squares, Chi-Square and Goodness of Fit" description: "Fit models by least squares, using uncertainty weights when appropriate. Report parameter uncertainties under stated error assumptions. Use residuals and chi-square to assess consistency with those as" canonical: https://lightmysky.com/learn/science/fitting-a-model-to-data-least-squares-chi-square-and-goodness-of-fit-mt_NN0WYlp0md source: https://lightmysky.com/learn/science/fitting-a-model-to-data-least-squares-chi-square-and-goodness-of-fit-mt_NN0WYlp0md.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # Fitting a Model to Data: Least Squares, Chi-Square and Goodness of Fit Fit models by least squares, using uncertainty weights when appropriate. Report parameter uncertainties under stated error assumptions. Use residuals and chi-square to assess consistency with those assumptions without treating a good fit as proof of the model. Subject: Science · Area: Scientific Inquiry · Ages 22 to 23 Page: https://lightmysky.com/learn/science/fitting-a-model-to-data-least-squares-chi-square-and-goodness-of-fit-mt_NN0WYlp0md ## Ready when they can - Fits a straight line by weighted least squares and reports parameter uncertainties - Interprets a reduced chi-square value, including what too small a value suggests - Reads structure in the residuals and says what it implies about the model ## Lesson: Fitting lines and judging the fit A residual is observed y minus predicted y-hat at the same x. Ordinary least squares chooses parameters that minimise the sum of squared residuals, called SSE. Squaring prevents opposite signs from cancelling. A fit is possible without reported point uncertainties. Its standard errors and statistical interpretation need justified assumptions about the measurement errors. **Example.** For a constant model y-hat = a with equal weights, the least-squares estimate is the arithmetic mean. Add the readings and divide by their number. Readings 1, 4 and 7 give a = 12/3 = 4, with residuals -3, 0 and 3. The mean need not equal the middle reading. Treat x as exact here. For independent measurement errors with known standard deviations sigma, weighted least squares minimises chi-square: add (residual/sigma)² over all points. Each squared residual has weight 1/sigma². Smaller sigma gives a larger penalty for the same residual. Precision does not rule out bias. Residual and sigma use the same units, making their ratio dimensionless. **Example.** For residuals -3, 0 and 3 with sigma = 2 for every reading, first divide by sigma. The ratios are -1.5, 0 and 1.5. Squaring gives 2.25, 0 and 2.25. Add them to get chi-square = 4.5. A line predicts y-hat = a + b x. Intercept a has y units; slope b has y units per x unit. As in dimensional analysis, check the units of each term. Set w = 1/sigma² and W = sum(w). The weighted means are x-bar = sum(w x)/W and y-bar = sum(w y)/W. Here sum means add over all points. Calculate Cxx = sum[w(x - x-bar)²] and Cxy = sum[w(x - x-bar)(y - y-bar)]. If Cxx > 0, the weighted line has b = Cxy/Cxx and a = y-bar - b x-bar. It passes through the weighted mean point. If every x is equal, Cxx = 0 and separate slope and intercept cannot be determined. For this line, SE(b) = 1/sqrt(Cxx) and SE(a) = sqrt(1/W + x-bar²/Cxx). sqrt means positive square root. These standard errors assume a correct linear mean, independent errors, correct known variances and exact x. Normal errors support normal confidence intervals. A standard error is not a guaranteed bound. Report each coefficient and its standard error with units. If every known sigma doubles, every weight becomes one quarter as large. Relative weights stay unchanged, so the fitted line stays unchanged. W, Cxx and Cxy become one quarter as large. Both coefficient standard errors double, while chi-square becomes one quarter as large. For a full-rank linear model, n measurements and p fitted coefficients give residual degrees of freedom nu = n - p. Full rank means the coefficients can be separately determined. Reduced chi-square is chi-square/nu, requiring nu > 0. With the correct mean and independent normal errors of known correct sigmas, chi-square has mean nu. Reduced chi-square has mean 1 and standard deviation sqrt(2/nu). Near 1 is consistent with these assumptions, not proof. An unusually small reduced chi-square can suggest overstated uncertainties, ignored correlations or data-selection effects. Chance variation matters, especially with few degrees of freedom. Extra coefficients alone do not explain a small value: nu already accounts for fitted coefficients in the stated linear model. Selecting models after seeing the data can change this interpretation. Plot residuals against x and measurement order. Repeated curvature suggests missing model structure or another systematic effect. Check the model and measurement process before choosing a remedy. Chance can produce apparent patterns, and featureless scatter does not prove correctness. Fitted residuals can be correlated even when the original errors are independent. *(drawing: Read these residual signs in increasing x order. This recurring pattern would warrant investigation; it does not identify the cause.)* Check residual spread too. Raw residuals may spread out where sigma is larger, even with a suitable model. Divide each residual by its sigma before comparing spread across x ranges. A remaining widening pattern suggests checking the variance assumptions. This scaling does not remove every effect of fitting. With n > 2 and at least two distinct exact x values, fit an unweighted line. Assume a correct linear mean and independent errors of equal positive variance. Estimate s² = SSE/(n - 2). Evaluate the standard-error formulas with w = 1, then multiply the results by s. Normal errors support t intervals when s > 0. Using s as every sigma gives reduced chi-square 1 only when SSE > 0. If SSE = 0, s = 0 cannot be a chi-square denominator. Investigate unusually large residuals before removing observations. Check records, instruments and measurement conditions. A valid unusual observation may reveal a model limitation. Document any correction or exclusion. Comparing fits with and without a questionable observation can reveal sensitivity; report why you made that comparison. **Recap.** State the error assumptions, fit the coefficients, report their standard errors, and check residuals and chi-square without treating consistency as proof. *Lesson, questions and flashcards written for this dataset.* ## Practice 18 questions on this page, each with its working shown. ## Needs first - [Dimensional Analysis, Scaling and the Order-of-Magnitude Estimate](https://lightmysky.com/learn/science/dimensional-analysis-scaling-and-the-order-of-magnitude-estimate-mt_jXnXRMKnC3) - [Least-Squares Calibration and the Uncertainty of a Result Read From It](https://lightmysky.com/learn/science/least-squares-calibration-and-the-uncertainty-of-a-result-read-from-it-mt_L08U-Extq_) ## Optional connections - [Conditioning, Stability and Floating-Point Error](https://lightmysky.com/learn/mathematics/conditioning-stability-and-floating-point-error-mt_q8IoCElXIV) - [Simple Linear Regression and Inference on the Slope](https://lightmysky.com/learn/mathematics/simple-linear-regression-and-inference-on-the-slope-mt_W2huNTKsMj) - [Chi-Square Tests for Categorical Data](https://lightmysky.com/learn/mathematics/chi-square-tests-for-categorical-data-mt_yxc87pzaXX) ## Opens up - [Systematic Error, Controls and the Design of a Measurement](https://lightmysky.com/learn/science/systematic-error-controls-and-the-design-of-a-measurement-mt_rGjB_JcsLJ)