Fitting a Model to Data: Least Squares, Chi-Square and Goodness of Fit
A fit chooses parameters that minimise a weighted sum of squared residuals, and the size of that sum says whether the model is believable at all. Without uncertainties on the points, neither the parameters nor the verdict means anything.
What a learner can do afterwards
- Fits a straight line by weighted least squares and reports parameter uncertainties
- Interprets a reduced chi-square value, including what too small a value suggests
- Reads structure in the residuals and says what it implies about the model
1 · Read
A residual is the gap between one observed value and the value your model predicts there: y minus y-hat. The method of least squares chooses parameters that make the sum of the squared residuals as small as possible. Squaring makes misses above and below the line both count, and big misses count extra. Without uncertainties on the points, neither the fitted parameters nor any verdict on the model means anything.
You measure one temperature three times and get 2.0, 3.0, and 4.0 degrees with equal uncertainties. Your model is just a constant, y equals a. The sum of squared residuals is smallest at a equals 3, the middle of the three readings. The residuals there are 1, negative 2, and 1, each with uncertainty 1, so the chi-square sum is 1 plus 4 plus 1, which is 6.
The chi-square statistic weights each point fairly: divide each residual by that point's uncertainty, square it, then add all the terms. That is why points measured more precisely pull the fit harder than sloppy points. A reduced chi-square near 1 means the typical miss is about one sigma, so the model fits about as well as the stated uncertainties say it should. A value far below 1 suggests the uncertainties were overestimated, or the model has so many parameters it merely follows the noise. And if the residuals show a visible pattern, like a U shape from a straight line fitted to curved data, the model is systematically wrong: a good fit leaves only random scatter.
Always plot the residuals before you believe a fit. Random scatter around zero means the model shape is plausible, while curves or drifts mean you need a different model. Report the parameter uncertainties alongside the parameters, since a value without its uncertainty cannot guide any decision.
Minimize the squared misses, weight by the uncertainties, and let chi-square plus the residual pattern judge the model.
2 · Watch
Take it off screen
Where it sits
Learn first
Where this leads
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.