--- title: "Multiple Regression and Model Diagnostics" description: "Fit a response using several predictors. Interpret each coefficient as fitted response change with the other predictors held fixed. Check collinearity, influential observations and overfitting when as" canonical: https://lightmysky.com/learn/mathematics/multiple-regression-and-model-diagnostics-mt_TUx1tDF0jv source: https://lightmysky.com/learn/mathematics/multiple-regression-and-model-diagnostics-mt_TUx1tDF0jv.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # Multiple Regression and Model Diagnostics Fit a response using several predictors. Interpret each coefficient as fitted response change with the other predictors held fixed. Check collinearity, influential observations and overfitting when assessing the model. Subject: Mathematics · Area: Data & Statistics · Ages 21 to 22 Page: https://lightmysky.com/learn/mathematics/multiple-regression-and-model-diagnostics-mt_TUx1tDF0jv ## Ready when they can - Calculate fitted values and joint fitted-response changes; interpret a coefficient with the other predictors held fixed - Identify near and exact collinearity and influential observations, and explain their effects on the fitted model - Use residual diagnostics and more than training R² to assess model adequacy and compare candidates ## Lesson: Many predictors, honest reading The single-predictor model extends to y = a + b1x1 + b2x2 + e. For the usual exact t inference, errors have mean zero given the predictors, constant variance, independence across observations and conditional normality. In an additive fitted model, a coefficient gives the change in fitted response per unit of its predictor, with the others fixed. This describes association; observational regression alone does not establish causation. **Example.** For fitted y = 4 + 2x1 - 3x2, insert every predictor value. At x1 = 3 and x2 = 1, fitted y is 4 + 6 - 3 = 7. Holding x2 fixed, a one-unit increase in x1 adds 2 to fitted y. If both predictors change, calculate 2 times the change in x1 minus 3 times the change in x2. The intercept cancels when comparing two fitted values. *(drawing: Multiply each predictor by its coefficient, keep its sign, then add the intercept.)* Collinearity means predictors carry overlapping linear information. Strong near dependence can inflate standard errors and make coefficients sensitive to small data changes. Predictions near observed predictor combinations may remain stable. With exact dependence, one design-matrix column is a linear combination of others. The matrix loses rank and ordinary least-squares coefficients are not unique. This can happen without large pairwise correlations. A residual is observed minus fitted y. With an intercept, ordinary least-squares residuals sum to zero, apart from rounding; this does not establish model adequacy. Check residuals for curves and changing spread. An observation is influential if removing it changes the fitted model substantially. Unusual predictor values or a large residual alone do not establish influence. Investigate influential observations; do not delete valid data automatically. *(drawing: A substantial change after refitting shows influence. It does not by itself justify deleting the observation.)* For nested ordinary least-squares fits with an intercept, use the same observations and nonconstant response. Training R² = 1 - SSE/SST, where SST sums squared gaps from mean y. Adding predictors cannot increase SSE: setting their coefficients to zero recovers the old fit. SST stays fixed, so R² cannot fall but may stay equal. Adjusted R² accounts for predictor count and may fall. Model choice also needs residuals, influence, held-out errors and the model's purpose. Overfitting means learning sample-specific noise that predicts poorly on new data. Extra predictors can raise this risk, but higher training R² alone does not establish overfitting. Check predictions on data not used to fit or select the model. Predict near combinations of predictor values represented in the data; being inside each predictor's separate range is not enough. Even nearby predictions depend on model adequacy and uncertainty. **Recap.** Read coefficients with other predictors held fixed. Check collinearity, influence and residuals, and compare performance beyond the training fit. *Lesson, questions and flashcards written for this dataset.* ## Practice 12 questions on this page, each with its working shown. ## Needs first - [Simple Linear Regression and Inference on the Slope](https://lightmysky.com/learn/mathematics/simple-linear-regression-and-inference-on-the-slope-mt_W2huNTKsMj) ## Optional connections - [Dimension and the Rank-Nullity Theorem](https://lightmysky.com/learn/mathematics/dimension-and-the-rank-nullity-theorem-mt_aMG8PcwPAR) ## Opens up - [Reporting Chemical Data: Statistics, Figures and Data Availability](https://lightmysky.com/learn/science/reporting-chemical-data-statistics-figures-and-data-availability-mt_P3RFmE6LzI)