Multiple Regression and Model Diagnostics · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Many predictors, honest reading

Mathematics · Data & Statistics · ages 21-22
Name ______________________   Date ____________

  1. Given fitted y = 2 + 1.5x1 - 0.5x2, find fitted y at x1 = 4 and x2 = 2.

    Answer: ______________

  2. Two predictors are nearly, but not exactly, linearly dependent. What problem can this cause?

    • Imprecise coefficients sensitive to small data changes
    • A necessary drop of training R² to zero in every fitted sample
    • A necessary disappearance of every residual in the fitted model

  3. In fitted y = 2 + 1.5x1 - 0.5x2, what does the coefficient 1.5 describe?

    • A guaranteed causal increase of 1.5 in observed y whenever x1 rises
    • An increase of 1.5 in fitted y regardless of how much x2 changes
    • An increase of 1.5 in fitted y per unit of x1, holding x2 fixed

  4. In fitted y = 8 + 4x1 - 2x2, x1 increases by 2 and x2 increases by 3. Find the change in fitted y.

    Answer: ______________

  5. An ordinary least-squares model includes an intercept and x1, x2 and x3. Every row satisfies x3 = 2x1 + x2. What follows?

    • All three predictor coefficients must equal zero in every least-squares fit
    • The design columns must be independent because three predictors are included
    • The coefficients are not uniquely determined
    • No fitted response can be calculated because no least-squares solution exists

  6. An observation has unusual predictor values, but removing it changes the fitted model very little. Which conclusion is supported?

    • Unusual predictor values alone do not establish influence
    • Every unusual predictor combination must be deleted before fitting a model
    • The small change proves that the observation has the largest residual
    • The small change proves that all predictor coefficients are exactly zero

  7. Removing one observation substantially changes several fitted coefficients. What does this show about that observation?

    • It must have the largest residual among all observations
    • It must make two predictors exactly linearly dependent
    • It is influential for this fitted model

  8. An ordinary least-squares model has an intercept and residuals summing to zero. That fact alone establishes that the model is appropriate.

    Circle one:   True   False

  9. An additive regression fitted to observational data has a positive x1 coefficient. Does this alone establish that intervening to raise x1 will raise y?

    • No; the coefficient describes a conditional association
    • Yes; holding measured predictors fixed removes every possible source of bias
    • Yes; the sign of a fitted coefficient establishes the direction of a causal effect

  10. A team repeatedly adds predictors and selects its model using training R² alone. Which risk has it not checked?

    • Every added predictor must reduce training fit and increase prediction error
    • The model may fit sample noise and predict poorly on new data
    • Predictor collinearity must vanish once training R² becomes sufficiently high
LightMySky · lightmysky.comW1-mt_TUx1tDF0jv-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Many predictors, honest reading W1-mt_TUx1tDF0jv-s1

  1. 7 · Insert both values: 2 + 1.5(4) - 0.5(2) = 2 + 6 - 1 = 7.
  2. Imprecise coefficients sensitive to small data changes · Strong near dependence can inflate coefficient standard errors and make estimates sensitive to small changes.
  3. An increase of 1.5 in fitted y per unit of x1, holding x2 fixed · Holding x2 fixed leaves only the x1 term changing. Each extra unit of x1 adds 1.5 to fitted y.
  4. 2 · The fitted change is 4(2) - 2(3) = 8 - 6 = 2. The unchanged intercept cancels.
  5. The coefficients are not uniquely determined · The x3 column is twice the x1 column plus the x2 column. Exact dependence makes ordinary least-squares coefficients nonunique.
  6. Unusual predictor values alone do not establish influence · Influence concerns change in the fitted model. Unusual predictor values alone do not establish a substantial change.
  7. It is influential for this fitted model · An influential observation substantially affects the fitted model, as shown by the change after removing it.
  8. False · With an intercept, OLS residuals sum to zero apart from rounding. This fitting property does not establish model adequacy.
  9. No; the coefficient describes a conditional association · The coefficient describes fitted response change with other predictors held fixed. Observational regression alone does not establish an intervention's effect.
  10. The model may fit sample noise and predict poorly on new data · Training R² alone does not check whether fitted patterns generalise. Overfitting is a risk, not a proven result of adding predictors.
Worksheet · LightMySky