Orthogonal Projection and Least Squares · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Dropping straight down onto the data sheet

Mathematics · Linear Algebra · ages 20-21
Name ______________________   Date ____________

  1. Data points are (0, 1), (1, 2) and (2, 4). The least squares slope is (n times sum xy minus sum x times sum y) over (n times sum x squared minus (sum x) squared). What is the slope?

    Answer: ______________

  2. Project the vector (3, 4) onto the direction (1, 0). What is the projection?

    • (3, 0)
    • (0, 4)
    • (1.5, 2)
    • (3, 4)

  3. Why is the least squares solution called the best solution of an inconsistent system?

    • It makes the residual vector as short as possible
    • It passes exactly through every data point
    • It makes all the residuals equal to each other
    • It forces the residual vector to be zero

  4. The projection of (2, 6) onto (1, 1) is (4, 4). Which residual is left, and is it orthogonal to (1, 1)?

    • (-2, 2), yes, the dot product is 0
    • (2, 2), yes, the dot product is 4
    • (-2, 2), no, the dot product is 8
    • (0, 0), yes, the fit is exact

  5. Project (2, 6) onto (1, 1). The answer equals k times (1, 1). Type k.

    Answer: ______________

  6. You fit a line without squaring the misses. What can go wrong?

    • The line always becomes vertical
    • Every data set gets zero lines
    • Positive and negative misses cancel and hide bad fits

  7. Why is the least squares answer the best compromise?

    • It makes the residual zero
    • It minimises the length of the residual
    • It uses the most columns

  8. There is no exact solution to A times x equals b. What do the normal equations give you?

    • An exact solution anyway
    • A bigger residual than any other x
    • The x whose product is the projection of b

  9. A student solves the normal equations and gets a residual that is not perpendicular to one column. What does that mean?

    • The arithmetic slipped somewhere, since perpendicularity must hold
    • The data set has no least squares line
    • Perpendicularity only matters for square matrices

  10. Four points refuse to sit on one straight line. What does least squares hand you?

    • No line at all
    • The single line with the smallest total of squared misses
    • The line through the first two points only
LightMySky · lightmysky.comW1-mt_kGvRrYJrat-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Dropping straight down onto the data sheet W1-mt_kGvRrYJrat-s1

  1. 1.5 · Sums give (30 minus 21) over (15 minus 9), which is 9 over 6, or 1.5.
  2. (3, 0) · The dot products give a scalar of 3, and 3 times (1, 0) is (3, 0).
  3. It makes the residual vector as short as possible · No exact solution exists, so best means the smallest miss, which is the shortest residual.
  4. (-2, 2), yes, the dot product is 0 · Subtracting gives (-2, 2), and its dot product with (1, 1) is minus 2 plus 2, which is 0.
  5. 4 · The dot product is 8 and the squared length is 2, so k is 4.
  6. Positive and negative misses cancel and hide bad fits · Raw misses can sum to zero for a terrible line, so squaring keeps score honest.
  7. It minimises the length of the residual · The perpendicular drop is the shortest path, so no other x leaves a smaller gap.
  8. The x whose product is the projection of b · They find the x that lands exactly on the closest reachable point.
  9. The arithmetic slipped somewhere, since perpendicularity must hold · A true least squares residual is orthogonal to the whole sheet, every column included.
  10. The single line with the smallest total of squared misses · Inconsistent systems still have exactly one minimiser of the squared score.
Worksheet · LightMySky