Orthogonal Projection and Least Squares · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Projection and least squares

Mathematics · Linear Algebra · ages 20-21
Name ______________________   Date ____________

  1. For the least squares line y = a + ct fitted to (0,0), (1,2), (2,1), the normal equations are 3a + 3c = 3 and 3a + 5c = 4. Enter the slope c as a decimal.

    Answer: ______________

  2. The projection of b = (2,0) onto the line spanned by u = (1,1) is (1,1). What is its projection onto the line spanned by 2u = (2,2)?

    • (2,2), because doubling a direction doubles every projection.
    • (1,1), because both directions span the same line.
    • (0.5,0.5), because doubling a direction halves every projection.
    • (2,0), because a longer direction makes the residual vanish.

  3. Let A = [[1,1],[1,1]] and b = (1,3). Both normal equations are 2x1 + 2x2 = 4. What follows about x = (0,2) and x = (1,1)?

    • Both minimise the squared residual and give the same fitted vector.
    • Only (0,2) minimises the squared residual because one coefficient is zero.
    • Only (1,1) minimises the squared residual because its coefficients agree.
    • Neither minimises the squared residual because neither fits b exactly.

  4. Using the standard dot product, b = (5,1) has projection p = (3,3) onto the line spanned by u = (1,1). Which gives r = b - p and r dot u?

    • (2, 2); dot product 4
    • (-2, 2); dot product 0
    • (0, 0); dot product 0
    • (2, -2); dot product 0

  5. Using the standard dot product, project (3, 7) onto the line spanned by (1, 1). The projection is k(1, 1). Enter k.

    Answer: ______________

  6. A fitted line has vertical residuals 3 and -3. Why does adding these signed residuals fail to measure its total error?

    • Their sum makes both observed points lie on the fitted line.
    • Their sum measures the perpendicular distance to the fitted line.
    • Their sum is zero even though both observations have nonzero errors.

  7. Let p be the projection of b onto the column space and r = b - p. Why does any x with Ax = p minimise the squared residual length?

    • The residual becomes zero for every possible data vector b.
    • Every other fitted vector adds a positive squared distance.
    • Every column becomes longer than every possible residual vector.

  8. For a real matrix A, Ax = b is inconsistent. What does any solution x of A^T A x = A^T b produce?

    • Ax equals b despite the stated inconsistency.
    • Ax has the largest possible distance from b.
    • Ax equals the projection of b onto the column space.

  9. Using exact arithmetic and the standard dot product, a student claims x solves A^T A x = A^T b. The computed residual b - Ax has a nonzero dot product with a column of A. What follows?

    • The claimed solution or the residual calculation contains an error.
    • The normal equations require nonzero residual dots for rectangular matrices.
    • The nonzero dot proves that the original system has no least squares solution.

  10. Fit y = a + ct by least squares to (0,1), (1,2), (2,2), (3,3), using vertical residuals. Which line results from the normal equations?

    • y = 0.6 + 1.1t
    • y = 1.1 + 0.6t
    • y = 1.0 + 0.7t
LightMySky · lightmysky.comW1-mt_kGvRrYJrat-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Projection and least squares W1-mt_kGvRrYJrat-s1

  1. 0.5 · Subtracting the first equation from the second gives 2c = 1, so c = 0.5.
  2. (1,1), because both directions span the same line. · The line is unchanged. The new coefficient is ((2,0) dot (2,2))/((2,2) dot (2,2)) = 4/8. Thus the projection is (1/2)(2,2) = (1,1).
  3. Both minimise the squared residual and give the same fitted vector. · Both have x1 + x2 = 2 and satisfy the normal equations. Both give Ax = (2,2), with residual (-1,1). The fitted vector is unique, but x is not.
  4. (2, -2); dot product 0 · Subtract components: r = (5 - 3, 1 - 3) = (2,-2). Then r dot u = 2 - 2 = 0, verifying orthogonality.
  5. 5 · The dot product is 10 and the direction's squared length is 2, so k = 10/2 = 5.
  6. Their sum is zero even though both observations have nonzero errors. · The signed sum is 3 + (-3) = 0, but the sum of squares is 9 + 9 = 18. Opposite signs cancel in the signed sum.
  7. Every other fitted vector adds a positive squared distance. · Orthogonality gives ||b - Ax|| squared = ||r|| squared + ||p - Ax|| squared. The second term is zero when Ax = p and positive for any different fitted vector.
  8. Ax equals the projection of b onto the column space. · The normal equations give A^T(b - Ax) = 0. Thus Ax belongs to the column space and its residual is orthogonal to it, so Ax is the projection.
  9. The claimed solution or the residual calculation contains an error. · A normal-equation solution must satisfy A^T(b - Ax) = 0. A nonzero column dot product contradicts that condition.
  10. y = 1.1 + 0.6t · The equations are 4a + 6c = 8 and 6a + 14c = 15. Eliminating a gives 10c = 6, so c = 0.6 and a = 1.1.
Worksheet · LightMySky