Symmetric Matrices and the Spectral Theorem
A real symmetric matrix has real eigenvalues and an orthonormal eigenbasis. That single fact is behind quadratic forms, covariance matrices and principal axes.
What a learner can do afterwards
- Diagonalise a symmetric matrix with an orthogonal matrix
- Classify a quadratic form from the signs of its eigenvalues
- Explain why symmetry forces eigenvectors for distinct eigenvalues to be orthogonal
1 · Read
A real square matrix is symmetric if A^T = A; T means transpose. The spectral theorem gives real eigenvalues and a complete orthonormal eigenbasis. For n-by-n A, put n unit eigenvectors in square Q and matching eigenvalues in diagonal D. Check Q^T Q = I and AQ = QD using column lengths, pairwise dot products and Av. These checks give Q^T A Q = D and A = Q D Q^T. Here I is the identity, and Av = lambda v defines a nonzero eigenvector.
For A = [[4, 1], [1, 4]], solve det(A - lambda I) = 0. In two dimensions the polynomial is lambda^2 - trace(A) lambda + det(A). Here it is lambda^2 - 8lambda + 15 = (lambda - 5)(lambda - 3). For 5, solve -x + y = 0; for 3, solve x + y = 0. Choose (1, 1) and (1, -1). Both have length sqrt(2). Divide each by sqrt(2) to form Q, and set D = diag(5, 3).
Suppose Au = lambda u and Av = mu v. Symmetry gives (Au) dot v = u dot (Av), so lambda(u dot v) = mu(u dot v). Distinct eigenvalues therefore force u dot v = 0. Equal eigenvalues do not force this: for 3I, (1, 0) and (1, 1) are independent eigenvectors with dot product 1. Use Gram-Schmidt within each repeated eigenspace to choose an orthonormal basis.
For symmetric A = [[a, b], [b, c]], f(x, y) = [x, y]A[x, y]^T = ax^2 + 2bxy + cy^2. With x = Qz, a quadratic form becomes sum(lambda_i z_i^2). All positive eigenvalues mean positive definite; all negative mean negative definite. Both signs mean indefinite, a saddle in two variables. Nonnegative eigenvalues including zero give positive semidefinite with a flat direction. Reverse the signs for negative semidefinite.
An orthogonal Q needs perpendicular columns of length one. A vector's length is sqrt(v1^2 + v2^2 + ...); divide by that length to normalise it. Perpendicular columns of lengths 3 and 4 give Q^T Q = diag(9, 16), not I. Rescaling an eigenvector leaves its eigenvalue unchanged. If you reorder Q's columns, reorder D's diagonal in the same way.
A real covariance matrix C is symmetric: swapping the variables leaves the centered products unchanged, so C_ij = C_ji. Its principal axes are lines spanned by eigenvectors. To test a pair of axes, check that their spanning vectors are perpendicular and each Cv is a scalar multiple of v. Normalising these vectors gives Q; put their eigenvalues in matching order in D. Repeated eigenvalues allow different orthonormal axes within that eigenspace.
Without symmetry, a real matrix need not have an eigenbasis. For B = [[1, 1], [0, 1]], the only eigenvalue is 1. Solving (B - I)v = 0 gives y = 0, so every eigenvector lies on the x-axis. A basis of the plane needs two independent vectors. This matrix has no eigenbasis and refutes the claim that every real square matrix has an orthonormal eigenbasis.
A real symmetric matrix has an orthonormal eigenbasis. Match Q's columns with D's eigenvalues; their signs classify the quadratic form.
The lesson, practice questions and flashcards were written for this dataset.
The readings and extra questions were written for this dataset. Flashcards summarise mathematical facts. Linked videos are credited to Khan Academy; their licenses have not been established.
More to read
Keeping eigenvectors and eigenvalues paired
An orthogonal diagonalisation uses an orthonormal eigenbasis. Find each eigenspace by solving (A - lambda I)v = 0, then choose perpendicular unit vectors within it.
For A = diag(4, 7), the coordinate vectors form Q = I and D = diag(4, 7). You may exchange the two columns of Q, but then D must become diag(7, 4). Each eigenvector must remain paired with its own eigenvalue.
Check Q^T Q = I and AQ = QD. The first checks orthonormality. The second checks both the eigenvector equations and their order.
Distinct and repeated eigenvalues
For symmetric A, suppose Au = lambda u and Av = mu v. Symmetry lets us write (Au) dot v = u dot (Av). Substitution gives (lambda - mu)(u dot v) = 0.
When lambda and mu differ, the dot product must be zero. When they agree, the equation imposes no condition on the dot product.
For 2I, the vectors (1, 0) and (1, 2) are independent eigenvectors, yet their dot product is 1. An orthonormal basis can still be chosen: (1, 0) and (0, 1) work. The theorem promises that such a basis exists, not that every eigenbasis is orthonormal.
Signs in principal coordinates
Write a quadratic form as f(x) = x^T A x with symmetric A. Substituting x = Qz gives f(x) = z^T D z, a sum of squared coordinates multiplied by eigenvalues.
For f(s, t) = 4s^2 - 9t^2, the inputs (1, 0) and (0, 1) give opposite signs. This form is indefinite.
All positive eigenvalues make the form positive at every nonzero input. Nonnegative eigenvalues with a zero give a flat direction instead. For example, 4s^2 vanishes along the t-axis. That form is positive semidefinite. Reversing all signs gives the corresponding negative classifications.
Explore more
2 · Watch
به انگلیسیTake it off screen
Where it sits
Learn first
Optional connections
This opens up
14 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.