--- title: "The Singular Value Decomposition" description: "Every real matrix has an SVD with orthogonal coordinate changes and nonnegative axis scalings. The orthogonal factors can rotate or reflect. Singular values measure stretches and support low-rank appr" canonical: https://lightmysky.com/learn/mathematics/the-singular-value-decomposition-mt_UsUZbM9VRK source: https://lightmysky.com/learn/mathematics/the-singular-value-decomposition-mt_UsUZbM9VRK.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # The Singular Value Decomposition Every real matrix has an SVD with orthogonal coordinate changes and nonnegative axis scalings. The orthogonal factors can rotate or reflect. Singular values measure stretches and support low-rank approximation. Subject: Mathematics · Area: Linear Algebra · Ages 20 to 21 Page: https://lightmysky.com/learn/mathematics/the-singular-value-decomposition-mt_UsUZbM9VRK ## Ready when they can - Relate singular values to the eigenvalues of A^T A and A A^T, including extra zero eigenvalues for rectangular matrices - Read the rank of a matrix off its singular values - Explain how truncation gives a best approximation of rank at most k in Frobenius or spectral norm ## Lesson: Stretch directions, ranked by size For a real m-by-n matrix A, the full singular value decomposition is A = U Sigma V^T. U is m-by-m and V is n-by-n. Both are orthogonal, so their transposes are their inverses. Sigma is m-by-n with nonnegative diagonal entries and zeros elsewhere. The diagonal entries are singular values, listed largest first. The orthogonal changes can include reflections as well as rotations. The largest singular value is max ||Ax|| over inputs with ||x|| = 1. Here ||x|| means Euclidean length. Each right singular vector is an input direction; a positive singular value sends it to that value times its matching left singular vector. A zero singular value has a right singular direction sent to zero. These are stretch factors measured using unit inputs. **Example.** Transpose swaps rows and columns without changing entry signs. For A = diag(-2, 6), A^T A = diag(4, 36), so the singular values are 6 and 2. A negative entry can reverse orientation while the stretch size stays nonnegative. For B = [[0, -2], [2, 0]], B^T B = 4I. Both singular values are 2, and every unit input has output length 2. *(drawing: A negative diagonal entry changes orientation, but its singular value is nonnegative.)* Form A^T A and find its eigenvalues. The product is symmetric. Also x^T A^T A x = ||Ax||^2, which is nonnegative. If A^T A v = lambda v with v nonzero, then lambda ||v||^2 = ||Av||^2. Since ||v||^2 is positive, lambda cannot be negative. Take nonnegative square roots of these eigenvalues to obtain the singular values, with the rectangular convention below. The usual list has p = min(m, n) singular values, including zeros when needed. A^T A has n eigenvalues; A A^T has m. Their positive eigenvalues agree with multiplicity, but their zero counts may differ. For n > m, omit n - m extra zeros from A^T A's eigenvalue list before taking square roots for the p-entry list. A wide matrix can collapse input directions even when all listed singular values are positive. Rank counts positive singular values. Orthogonal U and V are invertible, so replacing either by another orthogonal factor does not change rank. Replacing a positive entry of Sigma by zero lowers rank; replacing it by another positive entry does not. For a tall m-by-n matrix with n positive singular values, every input coordinate has a positive scale, so no nonzero input is sent to zero. *(drawing: The zero value does not add an independent output direction.)* Write A as layers sigma_i u_i v_i^T, using matching left and right singular vectors. Keeping the first k layers gives A_k. For 0 <= k < p, this is a best approximation among matrices of rank at most k in both Frobenius and spectral norm. Keeping a single positive layer gives rank one. Keeping every positive layer reproduces A exactly. Tied singular values can make the best approximation nonunique. *(drawing: The retained matrix depends on the singular values and their matching vector pairs.)* The Frobenius norm squared is the sum of squared entries. It also equals the sum of squared singular values. The spectral norm is the greatest output length for a unit input. For A - A_k, the squared Frobenius error is the sum of squares of discarded singular values. Take its square root for the Frobenius error. The spectral error is the largest discarded singular value. The eigenvalue recipe uses A^T A, not A A. Real symmetry is a sufficient condition for equality: A^T = A gives A^T A = A^2. Other square matrices need not work. For a quarter-turn matrix, A^2 = -I but A^T A = I. Using A^2 would give negative eigenvalues instead of squared stretches. A numerical result can still agree by coincidence without symmetry. Before watching the linked Visual Kernel SVD video, note: full rank does not make all eigenvectors of a symmetric matrix perpendicular. For B = 4I, (1, 0) and (1, 2) are independent eigenvectors with dot product 1, although B has full rank. Distinct eigenvalues force perpendicularity. Within a repeated eigenspace, Gram-Schmidt preserves the span: keep a unit q, subtract (q dot w)q from the next vector w, then divide the remainder by its length. Choose V from an orthonormal eigenbasis of A^T A, using Gram-Schmidt within repeated eigenspaces. For positive sigma_i, set u_i = Av_i/sigma_i. Then u_i dot u_j = (v_i^T A^T A v_j)/(sigma_i sigma_j), which is 0 for distinct i and j and 1 when i = j. Complete these vectors to an orthonormal basis for U when needed. This division formula cannot be used for a zero singular value. **Recap.** A = U Sigma V^T separates orthogonal changes from nonnegative stretches. Positive singular values count rank; keeping the largest complete layers gives best low-rank approximations. *The lesson, practice questions and flashcards were written for this dataset.* ## Practice 14 questions on this page, each with its working shown. ## Needs first - [Symmetric Matrices and the Spectral Theorem](https://lightmysky.com/learn/mathematics/symmetric-matrices-and-the-spectral-theorem-mt_sdQv4m7Nbk) ## Optional connections - [Orthogonal Projection and Least Squares](https://lightmysky.com/learn/mathematics/orthogonal-projection-and-least-squares-mt_kGvRrYJrat) ## Opens up - [Principal Components and Dimensionality Reduction](https://lightmysky.com/learn/computing/principal-components-and-dimensionality-reduction-mt_l-9XHg_ChZ)