---
title: "Principal Components and Dimensionality Reduction"
description: "When features are many and correlated, much of the variation often lies along a few directions. Principal component analysis finds those directions from the covariance structure and re-expresses each "
canonical: https://lightmysky.com/learn/computing/principal-components-and-dimensionality-reduction-mt_l-9XHg_ChZ
source: https://lightmysky.com/learn/computing/principal-components-and-dimensionality-reduction-mt_l-9XHg_ChZ.md
retrieved: 2026-09-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# Principal Components and Dimensionality Reduction

When features are many and correlated, much of the variation often lies along a few directions. Principal component analysis finds those directions from the covariance structure and re-expresses each example in far fewer numbers, which makes data visible and models cheaper.

Subject: Computing · Area: Machine Learning · Ages 20 to 22
Page: https://lightmysky.com/learn/computing/principal-components-and-dimensionality-reduction-mt_l-9XHg_ChZ

## Ready when they can

- Project a correlated two-feature dataset onto its first component and say what was kept
- Read a scree plot and choose how many components to keep, with a reason
- Say why the components are hard to name in the language of the original features

## Lesson: Fewer numbers, most of the spread

When features are many and correlated, much of the variation often lies along a few directions. Principal component analysis finds those directions from how the features vary together, and re-expresses each example in far fewer numbers. Projecting onto the first component keeps one thing: each point position along the direction where the data spreads out the most.

**Example.** Read a scree plot to choose how many components to keep. Suppose the variances are 62 percent, 25 percent, 8 percent, 3 percent and 2 percent. The first two already capture 62 plus 25, which is 87 percent, while later gains are tiny, so keep two. Reducing ten features to those two keeps most of the variation and makes the data visible and models cheaper.

Scale matters before running PCA. Standardising every feature to mean 0 and standard deviation 1 stops big-scale features from dominating the raw covariance. When every variance equals 1, covariance PCA and correlation PCA agree. Recall the link: correlation equals covariance divided by both standard deviations, so covariance 4 with deviations 2 and 4 gives 4 divided by 8, which is 0.5.

**Tip.** Do not expect clean names for components. A component like 0.7 times height plus 0.7 times weight minus 0.1 times age blends several original features, so it is hard to label in their language. Keep the components that capture most of the variation, and accept the blend.

**Recap.** Chase the big-spread directions, standardise first, keep few components, and accept blended axes.

## Practice

15 questions on this page, each with its working shown.

## Needs first

- [Clustering: Structure Without Labels](https://lightmysky.com/learn/computing/clustering-structure-without-labels-mt_BAMGOlWgRY)
- [Symmetric Matrices and the Spectral Theorem](https://lightmysky.com/learn/mathematics/symmetric-matrices-and-the-spectral-theorem-mt_sdQv4m7Nbk)
- [The Singular Value Decomposition](https://lightmysky.com/learn/mathematics/the-singular-value-decomposition-mt_UsUZbM9VRK)

## Opens up

- [Generative Models: Learning a Distribution You Can Sample](https://lightmysky.com/learn/computing/generative-models-learning-a-distribution-you-can-sample-mt_3CHsmpWVX5)
- [Single-Cell Analysis: Normalisation, Embedding and Naming Cell States](https://lightmysky.com/learn/science/single-cell-analysis-normalisation-embedding-and-naming-cell-states-mt_diYHLaLmij)
- [Attention as a Learned Lookup](https://lightmysky.com/learn/computing/attention-as-a-learned-lookup-mt_rrdtS1lHvC)
- [From Perceptron to Multilayer Network](https://lightmysky.com/learn/computing/from-perceptron-to-multilayer-network-mt_TTa7KN1-IO)