--- title: "The Law of Large Numbers and the Central Limit Theorem" description: "For independent observations from the same distribution, the law of large numbers gives convergence of the sample mean under an integrability condition. With finite, positive variance, the central lim" canonical: https://lightmysky.com/learn/mathematics/the-law-of-large-numbers-and-the-central-limit-theorem-mt_NknG3L5rQp source: https://lightmysky.com/learn/mathematics/the-law-of-large-numbers-and-the-central-limit-theorem-mt_NknG3L5rQp.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # The Law of Large Numbers and the Central Limit Theorem For independent observations from the same distribution, the law of large numbers gives convergence of the sample mean under an integrability condition. With finite, positive variance, the central limit theorem gives a normal limit for standardised sample means. Subject: Mathematics · Area: Data & Statistics · Ages 19 to 20 Page: https://lightmysky.com/learn/mathematics/the-law-of-large-numbers-and-the-central-limit-theorem-mt_NknG3L5rQp ## Ready when they can - State LLN and CLT with their conditions and distinguish their two convergence claims - Calculate standard errors and standardised means, and assess sample-size conditions for normal approximations - Separate claims about averages from claims about individual observations or successive running means ## Lesson: Averages: convergence and normal limits For independent observations from the same distribution, let μ be the population mean. With finite expected absolute value E|X|, LLN says the sample mean converges to μ in probability: for every ε > 0, P(|sample mean - μ| > ε) tends to 0 as n grows. With finite, positive variance σ², CLT says (sample mean - μ) / (σ / √n) converges in distribution to a standard normal variable. LLN does not promise that each added observation improves a running mean. For independent fair coin flips, code heads as 1 and tails as 0, so μ = 1/2. Results 1, 0 have mean 1/2. Adding a 1 changes the mean to 2/3, farther from μ. LLN concerns the probability of an error exceeding a fixed tolerance, not improvement at every step. As in your sampling-distribution lesson, the standard error of an independent sample mean is σ / √n. It is exact for observations from one population with finite variance, even if the mean is not normal. With σ = 10, it is 2 at n = 25 and 1 at n = 100. Four times the sample size halves the standard error. For a target standard error at most s > 0, use n ≥ (σ / s)² and round up. For σ = 9 and s = 2, round 20.25 up to 21. *(drawing: These bars compare standard errors, not distribution shapes. Four times the sample size halves the standard error.)* There is no universal cutoff of 30 or 100 for an accurate normal approximation. Strong skew or heavy tails can require much larger samples, even with finite variance. Assess accuracy using the population model or a suitable simulation. Independent means from a normal population are exactly normal at every positive sample size. One observation keeps the population distribution. Infinite variance is outside the CLT version taught here. When a normal model fits the sample mean, use z = (sample mean - μ) / (σ / √n). Subtract μ, then divide by the standard error. For independent observations from a normal population with μ = 30, σ = 8 and n = 16, a mean of 34 gives z = (34 - 30) / 2 = 2. A left-cumulative normal table gives P(Z < 2) = 0.9772, so P(sample mean > 34) is approximately 0.0228. The previous joint-distribution lesson explains independence. Copying one random observation X into every entry does not create independent observations. All n entries then have mean (X + ... + X) / n = X. Its standard deviation stays σ, not σ / √n. The independent-sampling formulas and theorem assumptions do not apply to these copies. More data also do not repair biased selection. For independent fair coin flips, heads has probability 1/2 on the next flip, whatever the past results. Many tails do not make heads due. LLN concerns the long-run sample proportion, and CLT concerns the distribution of standardised sample proportions. Neither theorem changes the next flip's probability. For independent trials with fixed success probability p, code success as 1 and failure as 0. Their mean is the sample proportion. Expected successes are np and expected failures are n(1 - p). A common normal-approximation check asks for both to be at least 10. This is a rule of thumb, not a guarantee of accuracy. For n = 120 and p = 0.25, the expected counts are 30 and 90. The check does not remove bias or dependence. **Recap.** For independent observations from one distribution, finite E|X| gives LLN convergence of means to μ in probability; finite positive variance gives CLT's standard normal limit for standardised means. *Lesson, questions and flashcards written for this learning dataset.* ## Practice 13 questions on this page, each with its working shown. ## Needs first - [Sampling Distributions and the Central Limit Theorem](https://lightmysky.com/learn/mathematics/sampling-distributions-and-the-central-limit-theorem-mt_nugaEeUSii) - [Joint Distributions, Covariance and Independence](https://lightmysky.com/learn/mathematics/joint-distributions-covariance-and-independence-mt_YyTFTAqhSF) ## Opens up - [Point Estimation: Bias, Variance and Consistency](https://lightmysky.com/learn/mathematics/point-estimation-bias-variance-and-consistency-mt_I374r2UU_i) - [The Microcanonical and Canonical Ensembles](https://lightmysky.com/learn/science/the-microcanonical-and-canonical-ensembles-mt_NSCYLK-brt) - [Brownian motion and its defining properties](https://lightmysky.com/learn/mathematics/brownian-motion-and-its-defining-properties-mt_T5KtWIk07Z) - [Overfitting and the Bias-Variance Trade-off](https://lightmysky.com/learn/computing/overfitting-and-the-bias-variance-trade-off-mt_xznyNxeNNw)