The Law of Large Numbers and the Central Limit Theorem
For independent observations from the same distribution, the law of large numbers gives convergence of the sample mean under an integrability condition. With finite, positive variance, the central limit theorem gives a normal limit for standardised sample means.
What a learner can do afterwards
- State LLN and CLT with their conditions and distinguish their two convergence claims
- Calculate standard errors and standardised means, and assess sample-size conditions for normal approximations
- Separate claims about averages from claims about individual observations or successive running means
1 · Read
For independent observations from the same distribution, let μ be the population mean. With finite expected absolute value E|X|, LLN says the sample mean converges to μ in probability: for every ε > 0, P(|sample mean - μ| > ε) tends to 0 as n grows. With finite, positive variance σ², CLT says (sample mean - μ) / (σ / √n) converges in distribution to a standard normal variable.
LLN does not promise that each added observation improves a running mean. For independent fair coin flips, code heads as 1 and tails as 0, so μ = 1/2. Results 1, 0 have mean 1/2. Adding a 1 changes the mean to 2/3, farther from μ. LLN concerns the probability of an error exceeding a fixed tolerance, not improvement at every step.
As in your sampling-distribution lesson, the standard error of an independent sample mean is σ / √n. It is exact for observations from one population with finite variance, even if the mean is not normal. With σ = 10, it is 2 at n = 25 and 1 at n = 100. Four times the sample size halves the standard error. For a target standard error at most s > 0, use n ≥ (σ / s)² and round up. For σ = 9 and s = 2, round 20.25 up to 21.
There is no universal cutoff of 30 or 100 for an accurate normal approximation. Strong skew or heavy tails can require much larger samples, even with finite variance. Assess accuracy using the population model or a suitable simulation. Independent means from a normal population are exactly normal at every positive sample size. One observation keeps the population distribution. Infinite variance is outside the CLT version taught here.
When a normal model fits the sample mean, use z = (sample mean - μ) / (σ / √n). Subtract μ, then divide by the standard error. For independent observations from a normal population with μ = 30, σ = 8 and n = 16, a mean of 34 gives z = (34 - 30) / 2 = 2. A left-cumulative normal table gives P(Z < 2) = 0.9772, so P(sample mean > 34) is approximately 0.0228.
The previous joint-distribution lesson explains independence. Copying one random observation X into every entry does not create independent observations. All n entries then have mean (X + ... + X) / n = X. Its standard deviation stays σ, not σ / √n. The independent-sampling formulas and theorem assumptions do not apply to these copies. More data also do not repair biased selection.
For independent fair coin flips, heads has probability 1/2 on the next flip, whatever the past results. Many tails do not make heads due. LLN concerns the long-run sample proportion, and CLT concerns the distribution of standardised sample proportions. Neither theorem changes the next flip's probability.
For independent trials with fixed success probability p, code success as 1 and failure as 0. Their mean is the sample proportion. Expected successes are np and expected failures are n(1 - p). A common normal-approximation check asks for both to be at least 10. This is a rule of thumb, not a guarantee of accuracy. For n = 120 and p = 0.25, the expected counts are 30 and 90. The check does not remove bias or dependence.
Expected counts describe the model. They need not equal the observed counts.
For independent observations from one distribution, finite E|X| gives LLN convergence of means to μ in probability; finite positive variance gives CLT's standard normal limit for standardised means.
Lesson, questions and flashcards written for this learning dataset.
Readings, extra questions and flashcards written for this learning dataset. Video references credit Khan Academy.
More to read
A probability statement about averages
Fix a positive tolerance around the population mean μ. You might ask whether a sample mean differs from μ by more than two units. The law of large numbers describes how the probability of that event changes as the sample size grows.
For independent observations from the same distribution with finite expected absolute value, the probability tends to zero. The conclusion holds for every fixed positive tolerance. This is convergence in probability.
The statement does not give a sample size after which all sample means equal μ. It also does not say that each new observation improves one running average. A new observation can move a running average farther from μ.
The expected sample mean and an observed sample mean are different quantities. Under these assumptions, the expected sample mean is already μ for every positive sample size. It is not a centre that gradually moves toward μ.
If the population also has finite variance, the standard error is σ / √n. Smaller standard error means less variation across repeated sample means. This describes the sampling model; it does not repair a survey that selects people from an unrepresentative group.
Using a normal approximation carefully
The central limit theorem used here assumes independent observations from one distribution with finite, positive variance. Subtract μ from the sample mean and divide by σ / √n. As n grows, the distribution of this standardised quantity approaches a standard normal distribution.
This statement keeps track of scale. The unstandardised mean has a standard error that shrinks with n. Its population of individual observations need not become normal, and a single draw keeps that population's distribution.
A limiting theorem does not give a universal sample-size cutoff for an accurate approximation. Strong skew and heavy tails can delay a useful normal model. At a chosen sample size, assess the population model or a suitable simulation. Infinite variance is outside this CLT's assumptions.
A normal population gives an exact result without waiting for large n. For independent observations from a normal population with μ = 40 and σ = 9, a sample mean based on nine observations is normal with standard error 3. A boundary of 43 gives z = (43 - 40) / 3 = 1. Using the rounded table value P(Z < 1) = 0.8413, the probability of a mean above 43 is approximately 0.1587.
Exact normality and a large-sample normal approximation have different justifications. State which one supports a calculation.
A proportion is a mean
Record 1 for a successful trial and 0 for a failed trial. The total is the number of successes. Dividing by n gives both the sample mean of these numbers and the sample proportion.
Assume independent trials with the same success probability p, where 0 < p < 1. Each recorded value has mean p and variance p(1 - p). The sample proportion has expected value p and standard error √(p(1 - p) / n).
For example, take p = 0.4 and n = 150. The expected numbers of successes and failures are 60 and 90. The standard error is √(0.24 / 150) = 0.04. These centre and spread calculations follow from the independent-trial model.
A normal approximation is a separate question. A common check asks whether np and n(1 - p) are both at least 10. These are expected counts. It does not require every observed sample to have at least 10 of each outcome, and it does not give a bound on approximation error.
The assumptions still matter. Repeated copies of one response are not independent trials. A large poll can also have biased selection. Neither a large count nor a normal approximation fixes those problems.
When independent trials have fixed success probability p, past failures do not raise the next trial's chance of success.
Explore more
2 · Watch
Take it off screen
Where it sits
Learn first
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
13 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.