Sampling Distributions and the Central Limit Theorem

Use independent random observations from one population with finite mean μ and finite, positive standard deviation σ. Imagine repeatedly taking samples of the same size n and finding each mean. These means form a sampling distribution. Its expected mean is μ. Its standard deviation is called the standard error. Sampling without replacement from a small population needs a correction to the formula used here.

Under these assumptions, the central limit theorem says the standardised sample mean approaches a standard normal distribution as n grows. For a large enough n, sample means are approximately normal. There is no universal cutoff of 30. Strong skew or heavy tails can require larger samples. If the population is normal, independent sample means are exactly normal for every positive sample size.

When a normal model fits the sample mean, use z = (sample mean - μ) / (σ / √n). The denominator is the standard error, not σ. For μ = 40, σ = 10 and n = 25, a mean of 44 gives z = 2. A left-cumulative normal table gives P(Z < 2) = 0.9772, so P(sample mean > 44) = 1 - 0.9772 = 0.0228.

A larger independent random sample from the same population gives a more precise mean because its standard error is smaller. This is a statement about repeated sampling. A particular large sample can still be farther from μ than a particular small sample. Increasing sample size does not repair biased selection, poor measurements or dependence between observations.