Point Estimation: Bias, Variance and Consistency
An estimator is a random variable computed from a sample. Judge it by whether it is centred on the truth, how much it varies, and whether it settles down as the sample grows.
What a learner can do afterwards
- Show that the sample mean is unbiased for the population mean
- Explain the n - 1 divisor in the sample variance
- Compare two estimators on bias and variance together
1 · Read
An estimator is a number computed from a sample, and it varies from sample to sample. It is unbiased when its average over all possible samples equals the truth. The sample mean is unbiased for the population mean. Larger samples keep this aim while tightening the scatter.
Take the readings 4, 6 and 8. Their sum is 18 and dividing by 3 gives a sample mean of 6. That 6 estimates the population mean without bias. Its spread shrinks like sigma squared over n, so bigger samples pin the mean down.
Spread is measured from the true mean, but you only have the sample mean, which sits closest to the data. Distances from it run systematically short, so dividing by n underestimates the variance. Dividing by n minus 1 inflates just enough to remove that bias. For 4, 6 and 8 the squared deviations total 8, and dividing by 2 gives a variance of 4.
Judge estimators on bias and variance together through the mean squared error, which is variance plus squared bias. An estimator with bias 0.5 and variance 1 scores 1.25, beating an unbiased one with variance 4. Consistency means the estimator concentrates on the target as the sample grows, with bias and variance draining away.
Unbiased means centred on the truth, n minus 1 keeps the variance honest, and mean squared error plus consistency judge the whole package.
2 · Watch
Take it off screen
Where it sits
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.