--- title: "Point Estimation: Bias, Variance and Consistency" description: "An estimator is a random variable computed from a sample. Judge it by whether it is centred on the truth, how much it varies, and whether it settles down as the sample grows." canonical: https://lightmysky.com/learn/mathematics/point-estimation-bias-variance-and-consistency-mt_I374r2UU_i source: https://lightmysky.com/learn/mathematics/point-estimation-bias-variance-and-consistency-mt_I374r2UU_i.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # Point Estimation: Bias, Variance and Consistency An estimator is a random variable computed from a sample. Judge it by whether it is centred on the truth, how much it varies, and whether it settles down as the sample grows. Subject: Mathematics · Area: Data & Statistics · Ages 20 to 21 Page: https://lightmysky.com/learn/mathematics/point-estimation-bias-variance-and-consistency-mt_I374r2UU_i ## Ready when they can - Show that the sample mean is unbiased for the population mean - Explain why the n - 1 divisor gives an unbiased variance estimate for independent, identically distributed observations with finite variance - Compare estimators using bias, variance and mean squared error, and distinguish this comparison from consistency ## Lesson: Judging an estimator An estimator T is a rule applied to random data; its observed value is an estimate. Its sampling distribution describes variation across samples. E(T) is its probability-weighted average. Unbiased means E(T) equals the target. Write Xi for observation i and mu for the population mean. If E(Xi) = mu, then E(sample mean) = (E(X1) + ... + E(Xn))/n = mu. For equally likely samples, average their estimates directly. One estimate need not equal mu. **Example.** Assume independent observations from the same population, with mean mu and finite variance sigma^2. Readings 2, 5 and 8 give sample mean 15/3 = 5. This is one estimate; unbiasedness is a property of the sampling rule. The sample mean has variance sigma^2/n and standard deviation sigma/sqrt(n). The law of large numbers describes its convergence to mu. These properties do not require a normal sampling distribution. *(drawing: This calculation gives one estimate. Another sample can give a different value.)* For n >= 2 independent observations from the same population with finite variance sigma^2, let Q be the squared-deviation total about the sample mean. Q equals the squared-deviation total about mu minus n(sample mean - mu)^2. Taking expectations gives E(Q) = n sigma^2 - n(sigma^2/n) = (n - 1)sigma^2. Thus Q/(n - 1) is unbiased; Q/n has downward bias if sigma^2 > 0. With n = 1, Q = 0 and division by n - 1 is undefined. **Example.** For 2, 5 and 8, the mean is 5. The deviations are -3, 0 and 3; their squares are 9, 0 and 9. Their total Q is 18. The sample variance is 18/(3 - 1) = 9. Dividing by n would give 6. To change a value calculated with divisor n to one using n - 1, multiply by n/(n - 1): 6 × 3/2 = 9. An unbiased rule does not make every estimate equal the population variance. *(drawing: The bars show squared deviations, not the readings themselves. They total 18.)* For target theta, bias = E(T) - theta. Mean squared error is E((T - theta)^2) = Var(T) + bias^2. Bias -1 and variance 3 give MSE 4; an unbiased estimator with variance 6 has MSE 6. For the same target and sample size, lower MSE means smaller average squared error. Adding a fixed amount c to an unbiased mean estimator gives bias c and leaves its variance unchanged. Consistency means that, for every fixed positive tolerance, the probability of an error larger than that tolerance tends to 0 as n grows. Bias and variance both tending to 0 are sufficient, not necessary, conditions. The sample mean is consistent under the conditions above. If bias stays at 1 while variance tends to 0, the estimator concentrates on the target plus 1. Low MSE at one sample size does not establish consistency. **Recap.** Unbiasedness concerns expectation across samples. The n - 1 correction needs sampling conditions. MSE measures average squared error; consistency concerns convergence as samples grow. *Lesson, questions and flashcards written for this dataset.* ## Practice 13 questions on this page, each with its working shown. ## Needs first - [The Law of Large Numbers and the Central Limit Theorem](https://lightmysky.com/learn/mathematics/the-law-of-large-numbers-and-the-central-limit-theorem-mt_NknG3L5rQp) ## Optional connections - [Standard deviation and variance](https://lightmysky.com/learn/mathematics/standard-deviation-and-variance-mt_eNpow5tIX2) ## Opens up - [Errors, Power and the Design of a Test](https://lightmysky.com/learn/mathematics/errors-power-and-the-design-of-a-test-mt_D7ZXbvSD0l) - [Maximum Likelihood Estimation](https://lightmysky.com/learn/mathematics/maximum-likelihood-estimation-mt_ZRTrcy7-KV)