--- title: "Errors, Power and the Design of a Test" description: "Two ways to be wrong, and a trade-off between them. Power is the chance of detecting a real effect, and it is decided at design time by the sample size." canonical: https://lightmysky.com/learn/mathematics/errors-power-and-the-design-of-a-test-mt_D7ZXbvSD0l source: https://lightmysky.com/learn/mathematics/errors-power-and-the-design-of-a-test-mt_D7ZXbvSD0l.md retrieved: 2026-09-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # Errors, Power and the Design of a Test Two ways to be wrong, and a trade-off between them. Power is the chance of detecting a real effect, and it is decided at design time by the sample size. Subject: Mathematics · Area: Data & Statistics · Ages 20 to 21 Page: https://lightmysky.com/learn/mathematics/errors-power-and-the-design-of-a-test-mt_D7ZXbvSD0l ## Ready when they can - Describe a type I and a type II error in the context of a study - Explain how sample size and effect size change power - Say why a nonsignificant result is not evidence of no effect ## Lesson: Two ways to be wrong Every test ends in one of four cells: the null is true or false, crossed with reject or keep. Rejecting a true null is a Type I error, a false alarm whose chance is the significance level alpha. Keeping a false null is a Type II error, a miss whose chance is beta. A trial declaring a useless drug effective commits a Type I error. Power is 1 minus the Type II chance: the probability of catching a stated real effect. Bigger samples raise power by shrinking the standard error, and bigger effects raise it by clearing the bar by more. A test with Type II chance 0.2 has power 0.8. Power is decided at design time, before data are collected. A nonsignificant result never proves no effect; it only means the data cannot rule the null out, and small effects may still hide. That is why a study with 20 subjects may miss a real effect. Fix it by choosing the effect worth catching first, then sizing the sample to catch it with good power. **Recap.** Name each mistake with the grid, design for power up front, and never read a miss as proof of nothing. ## Practice 12 questions on this page, each with its working shown. ## Needs first - [Point Estimation: Bias, Variance and Consistency](https://lightmysky.com/learn/mathematics/point-estimation-bias-variance-and-consistency-mt_I374r2UU_i) - [Hypothesis Tests for a Mean with the t-Distribution](https://lightmysky.com/learn/mathematics/hypothesis-tests-for-a-mean-with-the-t-distribution-mt_ig2o39hAwh) ## Opens up - [Pooled Genetic Screens: Library Design, Selection and Reading a Hit List](https://lightmysky.com/learn/science/pooled-genetic-screens-library-design-selection-and-reading-a-hit-list-mt_0n-a7GfLOL) - [The Data Science Workflow](https://lightmysky.com/learn/computing/the-data-science-workflow-mt_99QNfiyQ55) - [Beyond Accuracy: Precision, Recall and the Cost of an Error](https://lightmysky.com/learn/computing/beyond-accuracy-precision-recall-and-the-cost-of-an-error-mt_BHGgZM7tak) - [Designing a Biological Experiment: Units, Controls and Randomisation](https://lightmysky.com/learn/science/designing-a-biological-experiment-units-controls-and-randomisation-mt_Hc6BAdlkLf) - [Testing Thousands of Genes at Once: False Discovery Rate and What q Means](https://lightmysky.com/learn/science/testing-thousands-of-genes-at-once-false-discovery-rate-and-what-q-means-mt_mIQ27rphrC) - [Systematic Error, Controls and the Design of a Measurement](https://lightmysky.com/learn/science/systematic-error-controls-and-the-design-of-a-measurement-mt_rGjB_JcsLJ) - [Comparing Two Populations](https://lightmysky.com/learn/mathematics/comparing-two-populations-mt_v34rZkMfqr) - [Power and Effect Size, Decided Before the Data](https://lightmysky.com/learn/science/power-and-effect-size-decided-before-the-data-mt_zYNmSWAJT8)