--- title: "Errors, Power and the Design of a Test" description: "A Type I error rejects a true null hypothesis. A Type II error fails to reject a false null. Power is the probability of rejecting the null for a specified real effect, and depends on the test, signif" canonical: https://lightmysky.com/learn/mathematics/errors-power-and-the-design-of-a-test-mt_D7ZXbvSD0l source: https://lightmysky.com/learn/mathematics/errors-power-and-the-design-of-a-test-mt_D7ZXbvSD0l.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # Errors, Power and the Design of a Test A Type I error rejects a true null hypothesis. A Type II error fails to reject a false null. Power is the probability of rejecting the null for a specified real effect, and depends on the test, significance level, variability and sample size. Subject: Mathematics · Area: Data & Statistics · Ages 20 to 21 Page: https://lightmysky.com/learn/mathematics/errors-power-and-the-design-of-a-test-mt_D7ZXbvSD0l ## Ready when they can - Describe a type I and a type II error in the context of a study - Interpret power as a probability, compare planned powers, and explain how sample size, effect size and significance level change power under stated test assumptions - Explain why nonsignificance alone does not prove a zero effect or establish low power, while a precise interval can rule out effects of practical importance ## Lesson: Two ways to be wrong A Type I error rejects a true H0. A valid level-α test has probability at most α of doing this when H0 is true; its actual rate can be smaller. A Type II error fails to reject a false H0. Its probability β must be stated for a specified true parameter in the alternative. Failing to reject is a test decision, not acceptance of H0 as true. Power is the probability of rejecting H0 for a specified true parameter in the alternative. Rejection and failure to reject exhaust the possible decisions, so power = 1 - β. For β = 0.35, power is 0.65. Power describes repeated use of the procedure under those conditions; it does not guarantee rejection in one study. *(drawing: When H0 is false at this specified alternative, rejecting has probability 65%; failing to reject has probability 35%.)* For a one-sided normal-mean test, fix α, variance and sampling method. More independent observations raise power at a fixed mean in H1. With n fixed, moving farther into H1 raises power: below μ0 for a lower tail. An opposite-direction departure need not help. Planned power uses assumptions; actual power depends on true conditions and implementation. *(drawing: Failures mean failures to reject H0. Simulation fractions estimate probabilities, not exact values or guarantees about the next study.)* For a fixed model and statistic, raising α enlarges a nested rejection region. The allowed Type I error rate rises and power cannot decrease; lowering α reverses this trade-off. If α and the target effect must stay fixed, increasing independent sample size can raise power. Plan before collecting new data and recalculate power under the model. Subtract old planned power from new planned power to quantify a gain. Nonsignificance means no rejection at the planned level, not proof of zero effect. A precise interval can exclude practically important effects against a stated threshold. A p-value alone does not establish low power. Review estimates, uncertainty, target effects and design assumptions before planning another study. **Recap.** Type I rejects a true H0; Type II fails to reject a false H0. Power concerns a specified alternative. Plan under stated assumptions and error-rate limits; interpret estimates and uncertainty. *Original lesson, questions and flashcards about testing errors and statistical power, authored for this learning dataset.* ## Practice 12 questions on this page, each with its working shown. ## Needs first - [Hypothesis Tests for a Mean with the t-Distribution](https://lightmysky.com/learn/mathematics/hypothesis-tests-for-a-mean-with-the-t-distribution-mt_ig2o39hAwh) ## Optional connections - [Point Estimation: Bias, Variance and Consistency](https://lightmysky.com/learn/mathematics/point-estimation-bias-variance-and-consistency-mt_I374r2UU_i) ## Opens up - [Pooled Genetic Screens: Library Design, Selection and Reading a Hit List](https://lightmysky.com/learn/science/pooled-genetic-screens-library-design-selection-and-reading-a-hit-list-mt_0n-a7GfLOL) - [The Data Science Workflow](https://lightmysky.com/learn/computing/the-data-science-workflow-mt_99QNfiyQ55) - [Beyond Accuracy: Precision, Recall and the Cost of an Error](https://lightmysky.com/learn/computing/beyond-accuracy-precision-recall-and-the-cost-of-an-error-mt_BHGgZM7tak) - [Designing a Biological Experiment: Units, Controls and Randomisation](https://lightmysky.com/learn/science/designing-a-biological-experiment-units-controls-and-randomisation-mt_Hc6BAdlkLf) - [Testing thousands of genes at once: false discovery rate and what q means](https://lightmysky.com/learn/science/testing-thousands-of-genes-at-once-false-discovery-rate-and-what-q-means-mt_mIQ27rphrC) - [Systematic Error, Controls and the Design of a Measurement](https://lightmysky.com/learn/science/systematic-error-controls-and-the-design-of-a-measurement-mt_rGjB_JcsLJ) - [Comparing Two Populations](https://lightmysky.com/learn/mathematics/comparing-two-populations-mt_v34rZkMfqr) - [Power and Effect Size, Decided Before the Data](https://lightmysky.com/learn/science/power-and-effect-size-decided-before-the-data-mt_zYNmSWAJT8)