Power and Effect Size, Decided Before the Data
Plan power for a specified effect, test, variability and sampling design. Calculate sample size under stated assumptions, check sensitivity, and distinguish scientific importance from statistical significance.
What a learner can do afterwards
- Compute and round up sample size using a stated power formula and its assumptions
- Explain why observed-effect power adds no independent evidence for the same fixed test
- Explain how selecting significant results can exaggerate estimated effect magnitudes in low-power settings
1 · Read
Power is the probability that a specified test rejects H0 at a specified true effect. Plan before collecting data using the test, significance level, variability and sampling design. As in the previous topic, distinguish assigned units from repeated readings and account for grouping. Planned power depends on assumptions and does not guarantee one study's result.
For independent normal observations with known standard deviation sigma, test H0: mu = mu0 against mu > mu0. For a target shift delta = mu - mu0 > 0, use n = ceil[((z_a + z_p) sigma / delta)^2]. Ceil means round up. z_a sets the significance threshold; z_p sets target power. For alpha 0.05 and power 0.80, use rounded constants 1.645 and 0.842. Other tests and grouped designs need different calculations.
Under this model, more independent observations or lower variability raise power at a fixed positive effect and alpha. A larger target effect needs fewer observations at fixed power. Effect size can use original or standardized units. A small p-value does not establish practical importance. Compare the estimated magnitude and interval with a justified scientific threshold.
Observed power substitutes the observed effect into a power calculation. For the same fixed test and assumptions, it is tied to the test result and adds no independent evidence. After a study, examine the estimate and interval. Later planning can still use scientifically chosen effects to assess a new design; it need not reuse the observed effect.
When power for a small true effect is low, selecting significant results can favor unusually large estimated magnitudes. Under common sampling models, selected magnitudes can exceed the true magnitude on average. This does not mean every significant estimate is inflated. Check uncertainty and selection risks, and seek independent replication.
Choose the smallest scientifically useful effect, then vary plausible effects and variability in planning. In the stated formula, halving delta multiplies the unrounded sample size by four. Sensitivity shows dependence on assumptions, not broken arithmetic. Sizing a replication around an inflated selected estimate can give too small a sample for a more modest true effect.
Plan for a specified effect and design. Check assumptions, round sample size up, and interpret estimates and uncertainty separately from significance.
Original lesson, questions and flashcards about power planning, written for this learning dataset.
The readings and flashcards were written for this learning dataset. The linked videos about significance tests and simulated p-values are credited to Khan Academy in the source catalog.
More to read
State the conditions behind power
Power belongs to a test and a specified true effect. Imagine repeating the planned procedure under those conditions and counting the fraction of studies that reject H0. That fraction estimates power. A calculation also needs assumptions about variability, sampling and dependence. Several readings from one culture do not create several independently assigned cultures. Use a calculation that matches the design rather than inserting the total reading count into an independent-observation formula.
Use estimates after the study
Observed power inserts the observed effect into a power calculation. For the same fixed test and assumptions, it is tied to the test result and supplies no independent evidence. Examine the estimated effect and its interval instead. A later planning calculation can still ask a new question: how many observations would another study need for an effect that matters scientifically? State that effect and the new design assumptions rather than presenting observed power as an additional finding.
Separate magnitude from significance
An effect size may be a difference in original measurement units or a standardized quantity. State which definition is used. A p-value does not measure magnitude. A small difference can become statistically significant when measured precisely, while still falling below a scientific threshold for usefulness. Compare the estimate and interval with a justified threshold. Generic labels such as small or large do not decide whether an effect matters for a particular question.
Check the significance filter
When a design has low power for a small true effect, many estimates do not reach significance. Reporting only those that do can favor unusually large magnitudes. Under common sampling models, the selected magnitudes exceed the true magnitude on average. This does not identify every selected estimate as inflated. Assess uncertainty, reporting practices and independent evidence. An inflated estimate can also mislead planning: treating it as the replication target can produce too small a sample for a more modest effect.
Calculate size under an explicit model
For independent normal observations with known standard deviation sigma, a one-sided mean test uses n = ceil[((z_a + z_p) sigma / delta)^2] for a positive target shift delta. The constants describe the significance threshold and target power. The ceiling rounds up to a whole observation. Keep the other inputs fixed and halving delta quadruples the unrounded expression. Check several plausible effects and variability values. Unknown variance, two-sided tests and grouped designs require other calculations.
Explore more
2 · Watch
En anglaisTake it off screen
Where it sits
Where this leads
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.