Power and Effect Size, Decided Before the Data · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Planning power before you collect data

Science · Scientific Inquiry · ages 22-24
Name ______________________   Date ____________

  1. Which change raises power?

    • Smaller samples with noisier measurement
    • Larger samples with quieter measurement
    • Stricter thresholds with tinier samples

  2. When is power fixed?

    • Before data collection, by design choices
    • After publication, by reader votes
    • During peer review, by editor preference

  3. Computing power after the fact from the observed effect is an honest planning step.

    Circle one:   True   False

  4. Your required n swings wildly across plausible effects. What does that mean?

    • The design is fragile and needs better measurement or budget first
    • The math is broken and power is irrelevant
    • You should enroll the smallest n and hope

  5. A tiny p value marks a trivial effect measured with huge n. What keeps the ideas apart?

    • Reporting the effect size alongside significance
    • Deleting the p value entirely
    • Collecting even more data blindly

  6. An underpowered study reports a significant huge effect. How do you read it?

    • As exact truth, since significance proves size
    • With caution, since low power filters for exaggerations
    • As proof that power was actually high

  7. Which planning sequence is honest?

    • Collect data, peek, then choose the effect that fits
    • State the smallest useful effect, then solve for n under stated error rates
    • Fix n by budget, then declare whatever appears as the target

  8. A colleague sizes a replication from the first flashy estimate. What do you warn?

    • Replications must always be smaller than originals
    • Effect sizes never matter for planning
    • Flashy early numbers inflate, so the replication will be sized too small
LightMySky · lightmysky.comW1-mt_zYNmSWAJT8-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Planning power before you collect data W1-mt_zYNmSWAJT8-s1

  1. Larger samples with quieter measurement · More units and less noise make a real effect easier to catch.
  2. Before data collection, by design choices · Effect, variability, sample size, and threshold settle power up front.
  3. False · It restates the p value in new clothes and adds nothing.
  4. The design is fragile and needs better measurement or budget first · Sensitivity checks reveal fragility before the study starts.
  5. Reporting the effect size alongside significance · Sizes state how large a difference is, free from sample size.
  6. With caution, since low power filters for exaggerations · Accurate estimates fall short while lucky overestimates sail through.
  7. State the smallest useful effect, then solve for n under stated error rates · The smallest useful effect is a scientific decision made before running.
  8. Flashy early numbers inflate, so the replication will be sized too small · Discounting early winners is arithmetic, not cynicism.
Worksheet · LightMySky