---
title: "Researcher Degrees of Freedom and the Pre-registered Analysis Plan"
description: "Every undeclared choice among exclusion rules, transformations, subgroups and stopping points widens the range of results a true null can produce. This stop writes an analysis plan that fixes those ch"
canonical: https://lightmysky.com/learn/science/researcher-degrees-of-freedom-and-the-pre-registered-analysis-plan-mt_UJgZQ2AADm
source: https://lightmysky.com/learn/science/researcher-degrees-of-freedom-and-the-pre-registered-analysis-plan-mt_UJgZQ2AADm.md
retrieved: 2026-09-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# Researcher Degrees of Freedom and the Pre-registered Analysis Plan

Every undeclared choice among exclusion rules, transformations, subgroups and stopping points widens the range of results a true null can produce. This stop writes an analysis plan that fixes those choices in advance.

Subject: Science · Area: Scientific Inquiry · Ages 22 to 24
Page: https://lightmysky.com/learn/science/researcher-degrees-of-freedom-and-the-pre-registered-analysis-plan-mt_UJgZQ2AADm

## Ready when they can

- List the analysis decisions in a stated study that could have reasonably gone another way
- Explain how looking at the data and deciding whether to continue raises the false positive rate
- Write a short pre-registration covering hypothesis, exclusion rules and the primary analysis

## Lesson: Locking your analysis plan in advance

You map the garden of forking paths before you enter it. Outlier rules, covariates, outcomes, and stopping points each look innocent alone, yet together they offer many routes. Trying them all and reporting the prettiest is not discovery.

You learn why peeking bends error rates. Each interim look is another throw at significance, and stopping on a lucky peak freezes noise at its best moment. Extreme early results regress toward the mean on repeat, so fixed samples or honest error spending keep the rate where promised.

**Example.** You run a small noisy study and only the exaggerated estimate clears the bar. You feel tempted to publish it at face value. Then you remember the filter: accurate estimates fell short and vanished, while the lucky overestimate sailed through.

**Tip.** You write a short pre-registration naming the hypothesis, the sample size, the exclusion rules, and the exact primary analysis. You share data and code so others can retrace your path. Exploration stays free, but you label it honestly as exploration apart from the confirmatory test.

**Recap.** You fence the garden beforehand, you fix the stopping rule, and you label exploration as exploration.

## Practice

8 questions on this page, each with its working shown.

## Needs first

- [Power and Effect Size, Decided Before the Data](https://lightmysky.com/learn/science/power-and-effect-size-decided-before-the-data-mt_zYNmSWAJT8)

## Opens up

- [Reading a Primary Paper: The Claim, the Figure and the Missing Control](https://lightmysky.com/learn/science/reading-a-primary-paper-the-claim-the-figure-and-the-missing-control-mt_DZE7z5vu9X)