Testing thousands of genes at once: false discovery rate and what q means
Testing thousands of genes requires error control across tests. This stop compares family-wise error and false discovery rate, interprets q-values under stated conventions, and explains conditions for valid filtering and covariate weighting.
What a learner can do afterwards
- Contrast Bonferroni with a false discovery rate procedure applied to the same p-value list
- Interpret a q-value of 0.05 as an estimated list-level error threshold under a stated definition, not one gene's null probability
- Explain how valid filtering or covariate weighting can change discoveries without guaranteeing an increase
1 · Read
A gene test can report a p-value. Under a true null, a valid p-value has probability at most t of being <= t. Rejecting a true null is a false positive. If 4,000 true nulls each have exact Type I error probability 0.05, expect 200 false positives. If their rates are only bounded by 0.05, expect at most 200. Sum over true nulls, not all tests; this expectation does not require independence.
Family-wise error is the probability of at least one false positive in the tested family. Bonferroni rejects when p <= alpha/m, where m is the total test count. With 5,000 tests and alpha = 0.05, use 0.00001. For valid p-values, the family-wise error is at most alpha even with dependent tests. This can reduce power, but it neither rules out true discoveries nor guarantees zero false positives.
Let R count all rejections and V count false rejections. The false discovery proportion is V/R when R > 0, and zero when R = 0. FDR is its expectation over repeated studies. FDR control at 0.05 bounds this expectation by 5%, not the false fraction in every observed list. It can allow more discoveries than family-wise control when some nulls are false.
Benjamini-Hochberg (BH) sorts m p-values from smallest to largest. Find the largest rank k with p(k) <= k*alpha/m, then reject every rank through k. If none qualifies, reject none. Standard BH controls FDR for valid p-values under independence or suitable positive dependence, not arbitrary dependence. Bonferroni instead compares every p-value with the fixed threshold alpha/m.
Positive FDR, or pFDR, is the expected false discovery proportion conditional on making at least one rejection. FDR also includes studies with no rejections, whose proportion is zero. The two targets can differ, especially when discoveries are rare. Neither target is the probability that one particular gene's null hypothesis is true.
Here a Storey-style q-value estimates the lowest pFDR over p-value cutoffs that include the gene. Selecting q <= 0.05 targets a conditional expected false fraction of at most 5%, subject to estimation and model assumptions. It does not guarantee the observed fraction. Some software uses q-value to mean a BH-adjusted p-value, which targets FDR instead. Check the reported method and definition.
Filtering removes tests before correction. Null p-values must remain valid after selection, and the procedure's dependence assumptions must still hold. A filter independent of null p-values can help preserve validity. Covariate weighting may keep all tests but allocate different thresholds. Its covariate and weight learning need safeguards against reusing the same null p-value signal. Neither approach guarantees more discoveries.
This sequence shows filtering. Weighting can retain the original test count and needs its own valid procedure.
Bonferroni bounds the chance of any false positive. FDR bounds an expected false fraction. Check a q-value's definition and assumptions; filtering and weighting need valid safeguards.
Original lesson, questions and flashcards about multiple-testing error control, written for this learning dataset.
Original reading and flashcards about family-wise error, false discovery rate and q-values, written for this learning dataset.
More to read
Error targets for a gene search
A false positive rejects a true null. If 3,000 nulls are true and each test has exact Type I error probability 0.05, the expected false-positive count is 150. This calculation uses true nulls, not every tested hypothesis.
Bonferroni addresses family-wise error: the probability of any false positive. Comparing valid p-values with alpha/m bounds that probability even with dependent tests. It can reduce power, but neither guarantees zero errors nor makes true discoveries impossible.
FDR averages the false discovery proportion over repeated studies, using zero when there are no rejections. BH sorts p-values, finds the largest rank k satisfying p(k) <= k*alpha/m, and rejects all ranks through k. Its usual guarantee requires valid p-values and independence or suitable positive dependence.
For four tests at alpha = 0.05, take sorted p-values 0.006, 0.016, 0.042 and 0.300. BH compares them with 0.0125, 0.025, 0.0375 and 0.05. Its largest qualifying rank is 2, so it reports two hits. Bonferroni uses 0.0125 throughout and reports one.
Positive FDR conditions the expected false proportion on having at least one rejection. A Storey-style q-value estimates the lowest positive FDR over cutoffs including a gene. Some software instead labels BH-adjusted p-values as q-values. Check the definition. Neither number is a gene's posterior null probability.
Filtering can remove tests; weighting can keep all tests. Both require valid null behaviour and the chosen procedure's assumptions after selection or weight learning. More discoveries are possible, not guaranteed.
2 · Watch
Take it off screen
Where it sits
Optional connections
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.