Who Was Asked: Samples and the People Left Out
A statistic is about the people who answered, not the people you care about. Covers the difference between a population and a sample, why the method of asking decides who is missing, and how a poll of a self-selected group can be precisely wrong.
What a learner can do afterwards
- State the population a given claim is really about, given how the data was collected
- Name two ways of choosing a sample that leave out a predictable group, and say who is missing in each
- Explain why a larger self-selected sample does not fix a biased one
1 · Read
A statistic describes the people who answered, not the people you care about. The population is everyone the claim is about. The sample is the people actually measured. A website poll of its own readers speaks about those readers, never the whole public.
The method of asking decides who is missing. Landline calls in work hours miss mobile-only homes and workers. A survey at one station at noon misses everyone elsewhere. Convenience and volunteer samples favour whoever is easiest to reach.
A bigger self-selected sample never fixes the tilt. Ten thousand extra clicks still exclude everyone offline. Bias is built into the method, while the normal wobble between fair samples is just variation.
Meet every headline with three questions: who was asked, who could not answer, and who chose to stay quiet. If any answer narrows the group, shrink the claim to match.
Ask who answered, who was shut out, and whether a bigger count would only measure the wrong group more precisely.
More to read
1.2 Data, Sampling, and Variation in Data and Sampling
Every dataset starts with two questions: who counts as the population, and how were the sampled people picked from it? Random sampling gives everyone a fair shot, while convenience and volunteer samples quietly favour whoever is easiest to reach. Variation between samples is normal, but bias is a tilt built into the method itself. Learn the sampling methods by name and you can smell a skewed poll in seconds.
1.4 Experimental Design and Ethics
Good experiments split people into groups by chance, treat the groups identically apart from one change, and hide who got what where possible. Without random assignment you cannot tell whether the treatment worked or the groups just differed to begin with. Ethics matter too: nobody should be harmed or tricked for the sake of cleaner data.
1.6 Sampling Experiment
Trying sampling yourself beats reading about it. Take a real population like the songs on your phone, draw samples by different rules, and watch the averages jump around. Random draws cluster near the truth while cherry picked ones drift off. The spread of your results shows sampling variation live, and the drift shows bias. One afternoon of this teaches more than ten headlines about polls.
1.5 Data Collection Experiment
Surveys, observations and experiments each collect data with different strengths and traps. A survey is cheap but answers bend toward whoever bothers to reply. Observations see real behaviour but cannot prove causes. Experiments can prove causes but only for people like those who took part. Matching the claim back to the collection method is the core move this topic trains.
Explore more
2 · Watch
Take it off screen
Where it sits
Optional connections
This opens up
Nothing builds on it yet.
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
14 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.