Benchmarks and the Traps in Them
A benchmark is a proxy that becomes a target and then stops measuring what it did. Test data leaks into training sets, baselines are tuned less than the proposed method, and a leaderboard everyone reports against is being overfitted by a whole field at once.
What a learner can do afterwards
- Give three ways a reported score can be correct and still misleading
- Say how contamination happens when a benchmark predates the pretraining corpus
- Describe an evaluation that would separate a real gain from a tuning artefact
1 · Read
Models learn on training data and are judged on separate testing data they never saw. A model can memorize training instead of learning rules, which is overfitting: brilliant on seen data, failing on new cases. The opposite failure is underfitting, where the model is too simple to catch the real pattern at all.
A reported score can be correct arithmetic and still mislead three ways. Test data may leak into training, the test set may be reused until teams quietly tune to it, or baselines may be tuned less than the new method. Each keeps the number right while breaking what the number means.
Old benchmarks contaminate easily: a test written before the pretraining corpus may already sit inside the training text, so the model has seen the exam. Worse, a whole field can overfit one leaderboard together, with every team optimizing the same number until it rises while real skill on fresh cases stalls.
Separate real gains from tuning artefacts with fresh data the model never saw, and tune the baseline as hard as the new method. Count tuning choices as part of the method, and list what would have to be true for the score to mean what it claims.
Keep the test separate, suspect every reused number, and prove gains on fresh data with equal tuning.
2 · Watch
Take it off screen
Where it sits
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.