Bias in AI Systems
AI bias can produce unfair patterns of results across groups. Explore how training data and design choices can contribute, how to compare group results, and ways to reduce bias.
What a learner can do afterwards
- Explain what bias in AI means using a real-world example
- Describe how biased training data leads to biased AI results
- Suggest one way to reduce bias in an AI system (use more diverse data, test with different groups)
1 · Read
You have learned that AI predictions can be mistaken. AI bias is a repeated pattern of results that can treat some groups unfairly. People write the software, and many systems learn patterns from examples. Missing examples, unfair labels and design choices can contribute to bias.
Imagine a face-matching tool trained mostly on lighter-skinned faces. It may make more mistakes on darker-skinned faces because it had fewer useful examples. This is a risk to check, not a guaranteed result. Test new photos from different groups to measure how well the tool works.
Imagine a translation tool repeatedly choosing a male word for 'doctor' when the sentence gives no gender clue. A pattern in its training text could contribute. The job title alone does not tell us the person's gender. Repeating that assumption can reinforce an unfair stereotype.
To reduce bias, check labels, add useful examples from missing groups and examine design choices. Compare results on new examples from each group before and after changes. A high overall score can hide poor results for one group. A gap shows what needs checking; it does not prove its cause. Accuracy is one check, not a complete test of fairness.
Data and design can contribute to AI bias. Compare group results, investigate gaps and test whether changes help.
Lesson, questions and flashcards written for this dataset. Face-matching examples, translation scenarios and chart results are hypothetical.
These readings were independently written for this dataset. The research summary describes work by Joy Buolamwini and Timnit Gebru, studied through Khan Academy's Bias in facial recognition.
More to read
Records can repeat an unfair past
Imagine a school once invited mainly students from one class to a club. Its records contain many members from that class and few from others.
A tool trained to copy those invitations might repeat the difference. Class membership could become a clue, even though students from other classes want to join too.
The records show past choices. They do not prove those choices were fair or that copying them is a good goal.
The team should ask who was missed, check which information guides invitations and test a revised system. Correctly copying an old record can still repeat unfair treatment.
Measure whether a change helped
Imagine an app that hears adult voices more accurately than children's voices. The team improves its training examples.
To compare versions, keep new test recordings from both groups. These recordings must not be used for training. Try the old and new versions on the same test recordings.
Report child results and adult results separately. A better combined score might still hide poor results for children. A change could also affect adults differently.
Compare the results before claiming success. Useful training changes need testing, and a better test score does not answer every question about fairness.
A real comparison across groups
Joy Buolamwini and Timnit Gebru studied commercial systems that assigned gender labels to face photos. They compared accuracy across groups. The systems they tested performed least accurately on darker-skinned women.
This task was gender classification. It was different from checking whether two photos show the same person. Tests of different tasks answer different questions.
The researchers' comparison revealed differences that one overall score could hide. It showed why teams need to examine group results.
An observed gap needs investigation. The gap alone does not establish every cause of the unequal results.
2 · Watch
En anglaisTake it off screen
Where it sits
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.