Designing Fair AI Rules
Apply design thinking to AI ethics. If you were designing an AI system, what rules would you give it? Who should it help? What should it not be allowed to do?
What a learner can do afterwards
- Propose at least three rules for a hypothetical AI system they are designing
- Explain why AI designers need to think about who might be harmed, not just who benefits
- Describe a real example of checking AI for unfair results and a real example of an unfair AI result
1 · Read
You have learned that people are responsible for checking important AI decisions. Designers set goals and limits: who a system should help, what it may do and what it must not do. Writing rules is a start. People must also test how the system behaves and address problems.
Imagine a school book helper. Three rules could be: offer useful books to every student; keep each student's reading level private; let a teacher check and change recommendations. These address access, privacy and mistakes. Helping every student does not mean suggesting the same book to everyone.
Ask who benefits and who could be missed or harmed. A benefit count does not answer both questions. Percent means out of 100. If 90 percent benefit, that is 90 out of 100. Find out what happens to the other 10. Ask affected people about their needs, then test and revise the rules.
Joy Buolamwini and Timnit Gebru tested commercial systems that assigned gender labels to face photos. They compared accuracy across groups and found unequal results. Checking for such differences is a responsible step. It does not prove that every use of a tested system is fair.
A Khan Academy account describes an experimental Amazon hiring tool. It lowered scores for some references to women, including 'women's' in a club name. The tool had learned from past resume data. This is an example of an unfair scoring pattern. The account does not establish that the tool rejected job applicants.
Choose tests that check the behavior a rule requires. To test usefulness, include different needs and ask users how well suggestions fit. For example, if a school helper must accept typed questions, type a question and check its reply. Record the result, repair any problem and repeat the test.
Write rules for access, privacy and correction. Check who could be harmed, test what happens and revise the design.
Text written for this dataset. Real examples summarize research by Joy Buolamwini and Timnit Gebru and a reported Amazon hiring-tool experiment described by Khan Academy. School-helper scenarios and benefit counts are hypothetical.
These readings were independently written for this dataset. The book-helper tests and accessibility scenario are hypothetical.
More to read
Give each rule a test
Imagine a school book helper with a rule to offer useful recommendations to every student. Try it with students who have different interests and reading needs. Ask whether its suggestions fit them. Include students who rarely borrow books.
For a privacy rule, check who can see a reading record. It should not appear on a display for classmates.
For a correction rule, try changing a recommendation that does not fit. Can the teacher review it and make the change?
Record what each test found. If the system breaks a rule, revise it and test again. A written promise needs checks of what actually happens.
Ask who the design misses
Imagine a book helper that displays recommendations only as pictures without readable book names. Some students like the display. A student using a screen reader cannot get the names from it.
Counting only satisfied users would miss that barrier. Ask the affected student what information they need. Add readable book names and test the change with the screen reader.
People who have difficulty using a system can explain problems a popularity score leaves out.
After revising the design, ask those people to try it again. Their feedback helps the team check whether the barrier remains.
2 · Watch
En anglaisTake it off screen
Where it sits
Where this leads
Jobs that lean on this skill. Follow one to see everything it is built on.
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.