A model can be accurate overall and still be unfair

Computing · Machine Learning · ages 21-22

Name ______________________ Date ____________

A model gets 171 of 180 group A cases and 9 of 20 group B cases correct. Overall accuracy is 90%. What does that score hide?
- Both groups have 90% accuracy on the cases tested.
- Group A has 95% accuracy and group B has 45%.
- Group B is more accurate because its sample is smaller.
A model is trained to imitate biased past lending decisions. What risk should an audit check?
- It may repeat patterns of unfair loan denial.
- It must remove bias once enough examples are supplied.
- It cannot learn patterns from past lending decisions.
Matching true positive rates and false positive rates across groups guarantees equal overall approval rates, even when the groups' eligible shares differ.
Circle one: True False
Which arrangement provides accountability for an automated lending system?
- A named team answers for decisions and handles review.
- The vendor reports accuracy but names no responsible team.
- Staff send every complaint back to the model's prediction.
What does redlining mean in this lesson's lending example?
- Using a map to choose nearby bank branches.
- Discriminatory loan exclusion by neighbourhood.
- Comparing local flood risks to estimate losses.
Group A has 4 approvals among 20 ineligible applicants. Group B has 8 approvals among 80 ineligible applicants. Which false positive rate comparison is correct?
- A has 20%; B has 10%, so A's rate is higher.
- A has 4%; B has 8%, so B's rate is higher.
- A has 10%; B has 20%, so B's rate is higher.
A loan tool rejects Maya. Which response meets this lesson's ethical standard, without assuming a particular country's law?
- Give only the overall accuracy and refuse further discussion.
- Promise approval before anyone reviews her application.
- Explain the reason and offer meaningful human review.
Groups A and B have eligible shares of 25% and 75%. Both have an 80% true positive rate and a 20% false positive rate. Which report is supported?
- Demographic parity holds; approval rates are 50% and 50%.
- Both criteria hold; approval rates are 80% in each group.
- Equalized odds holds; approval rates are 35% and 65%.

Answer key

For grown-ups. Fold this page away before handing over the rest.

A model can be accurate overall and still be unfair W1-mt_tTh4WtAyk5-s1

Group A has 95% accuracy and group B has 45%. · A has 171 / 180 = 95% accuracy; B has 9 / 20 = 45%. Combining counts gives 180 / 200 = 90%, weighted toward the larger group.
It may repeat patterns of unfair loan denial. · Imitating biased decisions may repeat unfair denials. More examples do not automatically make the decision target fair.
False · With shared rates t and f, approval share is f + (t - f)p. Different eligible shares p produce different approval shares when t > f, so equality is not guaranteed.
A named team answers for decisions and handles review. · Accountability requires people who remain responsible. Publishing a score or repeating a prediction does not answer a complaint or provide review.
Discriminatory loan exclusion by neighbourhood. · The defining issue is discriminatory restriction of lending by neighbourhood. Using geographic information for another purpose does not by itself establish redlining.
A has 20%; B has 10%, so A's rate is higher. · False positive rates use ineligible applicants as denominators: 4 / 20 = 20% for A and 8 / 80 = 10% for B. Comparing counts alone reverses the ordering.
Explain the reason and offer meaningful human review. · Maya should receive an understandable reason and a way to challenge the decision through human review with authority to change it. Review does not promise approval.
Equalized odds holds; approval rates are 35% and 65%. · Matching true positive and false positive rates meet equalized odds. Approval shares are 0.2 + 0.6(0.25) = 35% and 0.2 + 0.6(0.75) = 65%, so demographic parity is not met.