Training Data: Where It Came From and What It Leaves Out · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Ask what the model was shown

Computing · Artificial Intelligence · ages 14-15
Name ______________________   Date ____________

  1. A model's training material ends in 2021. No newer information has been supplied. What can you say about reports of events that happened in 2024?

    • Its training automatically contains reports of every 2024 event
    • Those later events have no reports in its training material
    • The training date proves all its answers about 2024 are correct

  2. Which dataset questions help identify risks before testing a model for its intended users?

    • Is the file large enough to guarantee accurate results?
    • Does recent collection prove equal results for every group?
    • Who is represented, who is missing, and when was it collected?

  3. A team wants to use records about people for training. Which permission check is appropriate?

    • Assume any record found online permits every training use
    • Treat accurate predictions as permission to use the records
    • Check allowed uses and whether consent is needed

  4. In an imagined vehicle dataset, truck photos have blue backgrounds and bus photos have yellow backgrounds. A classifier's labels change when only the background changes. Which clue might it be using?

    • Only vehicle shape, with background having no effect
    • Background color rather than the intended vehicle features
    • The correct vehicle category regardless of changes to the photo

  5. A vehicle model is tested on held-out photos that repeat the same background colors as training. Why should the team also test different backgrounds in both categories?

    • Held-out photos guarantee that all background shortcuts disappear
    • Repeating training backgrounds proves every future setting is covered
    • Varied backgrounds can help expose a repeated shortcut

  6. A bus-delay model is intended for this year's weekday city trips. Dataset A records those trips. Dataset B records rural holiday trips from years ago. Which conclusion about relevance is justified?

    • A better matches the intended setting; testing is still needed
    • B guarantees better accuracy because older records contain no errors
    • Either set proves reliability without testing any new city journeys
    • Neither set can help because travel conditions sometimes change

  7. In a hypothetical case, a fitness model trained only on male athletes aged 25 to 40 will also be used with people aged 12 to 16. What should the missing age coverage lead the team to do?

    • Declare equal accuracy because both groups can exercise
    • Test the intended groups separately before relying on it
    • Declare worse accuracy for ages 12 to 16 without testing them

  8. In a hypothetical test, a model gets 18 of 20 adult cases right and 10 of 20 teen cases right. What do these results establish?

    • Leaving teens out of training was proved to cause the difference
    • Accuracy differed between these two tested groups
    • The combined result proves equally accurate predictions for each group
    • Every future adult prediction will be correct and every teen one mistaken

  9. A hiring model uses broadly representative records with appropriate consent. Does that alone establish that it is suitable for hiring decisions?

    • No; intended-use performance, group outcomes and harms still need review
    • Yes; representation and consent guarantee accurate and fair decisions
    • Yes; permission to train replaces the need to examine hiring outcomes

  10. A training cutoff guarantees that every fact about earlier events is correct.

    Circle one:   True   False

LightMySky · lightmysky.comW1-mt_1Lw0TMHShk-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Ask what the model was shown W1-mt_1Lw0TMHShk-s1

  1. Those later events have no reports in its training material · Reports of events that happened in 2024 are later than the stated training period. This does not prove that the model cannot infer anything about them.
  2. Who is represented, who is missing, and when was it collected? · Coverage and dates help identify relevant risks. Neither file size nor recent collection guarantees performance.
  3. Check allowed uses and whether consent is needed · Allowed use and consent requirements need checking in context. Online access or accurate predictions do not establish permission.
  4. Background color rather than the intended vehicle features · The labels respond to background changes, which suggests a background shortcut. Further tests are needed to investigate it.
  5. Varied backgrounds can help expose a repeated shortcut · Separate test photos can retain the training association. Different backgrounds in both categories can challenge that shortcut.
  6. A better matches the intended setting; testing is still needed · A matches the intended place, days and period more closely. That makes it more relevant evidence, not proof of predictive accuracy.
  7. Test the intended groups separately before relying on it · Training covers ages 25 to 40, not 12 to 16. That gap raises a coverage risk. Test the intended groups rather than assuming equal or worse accuracy.
  8. Accuracy differed between these two tested groups · The groups have equal test sizes but different correct counts. That establishes a difference in this test, not its cause or every future result.
  9. No; intended-use performance, group outcomes and harms still need review · Coverage and consent are useful evidence but do not establish accuracy, fair outcomes or acceptable harms.
  10. False · A cutoff describes a training period, not a guarantee of factual accuracy within that period.
Worksheet · LightMySky