Generative Models: Learning a Distribution You Can Sample · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Models that make new things

Computing · Machine Learning · ages 22-24
Name ______________________   Date ____________

  1. What does the decoder in a latent-variable model do?

    • It turns hidden codes into data
    • It deletes the training set
    • It prints accuracy scores

  2. What is a generative model judged by?

    • The labels it predicts
    • What it produces when sampled
    • The speed of its typing

  3. Training a latent-variable model optimises a lower bound because the exact likelihood sums over too many codes.

    Circle one:   True   False

  4. What does the lower bound give up compared with the true likelihood?

    • Exactness, for an objective you can compute
    • All use of latent codes
    • The need for any data

  5. Two teams train on the same images. One classifies them, the other samples new ones. Which is generative?

    • The classifier, it labels fastest
    • Both equally, data is data
    • The sampler, it models the distribution

  6. Two generators compete with no labels available. How do you compare them?

    • Hold an election among engineers
    • Rate samples with judges and compare held-out bounds
    • Pick the one with more parameters

  7. A generator memorises training images pixel for pixel. Why do held-out bounds catch it?

    • Memorised models fail on unseen data, which bounds measure
    • Judges always spot copies
    • Large models cannot memorise

  8. A student maximises the bound and claims the likelihood rose equally. What is the error?

    • Bounds always equal likelihoods
    • Likelihoods cannot be discussed
    • The bound can loosen while rising, so likelihood may lag
LightMySky · lightmysky.comW1-mt_3CHsmpWVX5-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Models that make new things W1-mt_3CHsmpWVX5-s1

  1. It turns hidden codes into data · Codes in, data out: that mapping is the decoder.
  2. What it produces when sampled · Generation is about the quality of new samples.
  3. True · The full sum is intractable, so a tractable bound stands in.
  4. Exactness, for an objective you can compute · The bound trades exactness for tractability.
  5. The sampler, it models the distribution · Sampling new points is the generative job.
  6. Rate samples with judges and compare held-out bounds · Sample ratings plus held-out bounds judge without accuracy.
  7. Memorised models fail on unseen data, which bounds measure · Held-out measurement punishes anything that only replays training.
  8. The bound can loosen while rising, so likelihood may lag · A rising bound need not tighten against the likelihood.
Worksheet · LightMySky