Generative Models: Learning a Distribution You Can Sample · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Models that make new things

Computing · Machine Learning · ages 22-24
Name ______________________   Date ____________

  1. What does the decoder in a latent-variable model do?

    • It turns hidden codes into data
    • It compresses the training set into codes
    • It scores how convincing a sample looks

  2. When a generative model's job is to produce new samples, what is it judged by?

    • The accuracy of the labels it predicts
    • What it produces when sampled
    • The speed at which it decodes a code

  3. A latent-variable model with a continuous code and a neural decoder trains on a lower bound because its exact likelihood integrates over every possible code.

    Circle one:   True   False

  4. What does the lower bound give up compared with the true likelihood?

    • Exactness, for an objective you can compute
    • The latent codes, which the bound never sees
    • The decoder, which the bound replaces

  5. Two teams train on the same images. One only predicts labels for them, the other samples new ones. Which team must be using a generative model?

    • The classifier, since labels are what it learned
    • Both equally, data is data
    • The sampler, it models the distribution

  6. Two generators compete with no labels available. How do you compare them?

    • Compare their accuracy on a held-out set of labelled images
    • Rate samples with judges and compare held-out bounds
    • Pick the one whose training loss finished lowest

  7. A generator memorises training images pixel for pixel. Why do held-out bounds catch it?

    • Memorised models fail on unseen data, which bounds measure
    • Judges always spot a copied training image
    • Large models cannot memorise individual images

  8. A student maximises the bound and claims the likelihood rose equally. What is the error?

    • Bounds always rise by exactly as much as likelihoods
    • Likelihoods cannot rise once the bound has been maximised
    • The gap can change, so the likelihood need not rise by the same amount
LightMySky · lightmysky.comW1-mt_3CHsmpWVX5-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Models that make new things W1-mt_3CHsmpWVX5-s1

  1. It turns hidden codes into data · Codes in, data out: that mapping is the decoder.
  2. What it produces when sampled · Generation is about the quality of new samples.
  3. True · That integral has no closed form and cannot be computed, so a computable bound stands in.
  4. Exactness, for an objective you can compute · The bound trades exactness for tractability.
  5. The sampler, it models the distribution · Sampling new points is the generative job.
  6. Rate samples with judges and compare held-out bounds · Sample ratings plus held-out bounds judge without accuracy.
  7. Memorised models fail on unseen data, which bounds measure · Held-out measurement punishes anything that only replays training.
  8. The gap can change, so the likelihood need not rise by the same amount · The bound sits below the likelihood, and the gap between them can widen or narrow, so the likelihood does not have to rise by the same amount.
Worksheet · LightMySky