Generative Models: Learning a Distribution You Can Sample
A generative model is judged by what it produces rather than by a label it predicts. Latent-variable models put a simple distribution behind the data and learn a decoder from it, which is what makes sampling easy and the likelihood awkward.
What a learner can do afterwards
- Say what separates a generative model from a discriminative one trained on the same data
- Explain the job of a latent variable and why training optimises a bound rather than the likelihood
- Name two ways to judge samples when there is no accuracy to report
1 · Read
A discriminative model predicts a label for data it is shown. A generative model learns the distribution behind the data, so you can sample brand new points from it. Same data, opposite jobs.
A latent-variable model hides a simple code behind each example and learns a decoder from code to data. The exact likelihood sums over every possible code, which is far too many, so training optimises a lower bound instead. The bound trades exactness for an objective you can actually compute.
With no accuracy to report, you judge samples two ways. Ask human raters to score how convincing samples look, and compare likelihood bounds on held-out data across models.
When someone asks why not the likelihood itself, answer with the sum: every code must be considered, and that sum is intractable.
Generative models learn to sample, latent codes need a bound, and samples are judged by raters and bounds.
2 · Watch
Take it off screen
Where it sits
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.