---
title: "Generative Models: Learning a Distribution You Can Sample"
description: "A generative model is judged by what it produces rather than by a label it predicts. Latent-variable models put a simple distribution behind the data and learn a decoder from it, which is what makes s"
canonical: https://lightmysky.com/learn/computing/generative-models-learning-a-distribution-you-can-sample-mt_3CHsmpWVX5
source: https://lightmysky.com/learn/computing/generative-models-learning-a-distribution-you-can-sample-mt_3CHsmpWVX5.md
retrieved: 2026-09-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# Generative Models: Learning a Distribution You Can Sample

A generative model is judged by what it produces rather than by a label it predicts. Latent-variable models put a simple distribution behind the data and learn a decoder from it, which is what makes sampling easy and the likelihood awkward.

Subject: Computing · Area: Machine Learning · Ages 22 to 24
Page: https://lightmysky.com/learn/computing/generative-models-learning-a-distribution-you-can-sample-mt_3CHsmpWVX5

## Ready when they can

- Say what separates a generative model from a discriminative one trained on the same data
- Explain the job of a latent variable and why training optimises a bound rather than the likelihood
- Name two ways to judge samples when there is no accuracy to report

## Lesson: Models that make new things

A discriminative model predicts a label for data it is shown. A generative model learns the distribution behind the data, so you can sample brand new points from it. Same data, opposite jobs.

**Example.** A latent-variable model hides a simple code behind each example and learns a decoder from code to data. The exact likelihood sums over every possible code, which is far too many, so training optimises a lower bound instead. The bound trades exactness for an objective you can actually compute.

With no accuracy to report, you judge samples two ways. Ask human raters to score how convincing samples look, and compare likelihood bounds on held-out data across models.

**Tip.** When someone asks why not the likelihood itself, answer with the sum: every code must be considered, and that sum is intractable.

**Recap.** Generative models learn to sample, latent codes need a bound, and samples are judged by raters and bounds.

## Practice

8 questions on this page, each with its working shown.

## Needs first

- [Clustering: Structure Without Labels](https://lightmysky.com/learn/computing/clustering-structure-without-labels-mt_BAMGOlWgRY)
- [Principal Components and Dimensionality Reduction](https://lightmysky.com/learn/computing/principal-components-and-dimensionality-reduction-mt_l-9XHg_ChZ)
- [Adapting a Pretrained Model: Fine-Tuning and Low-Rank Updates](https://lightmysky.com/learn/computing/adapting-a-pretrained-model-fine-tuning-and-low-rank-updates-mt_NaA2iGtMPc)

## Opens up

- [Diffusion Models: Learning to Undo Noise](https://lightmysky.com/learn/computing/diffusion-models-learning-to-undo-noise-mt_kZdQJ96iZ3)