Start with the big idea
A generative model learns useful patterns from examples and produces candidate samples that resemble those patterns. This does not mean it stores a single template or truly understands every sample.
Understand it in more detail
An autoregressive model predicts a distribution for the next unit, such as a text token, conditioned on previous units; repeating the process generates a sequence. This is common in language models.
Diffusion models commonly learn to reverse progressive corruption, iteratively turning noisy inputs into structured samples. A VAE learns a probabilistic latent representation and decodes sampled latent variables into outputs.
GANs train a generator alongside a discriminator in an adversarial objective. Normalizing flows use invertible transformations to map between simple and complicated probability distributions. Real systems may blend techniques or use different inference schedules.
Worked example
To generate a short story, an autoregressive model samples a continuation token repeatedly. To synthesize an image, a diffusion approach may start from noise and refine it through multiple steps. These are different generation processes.
A misconception to avoid
BERT is primarily an encoder-style masked-language model, not a standard left-to-right autoregressive generator. Synthetic data is not automatically privacy-preserving.
Check your understanding
Think first, then open each answer to check your explanation. These self-checks do not record a score.
1. Does an image generator always create an image in one step?
No. Some diffusion systems use multiple denoising steps; different architectures vary.
2. Can synthetic examples accidentally reveal training data?
Yes. Privacy needs testing and controls even when data is synthetic.
Try it yourself
Make a flowchart comparing next-token generation and iterative image denoising. Label what a model learns, what it samples, and where errors could enter.
Explain it back: Summarize the main idea in your own words and name one limitation or risk.
Continue learning
Read an independent reference: https://arxiv.org/abs/2006.11239. This link is for reference, not a requirement to create an account. External websites may have different privacy and access policies.
Educational pilot. Terminology, age suitability and activities are subject to expert review. These pages do not send your answers to a server.