Models that produce rather than classify.
WHAT THEY DO
Learn a distribution and sample from it.
WHAT THE APPROACHES ARE
- Autoregressive: predicting the next element, repeatedly
- Diffusion: gradually removing noise from randomness
- Adversarial: a generator trained against a discriminator
- Variational autoencoders: encoding to a latent space and decoding
WHAT AUTOREGRESSIVE MODELS PRODUCE
Text, code, and increasingly other sequences.
WHAT DIFFUSION MODELS PRODUCE
Images, audio and video, at high quality.
WHY DIFFUSION DISPLACED ADVERSARIAL METHODS FOR IMAGES
More stable training, and better diversity.
WHAT SAMPLING PARAMETERS CONTROL
How deterministic or varied output is.
WHAT TEMPERATURE DOES
Flattens or sharpens the distribution, increasing or reducing variety.
WHAT TO SET IT TO
Low for factual or structured output; higher for creative variety.
WHAT GENERATIVE MODELS DO NOT DO
Retrieve facts reliably Guarantee correctness Know what they do not know
WHAT THAT REQUIRES
Verification of anything consequential.
WHAT EVALUATION IS LIKE
Difficult, since there is no single correct output.
WHAT TO USE
Task-specific measures, and human judgement.
