Generative AI (GenAI) systems create new content — sentences, images, audio, code — by sampling from a model of how data is distributed. They exist because many products need to produce artifacts, not just classify or score them.
A discriminative model answers: "Given this email, is it spam?" It learns the probability of a label given the input, or a decision boundary. A generative model answers: "What would a plausible email look like?" Generation means drawing a new sample from that learned distribution — not looking up a stored document (though retrieval can help ground generation).
Think of weather. Discriminative: given today's sensors, will it rain? Generative: simulate tomorrow's radar map consistent with climate statistics. Both are useful; they optimize different questions. Product teams often need both: generate a draft reply, then classify whether it is safe to send.
GenAI sits inside the AI -> ML -> DL nesting for most modern systems: deep networks learn a distribution; decoding samples from it. Older generative ideas (n-gram language models, hidden Markov models, classic topic models) share the sampling idea with far less capacity.
Discriminative vs generative:
| Plain-English idea | When to use it |
|---|---|
| Discriminative — predict a label or score for a given input | Spam filters, fraud scores, image classifiers |
| Generative — produce new text, images, audio, or structured objects | Chatbots, image generators, code assistants |
Many modern systems blend both: a Large Language Model (LLM) generates text (generative) while a separate classifier filters toxicity (discriminative). Choosing the wrong family wastes budget — do not sample essays when you only needed a calibrated yes/no.
Sampling new content. After training, you do not dump the entire distribution — you sample. Autoregressive language models generate one token at a time from the probability of the next token given previous tokens. Diffusion models iteratively denoise random noise toward an image. Generative Adversarial Networks (GANs) push a generator to fool a discriminator. Different architectures, same idea: randomness plus learned probabilities -> novel outputs. Decoding choices (greedy, temperature, top-k, top-p) trade diversity against coherence.
LLMs as generative models. Large Language Models are (usually) deep neural nets trained to predict the next token on massive text. At inference they are generative: given a prompt, they sample a continuation. Instruction tuning and Reinforcement Learning from Human Feedback (RLHF) shape which continuations users prefer, but the core act remains sampling from a conditional distribution over tokens. Tools, Retrieval-Augmented Generation (RAG), and structured outputs steer that sampling toward usefulness — they do not turn the model into a database by themselves.
Modalities. GenAI is not only text:
Not magic. Generation can be wrong, biased, or insecure. Models invent plausible citations, leak training snippets, or follow adversarial prompts. Treat outputs as proposals to verify, especially for facts, law, medicine, and security-sensitive code. Cost and latency also matter: every token or denoising step burns compute.
Illustrate "generation" by sampling from a categorical distribution — the same conceptual step an LLM takes at each token, stripped to bare Python.
import random
# Toy next-token distribution after the prompt "I love"
vocab = ["pizza", "coding", "rain", "meetings"]
probs = [0.40, 0.35, 0.15, 0.10] # must sum to 1.0
def sample_categorical(items, probabilities, rng=random):
r = rng.random()
cumulative = 0.0
for item, p in zip(items, probabilities):
cumulative += p
if r <= cumulative:
return item
return items[-1]
rng = random.Random(7)
samples = [sample_categorical(vocab, probs, rng) for _ in range(10)]
print("samples:", samples)
# Greedy "generation" always picks the mode — less diverse, more repetitive
greedy = vocab[probs.index(max(probs))]
print("greedy:", greedy)Real LLMs use huge vocabularies and context-dependent probabilities, plus decoding strategies (temperature, top-k, top-p). The tiny loop above is the heart of it: draw according to weights, not look up a single stored answer. Raise temperature (sharpen or flatten the distribution in real systems) and you change how wild the samples feel — same model, different sampling policy.
Generative AI samples new content from learned distributions — LLMs are the text-native case — powerful for creation, never a substitute for verification.