Generative AI is the kind of artificial intelligence that makes new things: a paragraph, a picture, a song, a block of code. When you ask ChatGPT to draft an email or ask an image app for a poster, you are using generative AI. This guide explains what the term means, how these systems learn, the main model types behind them, what they are used for, and the risks worth knowing before you rely on one.
What generative AI means
IBM defines generative AI as "artificial intelligence (AI) that can create original content such as text, images, video, audio or software code in response to a user's prompt or request." AWS uses a similar definition: "a type of AI that can create new content and ideas, including conversations, stories, images, videos, and music."
The US government has used a more technical wording. NIST's Generative AI Profile (NIST AI 600-1, July 2024) quotes Executive Order 14110, which defined generative AI as "the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content." Put plainly: the model studies examples, then produces new content that resembles them.
How it differs from other AI
MIT News draws the line this way. Before the recent boom, "when people talked about AI, typically they were talking about machine-learning models that can learn to make a prediction based on data." Its examples: a model that predicts whether an X-ray shows signs of a tumor, or whether a borrower is likely to default on a loan.
A generative model, MIT says, "is trained to create new data, rather than making a prediction about a specific dataset." It "learns to generate more objects that look like the data it was trained on."
AWS describes the same split in more technical terms. Traditional models were "discriminative or focused on classifying data points." Generative models "try to predict features given a certain label." Its example: a generative model studies animal images, learns what ears, eyes, tails and skin patterns look like, and can then "recreate new animal images that were not in the training set."
The boundary is not always sharp. MIT quotes Phillip Isola, an MIT associate professor: "Oftentimes, the same algorithms can be used for both."
Where generative AI fits in the bigger picture: it is a subset of AI, and AWS calls it "a subset that generates new content meaningfully and intelligently." For how AI, machine learning and deep learning nest inside each other, see our explainer on AI vs machine learning vs deep learning.
How generative AI works
IBM splits the process into three phases: training, tuning, and generation with evaluation and retuning.
Training a foundation model
Everything starts with a foundation model. AWS describes foundation models as "ML models trained on a broad spectrum of generalized and unlabeled data" that can handle "a wide variety of general tasks." IBM says the most common foundation models today are large language models (LLMs), built for text, but there are also foundation models for images, video, sound and music, and multimodal models that handle several kinds of content.
Training is mostly a guessing game played billions of times. IBM says the model "performs and evaluates millions of 'fill in the blank' exercises, trying to predict the next element in a sequence," such as the next word in a sentence, and keeps adjusting itself to shrink the gap between its guess and the real data. What comes out is a network of parameters: numbers that encode the patterns in the training data.
This step is costly. IBM says it "requires thousands of clustered graphics processing units (GPUs) and weeks of processing, all of which costs millions of dollars." That cost is one reason open models matter: a developer can start from a model someone else already trained.
For text models, the chunks being predicted are called tokens. Our guide to what tokens are explains how models split text and how that affects pricing.
Tuning it for a job
A foundation model is a generalist. IBM's phrase: "It knows a lot about a lot of types of content, but often can't generate specific types of output with desired accuracy or fidelity." Two common ways to specialize it:
- Fine-tuning: feeding the model labeled examples of the questions it will get and the answers you want, in the format you want. IBM's example is a customer service chatbot trained on hundreds or thousands of labeled support questions and correct answers.
- Reinforcement learning with human feedback (RLHF): people rate or correct the model's outputs, and the model is updated using those ratings. IBM says this often means people "scoring" different answers to the same prompt.
Generating, checking and retuning
Once deployed, the app's outputs are assessed and the model is tuned again. IBM says developers may retune "as often as once a week," while the underlying foundation model is updated far less often, "perhaps every year or 18 months."
A third option is retrieval-augmented generation (RAG), which lets the model look things up in outside sources at answer time. IBM says RAG "can ensure that a generative AI app always has access to the most current information." Our RAG explainer covers it in depth.
The main types of generative models
Generative AI is not one technique. Four model families come up again and again in the explainers from IBM, AWS and MIT.
Variational autoencoders (VAEs)
A VAE uses two networks. AWS says the encoder compresses input into a compact "latent space," a numeric code for the data's attributes, and the decoder turns a point in that space back into data that resembles the original. IBM says VAEs were introduced in 2013 and that early uses included anomaly detection, such as medical image analysis. The paper usually cited is Auto-Encoding Variational Bayes by Kingma and Welling.
Generative adversarial networks (GANs)
GANs set two networks against each other. The original 2014 paper by Ian Goodfellow and colleagues at the Université de Montréal describes "a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G." The authors compare the generator to "a team of counterfeiters" and the discriminator to "the police." MIT notes that the image generator StyleGAN is based on this type of model.
Diffusion models
A diffusion model learns to remove noise. The 2015 paper by Sohl-Dickstein and colleagues at Stanford and UC Berkeley describes the idea: "slowly destroy structure in a data distribution through an iterative forward diffusion process," then learn "a reverse diffusion process that restores structure in data." AWS describes the same two steps in plain terms: add noise over several rounds, then reverse the process to produce a new sample. MIT says a diffusion model is "at the heart of" Stable Diffusion, and IBM says DALL-E is driven by one. (IBM dates diffusion models to 2014; the paper above was posted in 2015, which matches MIT's account.)
Transformers
Transformers are behind chatbots like ChatGPT. MIT says Google researchers introduced the architecture in 2017 and that it "has been used to develop large language models, like those that power ChatGPT." A transformer encodes each word as a token and builds an attention map that "captures each token's relationships with all other tokens," which helps it keep track of context. IBM calls the transformer "the deep learning model architecture behind the foremost foundation models and generative AI solutions today." For a full walkthrough, see Transformers Explained.
What ties these together, MIT says, is that they "convert inputs into a set of tokens, which are numerical representations of chunks of data." If data can be turned into tokens, the same methods can in theory generate more of it.
A short history
Generative AI feels new, but the ideas are old. MIT points to the Markov chain, a statistical method Andrey Markov introduced in 1906, which has long been used for next-word prediction such as email autocomplete. The limit, MIT says, is that such simple models only look back a word or a few words, so "they aren't good at generating plausible text."
IBM's timeline adds a few milestones:
- 1964: Joseph Weizenbaum at MIT develops ELIZA, an early chatbot that used pattern-matching scripts.
- 2004: Google autocomplete appears, which IBM describes as based on a Markov chain.
- 2013 to 2017: VAEs, GANs, diffusion models and transformers arrive (see above).
- 2019 to 2020: OpenAI releases GPT-2 and GPT-3.
- 2022: OpenAI introduces ChatGPT.
MIT quotes Tommi Jaakkola, an MIT professor, on what changed: "the major distinction here is in terms of the complexity of objects we can generate and the scale at which we can train these models."
What generative AI can make
IBM lists the main output types:
- Text: documentation, emails, web copy, articles, reports and creative writing, plus summaries of long documents.
- Images and video: IBM names DALL-E, Midjourney and Stable Diffusion for images, and says video tools can create animations from text prompts.
- Sound, speech and music: natural-sounding speech for assistants and audiobook narration, and original music.
- Software code: writing new code, autocompleting snippets, translating between programming languages and summarizing what code does.
- Synthetic data: IBM's example is drug discovery, where models generate molecular structures with desired properties.
MIT adds research uses: Isola's group generates synthetic images to train computer vision systems, and Jaakkola's group uses generative AI to design protein structures and crystal structures for new materials.
It is not the right tool for everything. MIT quotes Devavrat Shah, an MIT professor, saying that for predictions on structured data, "like the tabular data in a spreadsheet, generative AI models tend to be outperformed by traditional machine-learning methods."
Generative AI and AI agents
Some products go a step further than generating content. IBM describes an AI agent as a program that can "perform tasks and accomplish goals on behalf of a user or another system without human intervention," using other apps and services as tools. Its example: a generative AI app can tell you the best time to climb Mount Everest given your schedule, while an agent can also book the flight and hotel. Our explainer on what an AI agent is covers how they work.
Risks and limits
The ability to produce convincing content on demand also brings new risks. NIST's Generative AI Profile lists 12 risks "unique to or exacerbated by" generative AI:
- CBRN information or capabilities (chemical, biological, radiological or nuclear weapons)
- Confabulation
- Dangerous, violent or hateful content
- Data privacy
- Environmental impacts
- Harmful bias or homogenization
- Human-AI configuration
- Information integrity
- Information security
- Intellectual property
- Obscene, degrading and/or abusive content
- Value chain and component integration
Below are a few of these, with how IBM and NIST describe them.
Made-up answers. NIST defines confabulation as "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications') by which users may be misled or deceived." IBM's example is a lawyer whose AI research tool produced fictional cases, complete with quotes and attributions. Our guide to AI hallucinations covers how to catch them.
Different answers to the same question. IBM notes that because the models are probabilistic, "the same inputs can result in slightly or significantly different outputs."
Bias. IBM says models "may learn societal biases present in the training data," and MIT notes they can "amplify hate speech and false statements."
Over-reliance. NIST's human-AI configuration risk covers people who anthropomorphize these systems or experience "automation bias, over-reliance, or emotional entanglement" with them.
Copyright and privacy. MIT says models "can generate content that looks like it was produced by a specific human creator, raising potential copyright issues." IBM warns users to make sure data they put into a model, in a prompt or during tuning, does not expose their own or others' intellectual property.
Deepfakes. IBM describes deepfakes as AI-generated or AI-manipulated media "created to convince people that they're seeing, watching or hearing someone do or say something they never did or said," and notes their use in voice phishing scams and fraud.
Black boxes. Both IBM and AWS say these models are often hard to explain. AWS calls them "often considered black boxes."
How to start learning
AWS says generative AI training "begins with understanding foundational machine learning concepts," then moves on to neural networks, Python libraries such as TensorFlow or PyTorch, model evaluation, fine-tuning and prompt engineering. You do not need all of that to use the tools, but it is the path if you want to build them.
Free places to start on llms.blog:
- Best free AI courses, from Google, Hugging Face, fast.ai and others
- Free LLM courses for going deeper on text models
- How to learn AI from scratch, a 12-week plan
- The llms.blog AI glossary, for terms like token, parameter and fine-tuning
The short version
Generative AI is AI that creates new content instead of only classifying or predicting. It works by training a large foundation model on huge amounts of data, tuning it for a task, and checking its outputs. Chatbots like ChatGPT are built on transformers, and image tools like Stable Diffusion on diffusion models. It is useful for drafting, coding, design and research, and it can also be confidently wrong, biased, or used to deceive, so check what it tells you.
Sources
- IBM: What is generative AI?
- AWS: What is Generative AI?
- MIT News: Explained: Generative AI
- NIST AI 600-1: Generative Artificial Intelligence Profile (PDF)
- Goodfellow et al.: Generative Adversarial Nets (arXiv)
- Sohl-Dickstein et al.: Deep Unsupervised Learning using Nonequilibrium Thermodynamics (arXiv)
- Kingma and Welling: Auto-Encoding Variational Bayes (arXiv)
