A large language model, or LLM, is a type of AI model that reads and writes text. IBM's explainer, What are large language models (LLMs)?, lists Anthropic's Claude, OpenAI's ChatGPT and Google's Gemini assistant among the interfaces that give the public access to LLMs, and defines LLMs as "a category of deep learning models trained on immense amounts of data, making them capable of understanding and generating natural language and other types of content to perform a wide range of tasks." Google's Machine Learning Crash Course puts it more plainly: LLMs "predict a token or sequence of tokens, sometimes many paragraphs worth of predicted tokens."
This guide explains what that means without math. It covers what the words "large" and "language model" refer to, how an LLM is trained, what happens inside one when you type a question, why it sometimes gets things wrong, and how the model relates to the chatbot apps people use every day.
The short version
- An LLM is a program that has learned patterns in a very large amount of text.
- It reads your prompt as small pieces called tokens.
- It writes a reply one token at a time, each time choosing a likely next token based on everything before it.
- It is trained first on broad text, then adjusted to follow instructions and to give answers people prefer.
- It can be fluent and still wrong, so facts that matter need checking.
What "language model" means
A language model is a system that predicts text. Google's course describes LLMs as "essentially autocomplete mechanisms that can automatically predict (complete) thousands of tokens." The autocomplete on a phone keyboard suggests the next word. An LLM does the same kind of prediction, but over whole paragraphs, and with far more context about what came before.
The unit of prediction is the token. Google's course says a token "can be a word, a subword (a subset of a word), or even a single character." OpenAI's help article on how ChatGPT and its foundation models are developed describes tokens as smaller units that "may represent whole words, parts of words, or punctuation." For a closer look at how tokens are counted, see our guide to what tokens are and how AI usage is billed.
What "large" means
"Large" refers to the number of parameters. OpenAI's help article explains that machine learning models "consist of large sets of numbers, known as 'weights' or 'parameters,' along with code that interprets and uses those numbers." Training adjusts those numbers so the model's predictions improve.
Google's course gives two reasons LLMs predict better than older language models: they "contain far more parameters than recurrent models" and they "gather far more context." It also says that "research shows that Transformers with more parameters consistently outperform Transformers with fewer parameters." IBM notes that smaller models exist too: "small language models" have comparatively few parameters, which makes them suitable for smaller devices.
How an LLM is trained
Training happens in stages. OpenAI's article names them as "preparing training data, pre-training, and post-training," followed by evaluation after deployment. IBM describes the post-training steps in more detail.
1. Pretraining. The model starts with a very large collection of text. IBM lists "books, articles, websites, code and other text sources," cleaned to remove errors, duplicates and unwanted content. The model then practices predicting text it cannot see. Google's course describes a version where "certain tokens in the training data are intentionally hidden" and the model tries to predict them. OpenAI's article gives a fill-in-the-blank example, "Instead of turning left, she turned ___," and says that early in training the model's answers "are largely random," improving as it processes more text. No one labels the data by hand for this step. IBM calls it self-supervised learning, because the correct answer is already in the text.
2. Instruction tuning. A pretrained model continues text, but it does not reliably follow requests. IBM describes instruction tuning as training on a dataset where the inputs "consist entirely of tasks that resemble requests users might make in their prompts," paired with good answers. Google's course calls it "An optional further training step" that "can improve an LLM's ability to follow instructions."
3. Human feedback (RLHF). IBM describes reinforcement learning from human feedback (RLHF) as "a form of fine-tuning where humans rank model outputs and the model is trained to prefer outputs that humans rank higher." IBM says it is often used for alignment, which it defines as making outputs "useful, safe and consistent with human values," and for style, such as a more casual or brand-consistent voice.
A company can also fine-tune a general model for one field. IBM's example is a model fine-tuned "on a corpus of legal Q&As in order to create a chatbot for the legal field."
What is inside: the Transformer
Google's course calls the Transformer "the most successful and widely used architecture for building LLMs." Its central idea is self-attention. For each token, Google says, self-attention asks: "How much does each other token of input affect the interpretation of this token?"
Google's example sentence shows why this matters: "The animal didn't cross the street because it was too tired." The word "it" could point to the animal or the street. Self-attention lets the model weigh the other words and link "it" to "animal". If the last word changes to "wide", the course says self-attention "would hopefully rate street as more relevant than animal."
IBM adds two details. First, each token is turned into a list of numbers called an embedding. Second, a Transformer has many layers, and "at each, the embeddings are slightly adjusted, becoming richer contextual representations from layer to layer." Google's course says that while earlier layers "might focus on basic syntax, deeper layers can integrate that information to grasp more nuanced concepts like sentiment, context, and thematic links."
What happens when you ask a question
IBM describes inference, the step where a trained model answers a prompt, as a loop. Here are the steps in order:
- The model splits your prompt into tokens.
- It turns each token into an embedding.
- Its Transformer layers process those embeddings with self-attention.
- It calculates a probability for each possible next token.
- It picks a next token, adds it to the text, and repeats from step 3 until the answer is complete.
IBM stresses that "The model does not 'know' the final answer in advance." It predicts one token at a time, "making its best guess at every step."
This is also why the same question can get different answers. OpenAI's article explains that "there is an inherent element of randomness in how the model responds," because a sentence can be completed in several plausible ways. IBM names two settings that control this: temperature, "which controls the randomness of text," and top-k or top-p sampling, which "limits the set of tokens considered to the most likely ones."
The context window
An LLM can only consider a limited amount of text at once. IBM defines the context window as "the maximum number of tokens that a model can 'see' and use at once when generating text." Our explainer on what a context window is covers what happens when a conversation runs past it.
The context window is also how a model can use information it was not trained on. IBM describes retrieval augmented generation (RAG) as connecting a model to outside knowledge, where "The retrieved information is passed into the model's context window." IBM's example is an LLM connected to a weather service database, so it can retrieve that day's weather report for a user.
Does an LLM copy its training data?
OpenAI says its models "do not store or retain copies of the data they are trained on." In its description, training adjusts the parameter values "slightly to reflect patterns it has identified," and when answering, "the model uses these learned weights to predict and create new content." That is OpenAI's account of its own models. It also notes that responses "may sometimes include personal information about individuals whose personal information appears multiple times on the public internet," such as public figures.
What LLMs are used for
IBM lists these common uses:
- Text generation, such as drafting emails or blog posts.
- Summarizing long articles, reports and documents.
- AI assistants and chatbots that answer questions.
- Code generation, finding errors and translating between programming languages.
- Sentiment analysis of customer feedback.
- Language translation.
- Reasoning tasks, such as math problems and multi-step plans.
OpenAI's article lists similar tasks for ChatGPT, including "organizing and summarizing information, assisting with translations, supporting coding, research and analysis."
Where LLMs go wrong
Both Google and IBM list the same main problems.
- Hallucinations. Google's course says "LLMs hallucinate, meaning their predictions often contain mistakes." IBM describes a hallucination as output that "is false or misleading while sounding plausible."
- Bias. Google says "Like all ML models, LLMs can exhibit all sorts of bias." IBM says LLMs "can also reflect and amplify biases present in their training data."
- Computing cost. Google says LLMs "consume enormous amounts of computational resources and electricity." IBM raises "both cost and environmental concerns."
Google's course also adds a caution about apparent reasoning. It says an LLM trained on many math word problems "can give the appearance of doing sophisticated mathematical reasoning," while "basically just autocompleting a word problem prompt."
Some newer models are trained to work through a problem in steps before answering. IBM calls these reasoning models, "LLMs that have been fine-tuned to break complex problems into smaller steps." Our guide to what a reasoning model is explains how they differ.
LLM, chatbot and AI agent: the difference
The terms overlap, but they are not the same thing.
- The LLM is the trained model. OpenAI's article refers to "the models that power ChatGPT."
- The chatbot is the product built around one or more models. OpenAI describes ChatGPT as "an artificial intelligence-based service that you can access via the internet or app."
- An AI agent connects a model to tools. IBM says that by themselves, "LLMs simply generate text based on context," but they "can be integrated with memory, APIs, decision logic and other external systems to perform specific tasks."
Developers can also reach many models directly. IBM notes that many models "are accessible to all developers through APIs," and that "many open-source models can be deployed locally or in the cloud."
How to get better answers from an LLM
These follow from how the models work, as the sources above describe them.
- Give context in the prompt. The model works from what is in its context window. IBM describes prompt engineering as "The easiest and fastest way to get domain-specific knowledge from a general-purpose LLM," with no extra training needed.
- Paste or attach the source text. When the answer must match a document, put the document in the conversation, which is the same idea as RAG.
- Check facts that matter. Fluent text can still be a hallucination. Verify names, figures and quotes against a primary source.
- Ask again or rephrase. Because generation has an element of randomness, a second try can produce a different answer.
- Do not treat it as an expert. IBM's own example notes that LLMs "are not recommended to be used for medical advice."
