AI Hallucinations Explained: Why Chatbots Make Things Up and How to Catch Them

Ask a chatbot for an obscure fact and it may hand you a confident, specific, well-formatted answer that is simply wrong. A court case that does not exist, a wrong birthday, a dissertation title nobody wrote. The AI field calls this a hallucination, and OpenAI describes it as a fundamental challenge for all large language models (LLMs).

This explainer covers what counts as a hallucination, the main reasons models produce them, what researchers have seen inside a model when it happens, how hallucination rates are measured, and what you can do to catch them.

What counts as an AI hallucination

OpenAI's research post on the topic defines hallucinations as "plausible but false statements generated by language models." Its example is a good one: asked for the title of the PhD dissertation of Adam Tauman Kalai, one of the paper's authors, a widely used chatbot gave three different answers, and none was correct. Asked for his birthday, it gave three different dates, all wrong.

Researchers split the problem into types. A survey by Huang et al., published in ACM Transactions on Information Systems, uses two main groups:

• Factuality hallucinations. The answer conflicts with real-world facts or cannot be verified at all. The survey's examples include crediting Thomas Edison with the telephone, and inventing a "Parisian tiger" that went extinct when the Eiffel Tower was built.
• Faithfulness hallucinations. The answer drifts from what you gave the model. It ignores your instruction (answers a question you asked it to translate), contradicts a document you supplied, or contradicts its own earlier reasoning steps.

The distinction matters in practice. If you paste a contract into a chatbot and ask for a summary, a faithfulness error is the risk. If you ask it a trivia question with no document attached, a factuality error is the risk.

A team at Oxford, writing in Nature in June 2024, narrowed things further. They used the word "confabulations" for answers that are both wrong and arbitrary, meaning the answer changes with irrelevant details such as the random seed. Their example: asked "What is the target of Sotorasib?", a model sometimes answered correctly and sometimes gave a different, wrong answer, with identical instructions. They separate this from a model being consistently wrong because its training data contained a common misconception, and from a model that "lies" in pursuit of a reward. The symptoms look alike, but the causes differ, and so do the fixes.

Why chatbots make things up

There is no single cause. The research points to several that stack on top of each other.

1. Next-word prediction cannot learn arbitrary facts

An LLM is first trained to predict the next word across a huge amount of text. OpenAI's post points out that this text carries no true or false labels. The model sees fluent language and has to approximate it.

Some patterns are easy to learn this way. Spelling and matching parentheses follow consistent rules, and OpenAI notes that errors of that kind disappear with scale. Other facts follow no pattern at all. OpenAI's analogy is labeling pet photos by the pet's birthday: since birthdays are essentially random, no algorithm can predict them from the photo. In the same way, arbitrary low-frequency facts, like a pet's birthday, "cannot be predicted from patterns alone and hence lead to hallucinations."

The Huang survey makes a related point about what it calls knowledge boundaries. Models do worse on long-tail facts that appear rarely in their training data, and their knowledge stops at a training cutoff. Asked about something past that cutoff, a model may invent an answer or give one that used to be true.

2. Tests reward guessing over saying "I don't know"

This is the central argument of OpenAI's September 2025 paper. Most evaluations grade models on accuracy alone: the share of questions answered exactly right. Under that scoring, a guess can only help. OpenAI's example: if a model does not know someone's birthday and guesses "September 10," it has a 1-in-365 chance of being right, while "I don't know" scores zero. Over thousands of questions, the guessing model looks better on the scoreboard.

OpenAI backs this with SimpleQA results from the GPT-5 system card. The older o4-mini model scored slightly higher on accuracy, but it almost never declined to answer, and its error rate was nearly three times higher:

• gpt-5-thinking-mini: 52% abstention, 22% accuracy, 26% error.
• OpenAI o4-mini: 1% abstention, 24% accuracy, 75% error.

In OpenAI's words, "Strategically guessing when uncertain improves accuracy but increases errors and hallucinations." Its proposed fix is to penalize confident errors more than abstentions, and to give partial credit for appropriate expressions of uncertainty, in the main benchmarks rather than in a few side tests. The post argues that as long as accuracy-only scoreboards dominate leaderboards and model cards, developers have a reason to build models that guess.

3. Fine-tuning and human feedback can push the wrong way

After pretraining, models are tuned on example conversations and then on human preference ratings. The Huang survey describes two ways this can add hallucinations.

First, supervised fine-tuning typically trains the model to complete every response. If the training answers go beyond what the model actually knows, it learns to produce confident text past its own knowledge, and it never learns to express uncertainty. The survey cites research (Gekhman et al., 2024) linking the learning of new facts during fine-tuning to more hallucinations.

Second, reinforcement learning from human feedback (RLHF) can teach sycophancy: telling people what they seem to want to hear. The survey cites work by Sharma et al. suggesting that both human raters and the preference models trained on their ratings sometimes favor sycophantic answers over truthful ones.

4. Decoding settings and snowballing

When a model writes, it samples each next word from a probability distribution. The survey notes that a higher sampling temperature flattens that distribution and makes rare words more likely, which raises the risk of hallucinations. It also describes a snowball effect: once a model writes one wrong token, later text is conditioned on it, and the error can cascade through the rest of the answer.

What happens inside the model

Anthropic's March 2025 interpretability research, which looked inside its Claude 3.5 Haiku model, adds a mechanical picture. Its summary post reports a "counter-intuitive" finding: Claude's default is to decline to answer.

Here is the mechanism as Anthropic describes it:

• A circuit that is "on" by default causes the model to say it has insufficient information to answer.
• When the question is about something the model knows well, such as the basketball player Michael Jordan, a competing feature representing "known entities" activates and inhibits that default, so the model answers.
• Asked about an unknown name ("Michael Batkin"), the default stays on and the model declines.
• When the researchers artificially switched on the "known answer" features, the model hallucinated, quite consistently, that Michael Batkin plays chess.

Anthropic says the same misfire can happen naturally when Claude recognizes a name but does not know anything else about the person. In such cases, the "known entity" feature might still activate and suppress the "don't know" default, and the model then confabulates a plausible but untrue answer.

How hallucination rates are measured

There is no single hallucination score, because each test measures a different type.

• Short factual questions. Benchmarks like SimpleQA ask questions with one correct answer. As the OpenAI table above shows, the useful numbers are the error rate and the abstention rate, not accuracy alone.
• Summarizing a document. Vectara's hallucination leaderboard uses its Hallucination Evaluation Model (HHEM) to check how often a model introduces hallucinations when summarizing short documents. That is a faithfulness test. In its October 16, 2026 update, the lowest listed rate was 1.8%, for Ant Group's finix_s1_32b, and the first 32 rows ran up to 7.0%. Model tiers do not line up neatly: OpenAI's gpt-5.4-nano is listed at 3.1%, while gpt-5.4 is listed at 7.0%.
• Real-world tasks. A preregistered Stanford and Yale study of AI legal research tools used over 200 legal queries and counted a hallucination as either a false statement or a false claim that a source supports a statement. It found that the tools from LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) "each hallucinate between 17% and 33% of the time," less often than GPT-4 but well above zero.

The paper notes that, alongside a later version of its tool, LexisNexis said "our promise is not perfection, but that all linked legal citations are hallucination-free." The study's point is that a citation can be real and still not support the claim it is attached to.

Catching hallucinations automatically

The Oxford Nature paper offers one detection method that needs no labeled examples, called semantic entropy. It works like this:

1. Ask the model the same question several times (the paper used ten generations).
2. Group the answers by meaning, using a check for whether each pair of answers implies the other. "Paris" and "It's Paris" land in the same group.
3. Measure how spread out the groups are. If the answers mean many different things, the model is probably confabulating.

Averaged across 30 combinations of tasks and models, semantic entropy reached an AUROC of 0.790, against 0.691 for naive entropy, where 1.0 is a perfect detector and 0.5 is no better than chance. The authors are clear about the limit: the method catches arbitrary errors, not cases where a model is confidently and consistently wrong.

Does RAG fix hallucinations?

Retrieval-augmented generation, or RAG, gives the model relevant documents at answer time instead of relying on memory. We covered how it works in our plain-English guide to RAG. It helps most with knowledge gaps: facts the model never learned or that changed after training.

It does not make a system hallucination-free. The Stanford study tested RAG-based legal tools precisely because vendors had described RAG as eliminating or avoiding hallucinations, and it still found the 17% to 33% range above. The paper traces failures to retrieval that pulls the wrong documents, sources that do not apply (overruled cases, the wrong jurisdiction), and reasoning errors even when the retrieved documents looked relevant. The Huang survey makes the same general point: irrelevant retrieved text can add noise and lower answer quality.

So RAG moves the problem from "does the model remember this?" to "did it find the right source, and did it read it correctly?" Both still need checking.

How to reduce hallucinations

If you use chatbots

• Treat names, numbers, dates, quotes, and citations as unverified until you check them. These are the arbitrary, low-frequency facts that OpenAI's analysis says pattern-matching cannot predict.
• Give the model permission to say it does not know. Anthropic's guide to reducing hallucinations says explicitly allowing Claude to admit uncertainty "can drastically reduce false information."
• Ask for sources, then open them. A real link is not proof. As the legal study showed, the cited source may not say what the answer claims.
• Ask the same question twice. Anthropic's guide recommends comparing outputs from repeated runs, since inconsistencies can indicate hallucinations. This is a manual version of the semantic entropy idea.
• Paste the document when you can. Asking a model to work from text you supply turns a memory test into a reading test, and you can check the answer against the text.

If you build with LLMs

Anthropic's guide, written for Claude, lists these techniques:

• For long documents (over 20,000 tokens), have the model extract word-for-word quotes first, then base its answer only on those quotes.
• Have the model cite a supporting quote for each claim, and retract any claim it cannot support.
• Instruct the model to use only the provided documents, not its general knowledge.
• Run the same prompt several times and flag answers that disagree.

The guide closes with the caveat that these techniques "significantly reduce hallucinations" but "don't eliminate them entirely," and it recommends validating critical information. If you evaluate models yourself, OpenAI's argument applies to your own tests too: score wrong answers worse than abstentions, or your evaluation will favor the model that guesses.

For a look at an architecture change aimed at the same problem, see our explainer on the Differential Transformer and its hallucination suppression claims. For how hallucination claims play out at a launch, see what people said about Gemini 4 Argon.

Will AI hallucinations go away?

The research gives a split answer. OpenAI's post lists several claims it considers misconceptions:

• Accuracy will never reach 100%, because some real-world questions are inherently unanswerable, whatever the model's size, search, or reasoning ability.
• Hallucinations are still not inevitable, because a model can abstain when it is uncertain.
• Knowing your limits does not require a huge model. OpenAI's example: a small model that knows no Māori can simply say "I don't know," while a model that knows some Māori has to judge its confidence. Being calibrated, the post says, requires much less computation than being accurate.

OpenAI also says GPT-5 "has significantly fewer hallucinations especially when reasoning," while noting that they still occur. OpenAI's proposal is for models that abstain when unsure, and for benchmarks that count confident errors rather than accuracy alone. Until that is standard, the practical rule stays the same: the more a fact matters, the more it needs a second source.

Sources

• OpenAI: Why language models hallucinate (September 5, 2025)
• Anthropic: Tracing the thoughts of a large language model (March 27, 2025)
• Farquhar et al., Detecting hallucinations in large language models using semantic entropy, Nature (June 19, 2024)
• Huang et al., A Survey on Hallucination in Large Language Models (arXiv 2311.05232)
• Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (arXiv 2405.20362)
• Vectara hallucination leaderboard (GitHub, updated October 16, 2026)
• Anthropic docs: Reduce hallucinations