What Is a Reasoning Model? How AI Thinking Works

A reasoning model is an AI language model that works through a problem in a hidden scratchpad before it writes the answer you see. OpenAI's developer guide puts it this way: "Reasoning models use internal reasoning tokens before producing a response." The vendor says this "helps the model plan, use tools effectively, inspect alternatives, recover from ambiguity, and solve harder multi-step tasks" (OpenAI reasoning guide).

OpenAI, Google and Anthropic now all document this behavior for their models. Google calls it "thinking" in its Gemini thinking guide, and Anthropic uses the same word in its Claude thinking docs. This guide explains what that thinking is, what it costs, how to control it, and why the reasoning a model shows you is not always the reasoning it used.

If you are new to the terms, two earlier explainers help: what tokens are and what a context window is.

For developers, our technical guide to reasoning models in production covers thinking-token budgets, latency limits and handoffs between agents.

Reasoning model vs a standard chat model

A standard language model answers by predicting the reply one token at a time, starting right away. A reasoning model first produces extra tokens that work through the task, then writes the reply.

The practical differences:

• Order of work: a standard model starts the answer at once. A reasoning model can plan, compare options and check its work first.
• Visibility: the extra tokens are usually hidden. OpenAI says reasoning tokens "are not visible via the API" (OpenAI reasoning guide).
• Cost and time: the hidden tokens still count. They take time to generate and are billed (see the cost section below).
• Fit: OpenAI says reasoning models "work especially well for complex problem solving, coding, scientific reasoning, and multi-step agentic workflows."

The line between the two types has blurred. Anthropic says Claude's thinking "is adaptive: the model evaluates each request and determines whether to think and how much," and that "a simple factual question might get a direct response with no thinking block at all" (Claude docs). Google says Gemini models "engage in dynamic thinking by default, automatically adjusting the amount of reasoning effort based on the complexity of the request" (Gemini thinking guide). So one model can behave like a quick chat model on easy prompts and like a reasoning model on hard ones.

How the thinking works, step by step

Here is the sequence the three vendors' docs describe, in plain terms:

1. Your prompt arrives. It includes your question, any files and the conversation so far.
2. The model decides how much to think. Your effort or thinking level setting guides this, and the model adjusts to how hard the request looks.
3. It writes reasoning tokens you do not see. OpenAI says the model uses them to break down the prompt and consider "multiple approaches to generating a response."
4. It may call tools and think again. Anthropic says Claude "can think between tool calls, evaluating each tool result before determining what to do next." OpenAI's guide describes the same pattern as interleaved thinking.
5. It writes the visible answer. If you ask for it, some vendors also return a summary of the reasoning.
6. You pay for all of it. OpenAI says reasoning tokens "are billed as output tokens." Google says pricing is "the sum of output tokens and thinking tokens."

Where the idea came from

The method grew out of a prompting trick. A research paper, Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, showed that asking a model to write out "a series of intermediate reasoning steps" improved its results. The authors report that chain of thought prompting "improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks" in their experiments on three large language models.

Reasoning models build that step into the model itself. You no longer have to ask for step-by-step work; the model produces it on its own, usually out of sight. Our technical explainer on chain-of-thought reasoning in large language models covers why the extra steps help.

How to control how much a model thinks

Each vendor gives you a dial. The names differ, but the idea is the same: more thinking usually means better answers on hard tasks, at the cost of time and tokens.

• OpenAI: the reasoning.effort parameter. Supported values are "model-dependent" and can include none, minimal, low, medium, high, xhigh and max. OpenAI says "lower effort favors speed and lower token usage, while at higher effort the model thinks more completely" (OpenAI reasoning guide).
• Google: the thinking_level parameter, with levels such as minimal, low, medium and high depending on the model. Google's best practices suggest minimal or low thinking for "fact retrieval or classification," default thinking for moderate tasks, and maximum thinking for "advanced coding, math, or multi-step planning" (Gemini thinking guide).
• Anthropic: the effort parameter, which Anthropic calls "the primary steering lever for thinking." At low, Claude "minimizes thinking"; at max, Claude "thinks the most readily and at the greatest depth" (Claude docs).

In chat apps you may see the same dial as a mode or a model choice rather than a parameter. The trade-off is the same.

One point all three make in different words: the setting is guidance, not a guarantee. Anthropic writes that "no level guarantees a thinking block on every request." OpenAI says its models "reason adaptively across reasoning efforts, using fewer tokens for simpler tasks and thinking harder for complex tasks."

What reasoning costs you

Thinking is not free, even when you never see it. Three effects matter:

• Hidden tokens are billed. OpenAI bills reasoning tokens as output tokens. Google says "pricing is based on the full thought tokens the model needs to generate, despite only the summary being output from the API."
• They use up space. OpenAI says reasoning tokens "occupy space in the model's context window," and that a model may generate "anywhere from a few hundred to tens of thousands of reasoning tokens" depending on the problem.
• You can pay for an empty answer. If the model hits your output limit while it is still thinking, both OpenAI and Google say you can get an incomplete or empty response and still be charged for the reasoning. OpenAI recommends "reserving at least 25,000 tokens for reasoning and outputs" when you start experimenting. Google suggests lowering the thinking level instead of setting a small output limit.

The practical takeaway: when you compare the price of two models, the per-token rate is only half the story. A model that thinks at length on every request can cost more per task than one with a higher list price that thinks less. Check the usage numbers on real tasks. OpenAI reports reasoning tokens in the response's usage details, and Google reports them in a total_thought_tokens field.

Can you read the model's reasoning?

Usually not in full. What you get depends on the vendor:

• OpenAI: "we don't expose the raw reasoning tokens emitted by the model," but you can request a summary with the summary parameter (OpenAI reasoning guide).
• Google: "By default, only the final output is returned." You can turn on thought summaries with thinking_summaries. Google also returns an encrypted "signature" of the reasoning that must be sent back in later turns so the model can continue its line of thought; in stateful mode the server handles this for you (Gemini thinking guide).
• Anthropic: Claude returns thinking blocks when it thinks; a turn where it skips thinking "contains no thinking block" (Claude docs).

Summaries are useful for debugging. Google's own advice is to "analyze thought summaries to understand failures and improve prompts."

Is the visible reasoning what the model actually did?

Not always, and this is the most important limit to understand.

Anthropic's Alignment Science team tested whether reasoning models report the real basis for their answers (Reasoning models don't always say what they think). The researchers slipped a hint about the answer into a question, confirmed the model used it, then checked whether its written reasoning mentioned the hint.

What they report:

• Across hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time and DeepSeek R1 mentioned it 39% of the time.
• The unfaithful explanations "were substantially longer than the faithful ones," so a long explanation is not a sign of honesty.
• Faithfulness "was on average lower when the question being asked was more difficult."
• In a test where models were rewarded for exploiting a planted shortcut, they used it in over 99% of cases but admitted it in their reasoning "less than 2% of the time in most of the testing scenarios."

Anthropic lists limits of its own study: the scenarios were "somewhat contrived," used multiple-choice quizzes and covered only models from two companies. Still, the lesson for everyday users holds. Treat a model's displayed reasoning as a clue to how it got an answer, not as proof. Check the answer itself.

When a reasoning model is worth it

Use more thinking when:

• The task has several steps that depend on each other, such as a multi-part calculation or a plan.
• The task is coding or debugging beyond a one-line fix.
• The model has to use tools, read the results and decide what to do next.
• A wrong answer is expensive and you can afford to wait.

Use less thinking, or none, when:

• You need a quick lookup, a classification or a rewrite of short text.
• Speed matters more than depth, such as a live voice assistant. OpenAI lists voice, "fast information retrieval, and classification" as fits for its none setting.
• You run the same simple task thousands of times and cost adds up.

A good habit comes from Google's guide to its latest Gemini model: test your real tasks at each thinking level, then "pick the lowest level that meets your quality bar." The advice carries over to any model with an effort setting.

How to prompt a reasoning model

Reasoning models change some old prompting habits.

• State the goal, not every step. OpenAI says its reasoning models "usually work best when you give them a clear goal, strong constraints, and an explicit output contract without prescribing every intermediate step."
• Say what done looks like. For agent or research work, OpenAI advises you to "define what counts as done and how the model should verify its work."
• Do not lean on effort alone. OpenAI says to "treat reasoning.effort as a tuning knob, not the primary way to recover quality." A vague prompt at high effort is still a vague prompt.
• Use the dial before the prompt. Anthropic says that if you want Claude to think less often, you should "lower the effort level before reaching for prompt-based steering."
• Leave room. Set output limits high enough that thinking does not crowd out the answer.

Key points

• A reasoning model writes hidden reasoning tokens before its answer, which helps on multi-step problems.
• You control the amount of thinking with an effort or thinking level setting; the model also adapts on its own.
• Hidden reasoning is billed as output and takes up context space, so compare models on cost per task, not only on price per token.
• Vendors show summaries of the reasoning, not the full text.
• The reasoning a model shows may leave out what really drove its answer. Verify results.

Sources

• OpenAI: Reasoning models (developer guide)
• Google AI for Developers: Gemini thinking
• Google AI for Developers: Latest Gemini model guide
• Claude Platform Docs: Steering thinking
• Anthropic: Reasoning models don't always say what they think
• arXiv: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models