Subscribe
Education
6 min read

What Is Overthinking? Why Reasoning Models Spend Thousands of Tokens on Easy Questions

Reasoning models often spend thousands of extra tokens on simple questions like '2+3' due to a training prior that thoroughness wins, leading to the 'overthinking' failure mode identified by researchers in 2024.

Diagram illustrating the inverted-U relationship between thought length and model accuracy in large language models
Diagram illustrating the inverted-U relationship between thought length and model accuracy in large language models

Overthinking in reasoning models is a behavior where systems spend significantly more computing resources than necessary for simple tasks, such as answering "what is 2 plus 3?" This inefficiency arises because models are trained with reinforcement learning signals that reward correct answers without penalizing excessive length, causing them to carry a prior that thoroughness always wins. Consequently, these models may consume thousands of additional tokens, increase latency, and even reduce accuracy by revisiting and discarding correct initial thoughts.

What Is Overthinking? Why Reasoning Models Spend Thousands of Tokens on Easy Questions

Ask a reasoning model "what is 2 plus 3?" and watch what happens. A conventional chatbot answers "5" in a handful of tokens. A model from the o1 lineage, per the paper that named the problem, produced 13 separate solution attempts before settling, and consumed, on average, 1,953 percent more tokens than a non-reasoning model to get to the same answer.

The researchers gave the behavior a name in December 2024: overthinking. The paper, "Do NOT Think That Much for 2+3=?" (Chen et al., later accepted at ICML 2025), was the first systematic study of the failure mode, and it turned out to be a durable research area. Overthinking is now the practical counterweight to the "more thinking is better" pitch that came with reasoning models, and it explains why every major AI lab has spent the past two years shipping dials, routers, and budgets for how much a model is allowed to think.

The definition

An overthinking model spends more reasoning compute than the problem warrants. The extra thinking does not improve the answer. It costs money, adds latency, and can make accuracy worse by giving the model room to walk back a correct answer and replace it with a wrong one.

The clean mental model comes from a McGill NLP analysis of DeepSeek-R1, published as "DeepSeek-R1 Thoughtology" (also accepted at TMLR 2025). Its finding: accuracy against thought length follows an inverted U. On medium-difficulty problems, more thinking helps up to a point, performance peaks, then declines considerably for extremely long thoughts. The authors call the peak the model's "sweet spot" of reasoning. Trivial problems sit far to the left of that peak. Hard problems sit to the right. The model, left to itself, does not know which problem you handed it.

How the labs measured it

Three published results sketch the scale of the problem.

First, the original overthinking study. Across mathematical benchmarks, the authors found that extended chain-of-thought contributed minimally to accuracy on simple problems while multiplying output length by an order of magnitude. They proposed efficiency metrics that score a model on whether its token usage matches the difficulty of the task, not just on whether the answer is right.

Second, the Thoughtology analysis of DeepSeek-R1, the first widely available open model that exposed its thinking trace to users. When prompted to answer within a specific token budget, the model largely ignored the instruction: it thought for about 8,000 tokens regardless of the budget specified, and raising the requested budget barely changed average length. The paper also documented rumination: the model repeatedly returned to problem formulations it had already tried and rejected, burning tokens on circular self-talk instead of new approaches.

Third, a benchmark study presented at ACL 2026 (the LLM-THINKBENCH evaluation of 53 models on basic math) put numbers on the cost-benefit inversion. Reasoning models averaged about 6,780 tokens per answer against 378 for standard models, while performing worse on the same simple questions. Models trained for long chains generated roughly 18 times more tokens while losing accuracy and collapsing under token constraints: one measured suite dropped from 72 percent accuracy to 44 percent once responses were capped at 1,024 tokens. Notably, the study found quantized models retained their reasoning ability, which points to the behavior being baked in by training rather than forced by hardware limits.

The inverted U: accuracy rises with thinking length, peaks at a sweet spot, then falls

Why models learn to overthink

Reasoning models are trained with reinforcement learning on verifiable answers: the reward signal says whether the final answer is correct and whether it is formatted properly. It mostly does not say anything about length. In the training data, long thinking chains correlate with hard problems that the model only solves by thinking long. The model absorbs a prior that thoroughness wins, and it carries that prior into problems that need none of it.

There is a second, subtler mechanism. Once a model starts a long chain, the chain itself creates opportunities to find things to reconsider. Self-doubt inserted as a virtue during training becomes a token sink at inference: "Wait, let me double-check" is cheap to emit and, on trivial problems, frequently re-derives nothing. The s1 paper (Muennighoff et al., 2025) demonstrated the effect in reverse. Its "budget forcing" technique simply cuts thinking off at a fixed token limit and appends an answer prompt, or injects the word "Wait" to extend thinking when the budget is not yet spent. The injected "Wait" sometimes triggers genuine self-correction, which is exactly why unconstrained models keep doing it to themselves.

What the industry did about it

The response has come in three layers.

Three control dials: the industry's answer is effort knobs, not smaller models

Dials. Every major provider now exposes a reasoning-effort control. OpenAI's API takes a reasoning_effort level. Anthropic's extended thinking takes a budget_tokens cap, the most precise of the three because it is an actual token count. Gemini moved from a token-count thinkingBudget in its 2.5 era to named thinking levels in 3.0. One caveat the documentation makes hard to overstate: a "medium" on one provider is not a "medium" on another. The levels are not a portable unit of compute.

Routers. OpenAI's GPT-5 launch in August 2025 made routing the headline architecture: a fast model for most requests, a deeper thinking variant for hard ones, and a real-time router that decides per message. OpenAI claimed GPT-5 with thinking would beat its o3 predecessor using 50 to 80 percent fewer output tokens, an explicit efficiency pitch. The launch also showed how fragile the split is: on day one the autoswitcher broke, misrouting queries in ways that made answers appear weaker, and CEO Sam Altman publicly acknowledged that "GPT-5 seemed way dumber" until it was fixed. Routing is overthinking control at product scale, and it fails visibly.

Prompting and decoding tricks. Chain of Draft (Xu et al., 2025) asks models to write reasoning as short draft steps, capped at roughly five words each, instead of full prose. It matched or beat chain-of-thought accuracy on math, symbolic, and commonsense benchmarks while using as little as 7.6 percent of the tokens, with latency cuts of up to 76 percent on some tasks. Budget forcing, as in s1, reaches the same place from the decoding side.

What this means if you run models in production

The published evidence supports a short checklist.

  • Do not treat thinking as quality. On easy problems, an inverted-U study says extra thinking can subtract accuracy while multiplying cost.
  • Measure output length per difficulty band on your own traffic. The efficiency metrics in the original paper exist precisely because accuracy alone hides overthinking.
  • Test whether your model respects budgets. Thoughtology showed DeepSeek-R1 ignored prompt-stated token budgets; API-level caps work because they are enforced at decoding, not requested in prose.
  • Route before you budget. If a request needs no reasoning, the cheapest fix is not a smaller thinking budget. It is not invoking the thinking model at all.
  • Do not compare effort levels across providers. The dialects differ and none of them is a unit.

Overthinking is the standard cautionary tale of the test-time-compute era: a scaling axis that genuinely raised ceilings on hard problems, and a training recipe that could not tell the difference between a hard problem and an easy one.

Sources

Frequently asked questions

Why do reasoning models overthink easy questions?

Models learn to overthink because they are trained using reinforcement learning where the reward signal primarily verifies answer correctness and format, rather than output length. Additionally, long thinking chains correlate with hard problems in training data, creating a prior that thoroughness is required, which persists even during inference on trivial tasks.

How does overthinking affect model accuracy?

Overthinking can worsen accuracy due to an inverted-U relationship between thought length and performance. While medium-difficulty problems benefit from increased thinking up to a point, extremely long thoughts allow models to walk back correct answers and replace them with wrong ones, particularly on simple problems.

What are common methods to prevent overthinking in production?

Industry solutions include implementing reasoning-effort dials (like token budget caps), using routers to direct simple queries to faster non-reasoning models, and applying decoding tricks such as Chain of Draft or budget forcing. These methods ensure models do not invoke deep thinking capabilities when unnecessary.

Education