Subscribe
educational
5 min read

Why Asking an AI to Double-Check Itself Often Makes It Worse

Every chat interface encourages you to tell the model to re-examine its answer. On reasoning tasks, that request can cost accuracy. Here is what the research says about self-correction, and the narrow conditions under which it works.

A machine reviews its own punched tape through a loupe and misses the obvious tear
A machine reviews its own punched tape through a loupe and misses the obvious tear

The most common reflex when a language model gives you a bad answer is to ask for a second opinion from the same model. "Are you sure?" "Double-check your reasoning." "Look at your answer again and fix any mistakes." The technique has a name in the research literature: self-correction, and in its purest form, intrinsic self-correction, where the model critiques and revises its own output with no outside feedback.

It sounds reasonable. A human accountant who reviews their own work usually finds something. But a line of papers from 2023 to 2026 says that when a model reviews its own reasoning, the review is the weakest link. Asking twice is often worse than asking once.

The result that set the frame

The canonical study is Large Language Models Cannot Self-Correct Reasoning Yet by Jie Huang, Xinyun Chen, Denny Zhou and colleagues at Google DeepMind and the University of Illinois, published at ICLR 2024. Earlier papers, notably Self-Refine and Reflexion, had reported that models improve by critiquing their own outputs. The DeepMind team retested the claim under deployment conditions, where nothing tells the model whether its first answer was right.

The numbers are blunt. On the GSM8K math benchmark, GPT-4's accuracy fell from 95.5% with standard prompting to 91.5% after one round of self-correction and 89.0% after two. GPT-3.5 dropped from 75.9% to 74.7%. On CommonSenseQA, GPT-3.5 collapsed from 75.8% to 38.1% after a single round. On GSM8K, 74.7% of the time GPT-3.5 kept its initial answer; when it did change, it was more likely to flip a correct answer to a wrong one (8.8%) than to fix a wrong answer (7.6%).

The same paper shows the other half of the picture. When researchers told the model which answers were wrong, an oracle label, self-correction worked well: GPT-3.5 rose from 75.9% to 84.3% on GSM8K, GPT-4 from 95.5% to 97.5%. But in production nobody has those labels. That is the whole problem.

Two skills, one missing

A useful decomposition comes from a Google Research study and its paper LLMs cannot find reasoning errors, but can correct them given the error location. Self-correction is actually two tasks: mistake finding and output correction. The team built BIG-Bench Mistake, a dataset of model-generated reasoning traces with human-annotated locations of the first logical error, with inter-rater agreement above 0.98.

The finding: state-of-the-art models are poor at finding logical errors, even unambiguous ones, and even on traces they generated themselves. But give the model the error location and its correction ability is solid. Feeding in known mistake locations boosted task accuracy across all five reasoning tasks tested. The bottleneck is diagnosis, not treatment.

A machine feeds its own printed tape back into itself while a hand offers an external correction strip

A 2024 critical survey of the field, Kamoi et al., reached the same verdict from the other direction: no prior work demonstrated successful self-correction from prompted self-feedback alone, except on tasks exceptionally suited to it, such as style rewriting. Self-correction works when reliable external feedback exists, for example a compiler or unit test, and works when models are specifically fine-tuned for it.

Why asking twice still feels like it helps

Two effects explain the gap between the lab result and the lived experience.

First, style is not reasoning. Self-Refine genuinely improved outputs by roughly 20% on average across seven tasks like dialog response generation and review rewriting. Editing your own prose is easy to verify against taste. Checking a ten-step arithmetic chain is not.

Second, the failure is partly an activation problem, not a knowledge problem. A 2025 paper, Self-Correction Bench (COLM 2026), injected the identical error into context either as a user-attributed mistake or a model-attributed one. Across 14 open-source non-reasoning models, the average blind spot rate was 64.5%: models corrected the error when the user "made" it but failed on their own identical output. The capability exists but is not switched on. Two switches work: fine-tuning on just 5,306 error-correction traces reduced the blind spot 76%, and simply appending the word "Wait" to the prompt reduced it 89.3% with no training at all.

The follow-up literature adds nuance without overturning the frame. A 2025 perturbation study found open-weight models can sometimes recover from corrupted reasoning chains in a single utterance, stronger than the 2023 results suggested. CorrectBench, a NeurIPS 2025 benchmark of the whole space, found self-correction methods can improve accuracy on complex reasoning when feedback is reliable, but that built-in reasoning models such as DeepSeek-V3 gain little from bolting additional correction loops on top. In other words: models that already think before answering have partially internalized the loop, and an extra prompted critic round mostly adds cost.

What this means in practice

If you run agents or use frontier models for reasoning work, the research supports a short list of rules.

  • Do not treat "review your answer" as a verification step. Without an external signal, the critic shares the reason the generator erred, and prompt-biased self-judgment is unreliable.
  • Buy correction where feedback is checkable: tests, compilers, retrieval, calculators, a second independent model, a human. Reflexion's 91% pass@1 on HumanEval, versus an 80% GPT-4 baseline, came from looping on real unit-test feedback, not from introspection.
  • Cheap prompts can help more than loops: the "Wait" prefix is nearly free and measurably activates dormant correction behavior, part of the same test-time nudge family as budget forcing in s1.
  • Expect double-checking to cost accuracy, not just latency, on math-like tasks. Two rounds of self-correction on GSM8K were 6.5 points worse than one clean answer for GPT-4.

The uncomfortable generalization: a model cannot be its own auditor, because auditing is a harder skill than the work being audited. The industry's fix is the same one the papers point at, and it is unglamorous. Put a verifier outside the model.

Sources

educational