The Referee Needs a Referee: What Five Years of Research Says About LLM-as-a-Judge Bias
The automated grader has won. Human preference labels are expensive and slow, so teams from frontier labs to startup eval pipelines have converged on the same workaround: point a strong model at two candidate answers and ask it which is better. The technique works well enough to have displaced whole generations of static benchmarks. It also leaks in four documented directions, and the research literature on those leaks is now specific enough to be useful.
This is what the papers say, and which of the proposed fixes actually hold up.
The bargain that made judges dominant
The founding document is still Zheng et al.'s MT-Bench paper (2023), which tested GPT-4 as a judge against two datasets: a controlled multi-turn question set (MT-Bench) and a crowdsourced battle platform (Chatbot Arena). Its headline finding made the technique a standard: strong LLM judges matched human preferences at over 80% agreement, which is roughly the same rate at which humans agree with each other. A judge that is as noisy as a human rater, at a fraction of the cost, is a good deal.
The same paper also named the failure modes that every subsequent study has measured: position bias (favoring whichever answer appears first), verbosity bias (favoring longer answers), self-enhancement bias (favoring the judge's own style of writing), and limited grading ability on math and complex reasoning. The literature since then is essentially a five-year audit of that list.
Position bias: the order of the menu decides the meal
The sharpest demonstration came from Wang et al. in 2023, and it is close to a reductio ad absurdum. Pairwise judging asks the model to compare Answer 1 against Answer 2. The authors swapped the two answers and re-ran. Rankings reversed. GPT-4 awarded Vicuna-13B a 51.3% win rate when Vicuna's answer sat in the first slot, and 23.8% for the identical answer set when it moved to the second. With ChatGPT as the evaluator, they could make Vicuna-13B "beat" ChatGPT on 66 of 80 tested queries by choosing which slot each answer occupied.
Two conclusions matter for practitioners. First, the bias is systematic, not random noise: a follow-up study presented at IJCNLP-AACL 2025 ran roughly 150,000 evaluation instances across 15 judges and confirmed that position preference tracks the quality gap and the task, not chance. Second, the direction of the bias is model-specific: the same GPT-4 that anchors on position one in some settings behaves differently in others. You cannot hard-code a correction; you have to measure your own judge.
Verbosity bias: length as a proxy for quality
MT-Bench flagged that judges reward longer responses regardless of content. What made this actionable was Dubois et al.'s length-controlled AlpacaEval (COLM 2024). The trick is statistical rather than prompt-based: fit a simple regression predicting the judge's preference from the length difference between the two answers, then read off what the preference would be if lengths were equal. Controlling length this way raised the metric's Spearman correlation with Chatbot Arena from 0.94 to 0.98, and made leaderboard positions robust to a team padding its outputs with filler.
The lesson generalizes beyond AlpacaEval: when a bias is a known, measurable confounder, post-hoc regression control is cheap and works, without touching the judge at all.

Self-preference: the judge recognizes its own handwriting
The most uncomfortable finding is that LLM evaluators favor text they generated themselves, and appear to do so because they can recognize it. Panickssery, Bowman and Feng (NeurIPS 2024) showed that GPT-4 and Llama 2 have non-trivial out-of-the-box accuracy at distinguishing their own outputs from those of other models and humans, and that experimentally tuning self-recognition up or down moves self-preference in lockstep. The correlation held up under controlled experiments, which makes it hard to dismiss as a confounder. A judge scoring a model from its own family is not a neutral party.
This has a structural consequence for leaderboards and reward models: if the same model family writes the answer, grades the answer, and generates the training signal, the loop has no outside check. Any team using a GPT-4-class judge to rank GPT-family candidates against Claude-family candidates is measuring something other than quality.
JudgeBench: competent stylist, weak fact-checker
The reality check arrived with JudgeBench (ICLR 2025). Earlier agreement results used preference labels; JudgeBench built hard response pairs from math, coding and reasoning datasets where the correct answer is objectively known. On those items, the paper reports, several strong judges including GPT-4o performed barely better than random guessing, and fine-tuned open judges scored worse still. The gap between "matches human preferences on chat quality" and "identifies the factually correct answer on a hard problem" turned out to be enormous.
Judges are reliable at what humans find easy (tone, structure, obvious errors) and unreliable at what humans find hard (verifying a proof, tracing a bug). Teams quietly assume the capability transfers. It does not.
What actually fixes it

The research suggests a short playbook, in descending order of cost-effectiveness:
- Swap and average. Run every pairwise comparison in both orders and aggregate. This is the Balanced Position Calibration idea from Wang et al.; their calibration framework (multiple evidence plus balanced position) raised ChatGPT-as-judge accuracy by 14.3 points and its Cohen's kappa agreement with humans from 0.06 to 0.31. Unstable across orders means the item is genuinely hard, which is itself signal.
- Control length in the score, not the prompt. Explicit rubric instructions only partially tame verbosity; regression control, as in LC-AlpacaEval, removes it arithmetically.
- Use a jury, not a judge. Cohere's PoLL paper replaced a single GPT-4 judge with a panel of three small models from different providers (Command-R, GPT-3.5-turbo, Claude 3 Haiku). The panel correlated better with human judgments across three judge settings and six datasets, showed less intra-model bias by construction, and cost seven to eight times less. Diversity of the panel, not the size of the models, is doing the work.
- Keep the judge out of the loop it grades. Never let a model family evaluate its own outputs on stakes-critical evals, and never use a judge's score both as a leaderboard metric and as a reward signal for the same policy it evaluates.
- Audit against ground truth where it exists. JudgeBench's core method (pairs with objectively labeled winners) is reusable internally: sample hard items from your own traffic, get them human-verified, and re-score them with your judge every time you change the prompt. Practitioner reviews like Eugene Yan's survey of LLM evaluator practice converge on the same point: an unvalidated judge is a random number generator with confident prose.
The honest summary
LLM-as-a-judge works: on conversational quality, frontier judges reproduce human consensus at human-level agreement. Every bias catalogued in 2023 has been replicated since with larger samples, and each has a cheap, measured mitigation. None of the mitigations change the underlying limitation. A judge that costs cents per call is buying you the reliability of a paid crowd worker who read the first answer twice, likes long memos, and shares a mother tongue with one of the candidates. That is a fine deal if you know what you bought. Most eval dashboards are configured by people who do not.
Sources
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, Zheng et al., 2023
- Large Language Models are not Fair Evaluators, Wang et al., 2023
- A Systematic Study of Position Bias in LLM-as-a-Judge, IJCNLP-AACL 2025
- Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators, Dubois et al., COLM 2024
- LLM Evaluators Recognize and Favor Their Own Generations, Panickssery, Bowman, Feng, NeurIPS 2024
- JudgeBench: A Benchmark for Evaluating LLM-based Judges, Tan et al., ICLR 2025
- Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models, Verga et al., 2024
- Evaluating the Effectiveness of LLM-Evaluators, Eugene Yan
