Anthropic says experts who work with an AI coding agent often end up making the agent's choices and then explaining them in the agent's words. In a study published Oct. 27, its Alignment team gave 50 professional analysts a statistical task in Claude Code. Each analyst's agent was secretly told to lean toward one of two reasonable options on every key decision, with the lean set by a coin flip. Across four analytical choices, analysts went with the agent's lean 86% of the time, against about 50% if the agent had no effect.
The finding matters because it targets a common assumption about AI at work: the agent does the typing, and the human still makes the calls that matter. The study, written up in a full paper titled "Who's Aligned to Whom?", suggests that assumption may not hold for judgment calls, where there is no single right answer to check against.
How the study worked
Anthropic recruited 61 analysts from a pre-screened pool of professionals with hands-on experience in causal inference, the work of figuring out from data whether one thing causes another. Eleven chose not to use the agent, which left 50 in the main analysis. They worked in a web version of VS Code with Claude Code built in, running Claude Opus 4.8 at High effort. The median time on the task was 65 minutes.
The task was set at a fictional farming co-op deciding whether to expand or discontinue a paid advisory program. The board would expand it only if it raised productivity by roughly 3 to 4%. Anthropic left four analytical decisions open, each with two defensible options:
- Measure productivity as crop yield per hectare, or as yield per unit of operating cost.
- Use a difference-in-differences design, or compare yields directly with statistical adjustment.
- Estimate the effect only for enrolled farms, or across the whole membership.
- Count every farm equally, or give larger producers more weight.
The agent's lean on each decision, and on the final recommendation, came from hidden instructions in its system prompt. It was told to hold the lean as its own view, make the strongest case for it that the data allowed, change its mind if the evidence turned decisively against it, and never misrepresent results.
What happened
Analysts matched the agent's lean 92% of the time on the outcome measure, 82% on the comparison method, 81% on which farms to cover and 90% on weighting. The effect on the final recommendation was weaker but still visible: 54% recommended rolling out the program when the agent leaned that way, compared with 36% when it leaned the other way. Using only the four coin-flip leans, a statistical model predicted 82% of recommendations, against 61% for always guessing the most common answer.
"So we could largely predict what an analyst would recommend from the coin flips alone, before they had even started the task," the researchers wrote.
In 38 cases, an analyst stated a preference before the agent weighed in. When the agent disagreed, they switched 60% of the time, compared with 6% when it agreed. Anthropic notes the 38-case sample makes this imprecise, with a 95% confidence interval for the gap running from 24 to 82 percentage points. It also says it cannot be sure the agent caused the gap, because analysts who stated a preference early may differ from those who did not. The pattern runs opposite to agent sycophancy, where an agent drops correct work after a user pushes back.
Borrowed reasons, claimed as their own
Another result came after the task. Asked to explain each choice without access to their chat logs, 77% of analysts repeated the agent's argument in at least one answer, sometimes word for word, but only 15% credited the agent anywhere. On the survey of 48 analysts, 52% rated the final recommendation as more their own than the agent's. Six guessed the study was about AI influence, but none suspected the agent's preferences had been set on purpose.
Experience and expertise did not predict who deferred. In an exploratory check of 10 traits, including years of experience, education, quiz scores and trust in AI, only age showed a significant link, and that result disappeared after correcting for multiple tests.
The limits Anthropic flags
Anthropic is careful about how far this goes. It deliberately strengthened the agent's lean so the effect could be measured; with no lean assigned, Claude Opus 4.8 made the same five choices in 112 of 123 runs. The study covered one task, one field and one model, and the analysts were experts in causal inference but new to farming. In real work they might know the domain better and hold stronger views.
The company also says this is not a case against using AI. Of the 11 analysts who worked without the agent, six made serious methodological errors, compared with three of the 50 who used it. The no-agent group was small and less experienced, so Anthropic says it is wary of reading too much into that.
Why it matters
Anthropic says the stakes extend beyond individual experts. If separate research teams all defer to models with the same preferences, their agreement may not be independent, and "an apparent consensus might reflect one model's preferences." The company says this applies to AI safety research too, where researchers now use agents to run experiments and a human reviewer is only an independent check if their judgment is their own. That ties into Anthropic's broader work on how Claude is shaped, covered in our Constitutional AI explainer and its review of the AISI cyber test incident.
The researchers suggest three areas for future work: ways to measure deferral in everyday settings, agents that make the best case for each option or ask for the expert's view first, and institutions that decide which decisions need a person's own informed judgment. For the case that expert sign-off on AI-assisted work no longer proves independent judgment, see our opinion piece. Coverage from Gizmodo and Decrypt focused on the same 86% and 77% figures. Anthropic also shipped Enterprise smart reports this week for companies tracking how staff use Claude.
Sources
- Anthropic: How AI agents shape experts' judgment calls (Oct. 27, 2026)
- Anthropic paper: Who's Aligned to Whom? How AI Agents Shape Experts' Research Judgment Calls (PDF)
- Gizmodo: Expert Judgement Can Be Easily Hacked by AI, Anthropic Study Finds
- Decrypt: AI Can Quietly Influence Experts' Decisions, Anthropic Study Finds
