Computer-use agents have a presentation problem, and two numbers explain it. In June, the OSWorld 2.0 team reported that the best system it tested, Anthropic's Claude Opus 4.8, completed 20.6% of its tasks under a strict binary metric. The same model family scores around 83.5% on OSWorld-Verified, the benchmark the industry has cited for two years. Both numbers are real. They measure different things, and the gap between them is the true state of the market for AI that drives computers the way people do: screenshots in, mouse clicks out.
How we got here
The pixel-clicker era is four and a half years old:
- April 2024. The OSWorld paper lands: 369 tasks in live operating systems, judged by execution-based checks on end state. Humans complete 72.36%. The best model of the day manages 12.24%, failing mostly at GUI grounding, knowing where on the screen to click.
- October 2024. Anthropic ships computer use as a public beta in its API, letting developers point Claude at a virtual screen and have it move a cursor and type.
- January 2025. OpenAI launches Operator, a consumer browser agent powered by its Computer-Using Agent model, a vision model tuned with reinforcement learning for interface interaction.
- 2025. ByteDance open-sources UI-TARS, the first serious open-weight GUI agent family. On OSWorld it goes from 24.6% (72B, early 2025) to 42.5% (UI-TARS-1.5) to 47.5% (UI-TARS-2, September 2025), at or above what the closed frontier systems of those months managed.
- July 2025. OpenAI folds Operator into ChatGPT agent, which works on a virtual computer with a visual browser, a text browser and a terminal.
- November 2025 - January 2026. Google ships Gemini Agent, then "auto browse" inside Chrome for paid subscribers. Anthropic launches Cowork in January as a research preview, agentic file work in the Claude desktop app, and adds live screen control to Cowork and Claude Code in a March research preview.
- May 2026. Google quietly shuts Project Mariner down, effective May 4, folding its technology into Gemini Agent, Chrome's Auto Browse, and a computer-use tool in the Gemini API.
Two things stand out. The standalone browser agents died as standalone products; the capability got absorbed into general assistants. And open weights kept pace with the frontier, which is unusual in a capability this hard.
What OSWorld 2.0 actually measures
The original OSWorld tasks average about 30 tool calls. That is long enough to test whether a model can click reliably, and short enough that an agent with decent planning racks up a respectable score without ever holding state across a working session. By mid-2026, models were saturating it: the OSWorld 2.0 paper notes current frontier systems at 79-83% binary accuracy on OSWorld 1.0.
OSWorld 2.0, from the same XLANG Lab group, replaces the short-task design with 108 end-to-end workflows drawn from real professional work. The median task takes a skilled human about 1.6 hours, and an agent needs an average of 318 tool calls to finish one, against the ~30 of OSWorld 1.0.
Under the primary binary-completion metric at 500 steps: Claude Opus 4.8 with maximum thinking and batched tool calls tops the board at 20.6% completed, with a 54.8% partial score. GPT-5.5 is far more token-efficient and still plateaus near 13%.

The failure analysis in the paper matters more than the headline scores. Agents no longer stumble on basic GUI control. They fail by losing track of constraints set hundreds of steps earlier, missing information that arrives mid-task, guessing instead of asking the user, and skipping verification of their own work. These are workflow-discipline failures, not perception failures. The screens are legible. The shift is too long.
The economics are as bad as the accuracy

Even when computer-use agents succeed, they are slow in ways that break the product case. The OSWorld-Human study, the first systematic measurement of agent efficiency on OSWorld, found that large model calls for planning, reflection and judging dominate end-to-end latency, and that each successive step can take up to 3x longer than the first as context accumulates. Even the best agents take 2.7x to 4.3x more steps than a human-demonstrated trajectory for the same task.
Put that next to the OSWorld 2.0 arithmetic: a professional task that costs a human 1.6 hours costs an agent 300-plus tool calls at API prices, each one slower than the last. That is why the products that shipped aim at chores measured in minutes, not deliverables measured in hours.
What computer use is actually good for in late 2026
Short-horizon web chores. Form filling, price comparisons, account maintenance, research loops with confirmation gates. This is where every major assistant now converges, and where the products are usable.
Fallback when there is no API. The comparison RPA vendors and enterprises keep drawing is apt: an agent that reads the screen tolerates UI changes that break scripted robots, but it is slower and less predictable than either an integration or a robot. It makes sense exactly where integrations do not exist, legacy internal tools, portals, kiosks.
Sandboxed, supervised deployment. The serious configurations run the agent in a VM or container with human takeover, and keep manual confirmation on purchases and sends. Nobody credible lets a 20%-on-long-tasks agent run unsupervised against a real profile.
And notice where the labs actually invested: connectors, APIs, MCP-style tool interfaces, terminal access. Every one of those routes around the pixel problem. When the agent can call a tool instead of squinting at a screenshot, grounding and verification mostly vanish as failure modes. Mariner's shutdown, six months after its showcase moment, is the clearest evidence that pixel-driving is a fallback layer, not the destination.
The takeaway
Computer use in November 2026 is where code generation was in 2022: impressive demos, meaningful-looking benchmarks, and nobody willing to hand over the keyboard unsupervised. The number to watch through 2027 is not the binary completion rate on OSWorld 2.0. It is the ratio of partial credit to full completion, which measures whether agents learn to sustain a workflow, verify their own output, and stop guessing. The models can see the screen fine. They just cannot yet remember why they opened it.
Sources
- OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments - osworld-v1.xlang.ai / arXiv:2404.07972
- OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks - arXiv:2606.29537
- OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agents - arXiv:2506.16042
- OpenAI, Introducing Operator (incl. July 17, 2025 integration into ChatGPT agent) - openai.com/index/introducing-operator
- UI-TARS repository (ByteDance), OSWorld results tables - github.com/bytedance/ui-tars
- UI-TARS-2 Technical Report - arXiv:2509.02544
- Google shuts down Project Mariner, May 4, 2026 - Startup Fortune and 5WPR Gemini Agent guide
- Claude Cowork launch and availability timeline - AI Wiki: Claude Cowork, gend.co explainer
- Claude computer use in Cowork and Claude Code, macOS research preview (March 2026) - Firstpost
