DeepSeek's V4 Flash has become the most-used AI model on OpenRouter and one of the highest-rated open-weight models available. In real agent tests, though, it finished barely more than half of the jobs it was given. The gap between leaderboard and workplace is the story.
What Composio found
The integration company Composio ran V4 Flash through eight agent harnesses, including Claude Code, Codex, and OpenCode, on 30 deliberately hard multi-step tasks. The tasks used live tools: Gmail, GitHub, Slack, and Google Sheets.
Across 240 runs, 129 passed, a completion rate of 53.8 percent. Only six of the 30 workflows were finished successfully by every harness tested.
Same model, different results
The same model produced very different scores depending on the harness, configuration, caching, and retries. That points to a shift in what matters for enterprise use. Raw model quality still sets the ceiling, but orchestration, the scaffolding that wires a model to tools, often decides whether a task actually gets done.
Prices are moving too
DeepSeek released V4 Flash in public beta on July 31 and made the larger V4 Pro generally available on August 13. Flash runs on 284 billion parameters and is built for speed and volume. Pro uses 1.6 trillion parameters for heavier workflows.
Both models are now getting more expensive. DeepSeek said it will raise prices for V4 Flash and V4 Pro, and reporting puts some increases at roughly four times the earlier cost. The low price was a big part of the model's appeal, so the move changes the math for teams that picked it to cut bills.
How it got here
V4 Flash took over OpenRouter's usage leaderboard the moment it shipped and remains the platform's most-used model by weekly token volume. ML researcher Nathan Lambert called the early adoption numbers insane and noted the model scored about the same as GLM 5.2. Epoch AI rated it an ECI of 153, between Anthropic's Opus 4.5 and 4.6, and the second strongest open-weight model behind Moonshot's Kimi K3.
The result is a more nuanced picture of the model. It is cheap, popular, and strong on benchmarks, yet real agent work still breaks roughly half the time. For builders, the lesson is that the harness around the model may matter as much as the model itself.
Sources
- VentureBeat: DeepSeek's top-ranked V4 Flash stumbles on real agent tasks as its prices surge (Aug 16, 2026) - https://venturebeat.com/orchestration/deepseeks-top-ranked-v4-flash-stumbles-on-real-agent-tasks-as-its-prices-surge - Composio agent benchmark post on X - https://x.com/composio/status/2085330847951970801 - Engadget: DeepSeek AI models get four times pricier - https://www.engadget.com/2236912/deepseek-ai-models-get-four-times-pricier/ - Epoch AI on X: V4-Flash-0731 ECI 153 - https://x.com/EpochAIResearch/status/2084788991153586600
