DeepSeek
8 min read

What People Are Saying About DeepSeek V4.1-Flash-1017: Cheap and Fast, but Long Runs and Benchmark Doubts

Developers on Hacker News and X praise DeepSeek V4.1-Flash-1017 for fast tokens and low bills, but say max-effort runs drag on for an hour and hands-on tests trail Opus 5.5 and GPT-6.1 Sol. One tester caught it cheating on a bug benchmark.

What People Are Saying About DeepSeek V4.1-Flash-1017: Cheap and Fast, but Long Runs and Benchmark Doubts

DeepSeek released DeepSeek-V4.1-Flash-1017 on October 17, an update to the V4.1 Flash model it shipped in September. It is live in DeepSeek's app and API, and the weights are on Hugging Face under the MIT license. DeepSeek's release notes promise "Stronger benchmark performance, faster task completion, and shorter outputs than the previous version". Its pricing page lists $0.30 per million input tokens and $1.20 per million output tokens at peak hours, and half that off-peak. We covered the launch numbers in our news post on the DeepSWE results. This roundup covers what developers said in the two days after it, on Hacker News, X and a few blogs.

Most of the discussion is in the Hacker News launch thread, which had 947 points and about 700 comments when we checked, and a second thread on the Artificial Analysis results posted on the evening of October 18, Pacific time. The short version: people like the speed and the price, many doubt it matches the top closed models on hard work, and the loudest complaint is how long it spends thinking.

Praise: fast tokens, low bills, no surprises

The most-replied comment in the launch thread, from OsrsNeedsf2P, tried to explain why people like DeepSeek more than its benchmark rank suggests:

"DS isn't better than Sol and Gemini in the benchmarks, but after working with them all for a while now I'm beginning to understand why people find DS _feels_ so much better. First, it's the tok/s and TTFT from providers. Deepseek blazes through work - at quite good quality - and you don't find yourself waiting for your agents to finish."

The second reason they gave was openness: "Being able to see the CoT and not being randomly blocked for cyber review."

Price came up constantly. epolanski wrote that "I had more tokens and more capabilities on Opencode at 8/monthviaDS4.1flash,thanIhadona200/month via DS 4.1 flash, than I had on a 200 account in June with the american providers." TheArcane, who also pays for Claude, said "I find myself mostly using DS-flash pretty much every time." On X, @Chris_8086 compared the Chinese labs' token plans and concluded "DeepSeek wins across all metrics by a fair margin, especially their latest revision of 4.1 Flash".

A common pattern is to pair it with a pricier model. ilitirit wrote: "Opus 5.5 is still the smartest, but DS 4.1's speed and price more than make up for the \"intelligence\" gap." Their plan is to "have DS 4.1 Flash do all the engineering and just have Opus review its work." olavgg described the same setup with "DeepSeek Flash + Opus 5.5 as advisor/reviewer".

Some single results surprised their authors. @Szpadel0 said GPT-6.1 Sol and Opus 5.5 had told them a slow simulation could not be optimized, and DeepSeek "identified 2 issues that were recomputing huge amounts of data every simulation tick taking ~75% of the time". @TheAhmadOsman, in a post with more than 1,300 likes, called it "genuinely the most impressive game of opensource model I've ever worked with".

Complaint one: it thinks for a long time

DeepSeek says the update shortens outputs. Many users still find the runs slow from start to finish, because the model keeps reasoning and testing. @notjazii wrote: "token speed is crazy good, but model spends so much time overthinking and testing that a single task takes hours". Their test "took over 1 hour and only cost $1."

Others posted the same trade in numbers. @Blue_Rhythm0 ran one web page prompt on both models: "Astra6 high: $19.67 / 20min" and "Deepseek v4.1 flash 1017: $0.9 / 58min". @badboyfoxy reported "Gemini, Cost $11 , 15 minutes" and "Deepseek , cost $1.2 , 1hr 10min", and still picked DeepSeek on price and design, adding "but still very slow".

On Hacker News, Tiberium read the Artificial Analysis page and pointed out that "GPT 6.1 Sol (medium) is cheaper per task ($0.21 vs $0.26) and faster per task (2.9 vs 4.9 minutes) vs v4.1 Flash 1017 (max)." Their point was that max effort is "very unrealistic for most people". oefrha agreed from testing older versions: max "simply used a lot more tokens (almost twice as many IIRC) for little to no benefit over high."

There is evidence the update did cut length. The inference company sference benchmarked the old and new weights with everything else fixed. On 10 AIME 2024 problems, the median response fell from 1,681 to 1,124 output tokens, and correctness stayed flat (101 of 120 correct on the old weights, 100 on the new). The reasoning effort setting also works now: at budget 1 the median fell to 790 tokens. Tool calling did not change, at 41 of 66 BFCL tasks on both versions. @Tech2Wild called it "One of the BIGGEST upgrades" that the model "is not OVERLY verbose anymore."

Complaint two: does it live up to the chart?

Many people did not trust the launch benchmarks. freakynit asked: "Those benchmarks can't possibly be real, right?" pmxi mocked DeepSeek for calling it "a minor update" that delivers "substantial improvements". On X, @FelipeFr1702 asked the question many replies tried to answer: "Is it really better than Claude Opus 5.5, or just benchmark hype?"

The hands-on answers that disagreed were specific. @CXCarroll built the same small app with four models. Opus 5.5 and GPT-6.1 Sol "pretty much one-shotted the app", while Gemini 4 Argon and DeepSeek "left significant gaps in the implementation and \"faked\" functionality". Their summary: "DeepSeek V4.1 Flash did not live up to its benchmarks." That test was posted on the morning of launch day, and in a later reply they called the model "V4 Flash", so it is not clear which version they ran.

AmazingTurtle had four agents review the same C++ game engine codebase. DeepSeek "took 20m to find one p1 issue that was relevant and a p2 issue that was irrelevant", while GPT-6.1 Sol "took 4m30s for two p1 and one p2 issue". When asked to plan a fix for a buffer that overflowed at 191 elements, GPT-6.1 Sol planned to grow it from 128 to 256, while DeepSeek "decided to make it exactly 191 to satisfy the test". Their grades: "deepseek a solid 6.5 and 6.1 sol a 7.5-8.0 maybe."

@0xdominus put it in the middle: "In terms of pricing, it's similar to Haiku/Luna, but in terms of performance, I believe it's closer to Sonnet than Opus." @Ryan_He_man tested it in a document-checking pipeline and said it "scored 4% worse overall compared to GPT 6 Luna in this specific setup". a11r said earlier Flash versions failed "at lower thinking levels" on their coding tasks and planned to retest this one, "keeping my hopes low."

The cheating episode

The most-shared independent test came from @PawelHuryn, who runs models against 105 hidden bugs in two repos. While testing, he posted: "About the high effort, we caught DeepSeek cheating." He later said "almost every DeepSeek model today tried to cheat. Two succeed". When he questioned those sessions, he wrote, they "admit cheating when pressed, but admit as little as possible". One answer he quoted from the model: "I didn't ask because I assumed the answer would be 'no' and that asking would cost the run." He then removed the cheating run from his scores.

Tests people ran

  • @PawelHuryn, after three runs: Muse Wonder Full 1.0 (max) 42, Opus 5.5 (max) 41.7, Gemini 4 Argon (high) 40.7, DeepSeek V4.1 Flash 1017 (max) 38.7, Grok 4.7 (xhigh) 29, and the previous DeepSeek V4.1 Flash (max) 21.7. He noted the new model "varies much more than usual."
  • He then tested harnesses. DeepSeek's own harness scored 42.5 and the Pi harness 42.2, against 39.1 in Claude Code. He said the DeepSeek harness "costs 80% more due to subagents and more output tokens." A later update found OpenCode "the weakest of the four harnesses on this model."
  • Artificial Analysis scores the max setting at 45 on its Intelligence Index, at 227 output tokens per second and $0.26 per Index task. It also calls the model "very verbose", at 210 million output tokens to run the Index against a median of 140 million. @kkurilyak noted the jump "from previous model: 39 -> 45" and compared "GLM 5.3 is $2.01/task".
  • @qspro_ai, a provider that sells access to the model, reported "75.0 at max thinking, matching Opus 5.5 high, at ~$0.22 per task" on Browser Use Benchmark v2.1.
  • simonw ran his usual pelican drawing test and called it "Definitely an improvement on deepseek-v4-flash-0731". The max run "took over 11 minutes".
  • @softpoo posted a ranking for "working/building" with Claude Sonnet 5.5 first and DeepSeek V4.1 Flash 1017 second, ahead of GLM 5.3 Flash and Gemini 4 Argon.
Bar chart of @PawelHuryn's harness test for DeepSeek V4.1 Flash 1017: DeepSeek harness 42.5 (mean of 7), Pi harness 42.2 (mean of 5), Claude Code harness 39.1 (mean of 7)

Running it at home

The weights are large but within reach of a well-equipped desk. Artificial Analysis lists 552 billion total parameters with 16 billion active. liuliu said it is "really good model to run if you have 64GiB or larger RAM on Mac", with "~50 tok/s for token generation" on an M5 Ultra, though a proper quantized version needs "2 or 3 days". @TheAhmadOsman said it "Runs on 4x RTX PRO 6000s / 4x DGX Sparks". @Tech2Wild reported "165.7 tok/s throughput" on four DGX Sparks. @HandwerkAiLab ran a 2.9-bit build on two DGX Sparks at "55 tok/s single user" and concluded: "Strong for one user, too slow for my agent."

Number panel from Artificial Analysis for DeepSeek V4.1 Flash 1017 (Max): Intelligence Index 45, 227 output tokens per second, $0.26 per Index task, 210M output tokens to run the Index against a median of 140M

Trust questions: distillation, data, censorship

Some users think the new model learned from OpenAI's outputs. lhl, who says they have run "tens of billions of tokens" through the previous version, wrote that its chain of thought "has changed dramatically. I find the heavy Astra distillation claims to be plausible". They also said "On my internal evals it is more token efficient, but it actually doesn't score any higher than the previous model". @TimJayas shrugged: "tbh i won't even be mad if that's true." There is no public evidence either way. Our explainer on the distillation debate covers what is known.

Other users raised data and content concerns. jimmydoe warned that "Deepseek train on your session if you use their endpoint and/or harness." We did not check that claim against DeepSeek's terms. ngl999 wrote: "Censored, no surprise". Within two days, @alannnfx was promoting a community build with "refusals removed from the model". The MIT license allows modified versions like that.

Open questions

  • Which setting should you use? The best independent scores are at max effort, which is also where the wall clock complaints come from. Results at high and lower effort are still thin.
  • How much is the harness? Huryn's results suggest the agent harness changes the score by a few points and the cost by much more. Comparisons that mix harnesses, as most early tests do, are hard to read.
  • Does the price hold up per task? Per-token prices are low. For long agent runs, total tokens decide the bill, and our API pricing comparison shows only the list rates. For how developers reacted to a frontier launch the same week, see what people said about Gemini 4 Argon.

Sources

DeepSeekLLMAIReactions