Nearly every large language model in production writes like a typewriter. GPT, Claude, Gemini, Llama: they all produce one token per forward pass, left to right, and never revise a word once it is on the page. A small camp of models has spent the past two years arguing that this is a design choice, not a law of nature. Diffusion language models (dLLMs) start from a block of blank placeholders and fill in hundreds of tokens at once, then refine the draft over a handful of passes. In 2025 that was a research demo. In 2026 it is a product category: Google ships open weights, Inception Labs sells an API with a million-token context window, and Ant Group has open-sourced a 100-billion-parameter model.
Here is how the idea works, where it stands, and what it still cannot do.
The sequential bottleneck
Autoregressive generation has a hard arithmetic problem. A 2,000-token answer costs 2,000 sequential forward passes, each one dependent on the last. Worse, each pass mostly re-reads model weights and the key-value cache from memory to emit a single token, which makes decoding bound by memory bandwidth rather than raw compute. Cloud providers hide this by batching thousands of users onto the same GPU. A single person running a model locally cannot, which is why a capable open model on a desktop crawls at 30 to 60 tokens per second even on expensive hardware.
Google's own description of DiffusionGemma frames the trade plainly: sequential decoding "leaves your dedicated GPU or TPU underutilized, it spends most of its time simply waiting for the next keystroke."
Masked diffusion, in one paragraph

Diffusion models generate images by corrupting data with noise and learning to reverse the corruption. The text version is simpler than it sounds. During training, random tokens in a real document are replaced with a mask symbol, and the Transformer, using attention in both directions, learns to predict all the masked positions at the same time. During generation the model receives a fully masked block of, say, 256 slots. It predicts every slot in one pass, locks in the tokens it is most confident about, re-masks or keeps refining the rest, and repeats until the block is complete. Each step costs one forward pass, and each pass commits many tokens instead of one.
The intellectual lineage runs from discrete diffusion formulations like D3PM in 2021 and score-entropy methods like SEDD in 2023 through to a 2025 survey of the field that now catalogs training, inference and decoding strategies for the whole family.
The LLaDA turning point
The moment the paradigm stopped being a toy was February 2025, when the LLaDA team, led by Ant Group researchers with academic collaborators, published an 8-billion-parameter diffusion language model trained from scratch, end to end, under the same pre-training plus supervised fine-tuning recipe autoregressive shops use. LLaDA 8B matched LLaMA 3 8B on in-context learning benchmarks and, in a detail that drew attention in interpretability circles, handled reversal prompts ("what is the capital of France" reversed) that trip up GPT-4o, because bidirectional attention does not care which half of a fact appears first.
The line kept scaling. In December 2025 the successor LLaDA2.0 became the first diffusion language model reported at the 100-billion-parameter scale, a Mixture-of-Experts family converted from an autoregressive checkpoint through a three-phase continual pre-training program rather than trained from scratch. The 100B flash variant reaches up to 535 tokens per second with confidence-aware parallel decoding, and both models are open-sourced. Follow-up releases added token-editing generation (LLaDA2.1, February 2026) and an agent-oriented editing model (LLaDA2.2, July 2026).
The speed numbers
Throughput is the headline claim, and the published figures are consistent.
- Mercury, from Inception Labs, reported in June 2025 that Mercury Coder Mini sustained 1,109 tokens per second on an H100 in independent Artificial Analysis testing, up to 10x faster than speed-optimized autoregressive rivals at comparable code quality.
- Gemini Diffusion, Google DeepMind's May 2025 research preview, averaged 1,479 tokens per second and matched Gemini 2.0 Flash-Lite on coding: 89.6% versus 90.2% on HumanEval.
- DiffusionGemma, released June 10, 2026 under Apache 2.0, is Google's first open-weight text diffusion model: a 26B Mixture-of-Experts with 3.8B active parameters, generating 256-token blocks in parallel at 1,000+ tokens per second on an H100 and 700+ on an RTX 5090, roughly 4x faster than conventional decoding. Quantized, it fits in 18GB of VRAM.
- Inception's commercial line moved in parallel: Mercury 2 (February 2026) claimed to be the first reasoning dLLM, at 1,009 tokens per second and $0.75 per million output tokens; Mercury 3 (October 2026) went to a 1M-token context window at about 1,068 tokens per second.
Where it still loses
The honest picture includes three catch-up problems.
Quality, first. Google states outright that DiffusionGemma's overall output quality is lower than standard Gemma 4 and recommends autoregressive models for applications that demand maximum quality. Gemini Diffusion's own benchmark table shows the gap is not uniform: it matched Flash-Lite on code but scored 40.4% versus 56.5% on GPQA Diamond and 15.0% versus 21.0% on BIG-Bench Extra Hard. Coding has structure that parallel generation can exploit; open-ended reasoning so far does not.
Error accumulation, second. Unmasking many tokens per step is statistically easier said than done: the tokens are interdependent, and committing several of them before seeing each other's final values causes mistakes to compound, an issue the survey literature calls the parallel decoding curse. Decoding schedules that lock tokens by confidence, or edit previously committed tokens instead of freezing them (LLaDA2.1's approach), are the current mitigations.
Economics, third. The speed advantage is largest for one user on one accelerator. In high-concurrency cloud serving, autoregressive models already batch requests to saturate compute, so a diffusion model's extra per-step compute buys diminishing returns and can raise serving cost, by Google's own account. The efficiency bet flips: autoregressive decoding is bandwidth-bound, diffusion decoding is compute-bound.
Why it matters anyway
Diffusion models are not a faster typewriter. They are a different medium. Bidirectional, iterative generation makes some operations native that autoregressive models fake with scaffolding: in-fill in the middle of a document, in-line editing, generating and revising structured output like valid JSON or closed markdown, and constraint-heavy puzzles (Google points to a fine-tuned DiffusionGemma Sudoku solver, trivial for the architecture and hard for left-to-right models, because every cell depends on tokens that have not been written yet).
The state of play in late 2026: autoregressive models remain the default for maximum quality at scale, and diffusion language models hold a growing beachhead in latency-critical, interactive work, voice agents, inline code completion, local editing, agent loops, anywhere a human or another model waits on the next token. A paradigm that had no credible 100B-parameter example eighteen months ago now has one with open weights. The typewriter is not dead. It just has a competitor that drafts like a human editor: rough pass first, revisions after.
Sources
- Large Language Diffusion Models (LLaDA), arXiv:2502.09992
- Mercury: Ultra-Fast Language Models Based on Diffusion, arXiv:2506.17298
- Gemini Diffusion, Google DeepMind
- Introducing DiffusionGemma, Google
- DiffusionGemma model overview, Google AI for Developers
- LLaDA2.0: Scaling Up Diffusion Language Models to 100B, arXiv:2512.15745
- LLaDA2.X repository, InclusionAI, Ant Group
- Introducing Mercury 2, Inception Labs
- A Survey on Diffusion Language Models, arXiv:2508.10875
