Z Lab has released DFlash 2 checkpoints for Alibaba's Qwen 3.8 27B model family, advancing block-diffusion speculative decoding for open-weights LLM serving. By replacing conventional autoregressive draft models with a non-causal diffusion mechanism paired with direct target key-value (KV) cache injection, the framework achieves up to 3x to 4.3x throughput speedups in production inference engines like SGLang and vLLM without altering output token distributions.
Speculative decoding conventionally relies on a smaller draft model generating tokens sequentially, which are then verified in a single forward pass by the larger target model. DFlash departs from this sequential bottleneck by formulating draft generation as a single-pass block-diffusion task.
Block Diffusion vs. Autoregressive Drafting
In standard speculative architectures such as EAGLE-3 or multi-token prediction (MTP), the drafter remains an autoregressive model that must iterate step-by-step to produce multiple candidate tokens. When drafting deeper sequences (e.g., 8 to 16 tokens), the cumulative latency of the draft loop erodes the wall-clock speedup achieved during the verification step.
DFlash utilizes a non-causal attention mask over masked draft token slots. Rather than predicting token t+1, then t+2, the diffusion draft model predicts an entire block of 8 to 16 draft tokens simultaneously in a single forward pass. This hardware-friendly execution profile maximizes GPU and TPU tensor core utilization during drafting.
Persistent Conditioning Through Target KV Injection
A core limitation in prior multi-layer drafters has been feature dilution: feeding target model representations only into the input layer of the draft network causes contextual signal loss in deeper layers.
DFlash addresses this with a direct KV injection mechanism:
- Target Feature Extraction: Hidden states from selected intermediate layers of the target model (such as Qwen 3.8 27B) are extracted and concatenated.
- Dimension Projection: The concatenated features are normalized via RMSNorm and projected into the draft model's hidden dimension.
- KV Cache Injection: Instead of passing these vectors purely as input embeddings, DFlash injects the projected features directly into the Key and Value projections across all layers of the draft model.
By storing these representations directly within the draft model's KV cache, the drafter maintains persistent conditioning across all layers and decoding iterations, preventing draft acceptance rates from degrading across longer sequence lengths.
Framework Integration and Deployment
The newly released z-lab/Qwen3.8-27B-DFlash2 checkpoint is supported across major LLM serving frameworks:
- SGLang: Supported natively under the
Spec V2engine with asynchronous scheduling and memory overlap. - vLLM: Accessible through the speculators extension framework, utilizing dual-cache paths to isolate paged attention from non-causal draft buffers.
- Transformers: Available for local testing and weight inspection.
Benchmark evaluations across coding (HumanEval), mathematical reasoning (GSM8K), and conversational datasets (MT-Bench) demonstrate that DFlash 2 maintains an average token acceptance length of 4.0 to 4.8 tokens per verification cycle, delivering 2.5x the generation speed of EAGLE-3 and up to 4.3x the throughput of baseline autoregressive decoding at concurrency 1.
