The open-source llama.cpp project has merged native support for DFlash 2, bringing parallel speculative decoding and substantial inference throughput improvements to local LLM serving across CPU, Apple Silicon, and GPU backends.
The implementation, integrated via Pull Request #27342, adds local convolution operators and candidate selector mechanics designed specifically for the DFlash 2 architecture.
Non-Autoregressive Speculative Drafting
Standard speculative decoding uses a smaller autoregressive draft model to predict subsequent tokens sequentially before passing them to the target foundation model for batched verification. While effective at short sequence lengths, conventional drafters often suffer throughput degradation as context windows expand due to accumulating draft latency and memory bandwidth pressure.
DFlash 2 addresses this bottleneck by replacing sequential drafting with a non-autoregressive architecture. Using local convolution layers and an integrated candidate selector, the draft model proposes multiple candidate tokens in parallel during a single forward step.
- Parallel candidate generation eliminates the sequential loop inside the draft phase.
- Consistent throughput scaling preserves roughly 2x decoding speedups across long context lengths up to 32K tokens.
- Exact target model distribution preservation guarantees zero output quality degradation during verification.
Serving Architecture and Local Deployment
The integration allows developers running llama-server or the llama-cli binary to pair target models (such as Qwen3.8-27B GGUF weights) with corresponding DFlash 2 drafter models using the --spec-type draft-dflash flag.
By offloading candidate generation to lightweight parallel structures, local hardware configurations (including AMD Strix Halo APUs, Apple M-series chips, and discrete NVIDIA GPUs) can achieve double the decoding throughput without modifying precision or base weights.
