llama.cpp2 articles

llama.cpp

Articles

llama.cpp Merges DFlash 2 Support for Up to 2x Faster Speculative Decoding Across Long Contexts
The open-source llama.cpp project has merged native support for DFlash 2, bringing parallel speculative decoding and substantial inference throughput improvements to local LLM serving across CPU, Apple Silicon, and GPU backends. The implementation, integrated via Pull Request #27342, adds local convolution operators and candidate selector mechanics designed specifically for the DFlash 2 architecture. Non-Autoregressive Speculative Drafting Standard speculative decoding uses a smaller autoreg
August 27, 20261 min
Local LLM Inference on Apple Silicon: Architecture, Unified Memory, and Serving Benchmarks for MLX, llama.cpp, and Ollama
Local large language model (LLM) serving on consumer hardware has historically faced a hard trade-off between memory capacity and execution bandwidth. Discrete consumer GPUs offer high memory bandwidth (up to 1,008 GB/s on an Nvidia RTX 4090) but are capped at 24 GB of VRAM, requiring model sharding or quantization to fit models beyond 14 billion parameters. Apple Silicon platforms bypass this capacity ceiling through a Unified Memory Architecture (UMA), where the CPU, GPU, and Apple Neural Eng
August 18, 20261 min