Liquid AI has released Quantization-Aware Distillation (QAD) Q4_0 GGUF checkpoints for its LFM2.5 model series, allowing edge runtimes to execute 4-bit quantized non-transformer architectures without the accuracy degradation typically associated with standard post-training quantization (PTQ).
The release covers four models in the LFM2.5 family: LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B. The checkpoints are packaged in the standard GGUF format and run across llama.cpp and compatible inference backends.
Benchmark Recovery and Accuracy Retention
Traditional post-training quantization methods, such as standard rounding or calibration-based PTQ, often introduce noticeable performance penalties when compressing models below 3 billion parameters down to 4 bits. Liquid AI addressed this by combining knowledge distillation directly with quantization objectives during post-training.
Evaluated across a benchmark suite spanning reasoning (GPQA Diamond, MMLU-Pro), instruction following (IFEval, IFBench, Multi-IF), tool calling (BFCLv4), and mathematics (GSM8K for sub-1B models, AIME25 for 1.2B and 2.6B models), the QAD Q4_0 checkpoints retained near-uncompressed capability:
- LFM2.5-230M: Retained 97.1% of original BF16 baseline performance.
- LFM2.5-350M: Retained 96.5% of original BF16 baseline performance.
- LFM2.5-1.2B-Instruct: Retained 97.4% of original BF16 baseline performance.
- LFM2.5-2.6B: Retained 96.6% of original BF16 baseline performance.
In head-to-head comparisons against standard post-training quantization levels, the 230M and 350M QAD Q4_0 models matched the accuracy of Q5_K_M checkpoints while running at significantly smaller memory footprints. The 1.2B and 2.6B models matched Q4_K_M and Unsloth UD-Q4_K_XL quality thresholds.
Hardware Profiling and Edge Throughput
Liquid AI benchmarked decode throughput across four edge deployment targets representing different compute architectures: Apple MacBook Pro (Metal GPU), Minisforum NucBox EVO-X2 (integrated GPU), Samsung Galaxy S26 Ultra (Arm CPU), and Raspberry Pi 5 (Arm CPU).
Because standard Q4_0 formats allow simplified vector dequantization compared to complex block-level k-quantization schemes (such as Q4_K_M or Q5_K_M), the QAD checkpoints delivered measurable throughput advantages:
- Sub-1B Tier (230M and 350M): Achieved 4% to 33% higher token decode throughput compared to Q5_K_M while maintaining equivalent evaluation accuracy.
- Mid-Tier (1.2B and 2.6B): Achieved 3% to 14% higher decode throughput relative to Q4_K_M formats.
Edge Runtime Integration
The updated models are available on Hugging Face under the LiquidAI organization repository. Developers can run the QAD checkpoints directly via llama-cli or downstream runtimes without specialized custom kernels, making low-latency local agent execution viable on low-power consumer devices and embedded single-board computers.
