llms.blog
Model Compression4 articles

Model Compression

Articles

  • Knowledge Distillation: Mathematical Foundations, Dark Knowledge, Soft Target Regularization, and Sequence-Level Policy Transfer

    Knowledge Distillation: Mathematical Foundations, Dark Knowledge, Soft Target Regularization, and Sequence-Level Policy Transfer Knowledge distillation is a foundational model compression and transfer technique wherein a compact "student" neural network is trained to reproduce the functional behavior, internal representations, or output distributions of a larger, high-capacity "teacher" model or ensemble. First formalized in modern deep learning by Hinton, Vinyals, and Dean (2015), following ea

    1 min

  • Multiverse Computing Releases Quantization-Aware Healing to Boost 4-Bit Model Accuracy Above Full-Precision Baselines

    AI infrastructure firm Multiverse Computing has introduced Quantization-Aware Healing (QAH), a post-compression optimization technique designed to restore model accuracy after structural pruning and extreme quantization. Detailed in research paper 2608.20953, the method allows 4-bit compressed large language models to exceed the benchmark performance of their intermediate 16-bit unquantized counterparts. In standard model optimization workflows, teams apply structural pruning (removing layers,

    1 min

  • Quantization-Aware Training in Large Language Models: How Fake Quantization, STE, and Learned Step Sizes Preserve Low-Bit Accuracy

    Quantization-Aware Training (QAT) is a model compression paradigm that integrates precision loss directly into the training or fine-tuning graph. While Post-Training Quantization (PTQ) calibrates fixed floating-point weights without updating underlying network parameters, QAT simulates the numerical discretization of weights, activations, and key-value (KV) caches during both forward and backward passes. This closed-loop optimization forces neural network weights to co-adapt to discrete integer

    1 min

  • Weight Pruning in Large Language Models: How SparseGPT, Wanda, and Semi-Structured 2:4 Sparsity Compress Neural Networks Without Retraining

    Modern large language models require tens to hundreds of gigabytes of high-bandwidth memory to store billions of parameters. While post-training quantization compresses model footprints by reducing numerical precision from 16-bit floating point to 8-bit or 4-bit integers, weight pruning attacks model size along an orthogonal axis: setting redundant parameter values exactly to zero. Historically, pruning deep neural networks required iterative cycles of magnitude thresholding and compute-heavy r

    1 min