1-Bit Large Language Models and BitNet b1.58: How Ternary Weights and Additive Kernels Eliminate Matrix Multiplications

Standard large language model architectures run on high-precision floating-point arithmetic. Foundation models are typically pre-trained in 16-bit bfloat16 or 8-bit FP8 formats, requiring compute-heavy Multiply-Accumulate (MAC) units inside GPU Tensor Cores. During autoregressive decoding, these models face a severe memory-bandwidth bottleneck: every generated token requires streaming gigabytes of model weights from high-bandwidth memory (HBM) into on-chip cache and registers.

The 1-bit model paradigm, pioneered by researchers at Microsoft Research through the BitNet architecture and its successor BitNet b1.58, challenges this foundation. By constraining every parameter in the linear projection layers to ternary states {-1, 0, +1}, BitNet eliminates floating-point matrix multiplications from the core Transformer pipeline. Instead, standard General Matrix Multiply (GEMM) operations are replaced by integer additions and subtractions.

The Mathematical Evolution from Binary to Ternary Weights

Early attempts at extreme weight binarization, such as BNNs and 1-bit Transformers, mapped continuous parameters strictly to binary states {-1, +1} using sign functions. While 1-bit binary weights reduced storage to a single bit per parameter, they suffered from significant degradation in language modeling perplexity. Binary networks lacked the capacity to represent feature absence or suppress noisy activations.

BitNet b1.58 resolved this limitation by introducing a third state: zero. Because a ternary system holds three distinct discrete values, the information entropy per parameter is calculated as:

$\log_2(3) \approx 1.58496 \text{ bits}$

Adding zero allows the network to explicitly filter out irrelevant tokens, features, and attention dimensions. This explicit sparsity mechanism restores model expressivity, enabling 1.58-bit models to match full-precision 16-bit Transformer baselines across both perplexity and downstream reasoning benchmarks at equivalent parameter scales.

BitLinear layer architecture and ternary addition mechanics

Inside the BitLinear Layer

In a standard Transformer block, feed-forward networks (FFN) and multi-head attention projections rely on standard linear layers $Y = W X$ . BitNet replaces these dense layers with custom BitLinear layers that perform weight and activation quantization prior to projection.

1. Absmean Weight Quantization

BitNet b1.58 compresses full-precision latent weights into ternary representations using an absolute mean (Absmean) quantization function. First, the layer calculates an average absolute scale factor $\gamma$ across the weight matrix $W \in \mathbb{R}^{n \times m}$ :

$\gamma = \frac{1}{nm} \sum_{i=1}^n \sum_{j=1}^m |W_{ij}|$

Each weight is then scaled by $\gamma$ , rounded to the nearest integer, and clamped to the range $[-1, +1]$ :

$\widetilde{W}_{ij} = \text{Clip}\left(\text{Round}\left(\frac{W_{ij}}{\gamma}\right), -1, +1\right)$

This constrains every quantized parameter $\widetilde{W}_{ij}$ strictly to $\{-1, 0, +1\}$ .

2. Absmax Activation Quantization

Activations must retain higher dynamic range than weights to preserve sequence modeling capacity. BitLinear applies per-token 8-bit quantization using an absolute maximum (Absmax) scaling rule.

Prior to quantization, activations are normalized using SubLN (Sub-Layer Normalization) or RMSNorm to prevent activation outliers from destabilizing low-precision ranges. For an input vector $X$ , the layer determines the maximum absolute value:

$\beta = \max_{j} |X_j|$

Activations are then mapped onto an 8-bit signed integer grid spanning $[-Q_b, Q_b]$ , where $Q_b = 2^{b-1} = 128$ for 8-bit precision:

$\widetilde{X}_j = \text{Clip}\left(X_j \times \frac{Q_b}{\beta}, -Q_b + \epsilon, Q_b - \epsilon\right)$

Here, $\epsilon$ represents a small floating-point constant that prevents zero-division during backpropagation.

3. Additive Integer Accumulation

With ternary weights $\widetilde{W} \in \{-1, 0, +1\}^{n \times m}$ and INT8 activations $\widetilde{X} \in [-128, 127]^{m \times k}$ , matrix multiplication simplifies into pure addition and subtraction:

$Y = \left( \widetilde{W} \widetilde{X} \right) \times \left( \frac{\beta \gamma}{Q_b} \right)$

Evaluating the inner product $\sum_k \widetilde{W}_{ik} \widetilde{X}_{kj}$ requires no hardware multipliers. For each element:

If $\widetilde{W}_{ik} = +1$ , the INT8 activation $\widetilde{X}_{kj}$ is added to an integer accumulator.
If $\widetilde{W}_{ik} = -1$ , the activation is subtracted from the accumulator.
If $\widetilde{W}_{ik} = 0$ , the operation is skipped.

The accumulated integer results are rescaled once per matrix block by the scalar factor $\frac{\beta \gamma}{Q_b}$ , restoring the tensor to the hidden dimension scale with minimal floating-point overhead.

4. Quantization-Aware Training with STE

Because the $\text{Round}(\cdot)$ and $\text{Clip}(\cdot)$ functions have zero gradients almost everywhere, BitNet uses a Straight-Through Estimator (STE) during pre-training.

In the forward pass, quantized weights $\widetilde{W}$ and activations $\widetilde{X}$ are computed and used in the additive GEMM. In the backward pass, gradients pass directly through the quantization operators to update high-precision (FP16 or BF16) master weights:

$\frac{\partial \mathcal{L}}{\partial W} \approx \frac{\partial \mathcal{L}}{\partial \widetilde{W}}$

This enables standard optimizers such as AdamW to accumulate sub-threshold updates until latent parameters cross the rounding boundaries.

Energy, Memory Bandwidth, and Serving Economics

The arithmetic shift from floating-point multiplication to integer addition directly alters the energy economics of LLM inference.

According to standard semiconductor energy benchmarks compiled by Mark Horowitz, the energy cost of arithmetic operations at 45nm process nodes breaks down as follows:

16-bit FP Multiply: ~3.7 pJ
16-bit FP Addition: ~0.9 pJ
8-bit INT Addition: ~0.1 pJ

Replacing FP16 multiplications with INT8 additions reduces the arithmetic energy consumed per operation by more than 35x.

However, in modern memory-bound LLM serving regimes, DRAM memory bandwidth dominates power and latency. During token-by-token generation with a batch size of 1, each parameter must be transferred from off-chip VRAM to on-chip SRAM registers for every generated token.

1.58-bit weights require only 2 bits of physical storage per parameter (or 1.58 bits under compact ternary packing). Compared to standard 16-bit models (16 bits per parameter), BitNet achieves a 7x to 8x reduction in memory bandwidth consumption. A 70-billion-parameter model, which typically requires roughly 140 GB of VRAM in FP16, compresses to under 20 GB in BitNet b1.58, fitting within the unified memory of a single consumer device or edge accelerator.

Scaling Laws and Empirical Benchmarks

Published evaluations in the BitNet b1.58 technical paper and subsequent large-scale pre-training runs detailed in the BitNet b1.58 2B4T Technical Report demonstrate clear scaling properties:

Capacity Parity at Scale: While 1.58-bit models show a minor capacity deficit at tiny scales (under 700 million parameters), models at 3 billion parameters and above match full-precision 16-bit LLaMA baselines in validation perplexity and downstream benchmarks (ARC, HellaSwag, Winogrande, MMLU, GSM8k).
Throughput Scaling: At 70B scale, BitNet b1.58 delivers up to 4.1x lower latency and up to 8.9x higher batch throughput compared to FP16 LLaMA running on identical hardware constraints.
CPU and Edge Execution: The official bitnet.cpp inference framework achieves 1.5x to 4.0x speedups on standard x86 and ARM CPUs using specialized SIMD/AVX2/NEON integer addition kernels, enabling native local LLM execution without discrete GPUs.

Studies examining smaller networks, including BitNet b1.58 Reloaded, confirmed that adjusting quantization baselines to the median (Absmedian) rather than the mean further stabilizes lower-capacity models.

Architectural Trade-offs and Hardware Realities

Despite its theoretical advantages, deploying 1.58-bit architectures in production introduces distinct engineering constraints:

Pre-Training Requirement: BitNet b1.58 is not a Post-Training Quantization (PTQ) method. Dense 16-bit checkpoints cannot be trivially converted to ternary representations without substantial degradation. Models must be trained from scratch with BitLinear layers or initialized through compute-intensive distillation pipelines.
Residual Full-Precision Components: BitLinear replaces the majority of matrix operations in self-attention and MLP blocks, but specific layers remain in higher precision. The initial token embedding table, output logits projection, and multi-head attention softmax calculations remain in 16-bit or 8-bit formats to prevent unrecoverable representation collapse.
Hardware Mismatch on Modern GPUs: Current data center GPUs (such as Nvidia H100 and B200) are heavily optimized for structured FP8 and FP4 Tensor Core GEMM pipelines. Because contemporary GPUs lack native ternary addition hardware blocks, BitNet models running on CUDA run through integer SIMD kernels or custom fused kernels. Full hardware efficiency gains will depend on emerging domain-specific architectures (DSAs) and specialized 1-bit neural processing units.

1-bit architectures demonstrate that high-precision floating-point weights are not a fundamental prerequisite for deep language understanding. By turning matrix multiplication into basic addition, ternary networks present a viable path toward scaling LLMs across both hyper-scale data centers and resource-constrained edge hardware.