Rotational Quantization: How Orthogonal Transforms and Hadamard Incoherence Enable Outlier-Free Low-Bit Inference

Large language model serving is governed by two physical bottlenecks: memory bandwidth during autoregressive decoding and compute throughput during prompt prefill. While weight-only post-training quantization (such as GPTQ or AWQ) reduces parameter footprint to 4 bits, it leaves activations in 16-bit floating-point formats. As a result, inference engines cannot utilize high-throughput INT4 tensor cores, and the key-value (KV) cache continues to consume massive memory pools.

Attempting to quantize activations to 4 bits (W4A4) using traditional uniform rounding causes immediate catastrophic accuracy degradation. The root cause is the emergence of systematic activation outliers: sparse, high-magnitude features that dominate specific hidden channels. Rotational quantization solves this structural problem by transforming the internal coordinate system of the transformer using orthogonal rotations. By rotating activation vectors with randomized Hadamard matrices or learned orthogonal transformations, these methods disperse outlier energy across all dimensions, converting heavy-tailed distributions into uniform sub-Gaussian profiles that fit standard low-bit integer grids without altering network output.

Rotational Quantization Incoherence and Outlier Elimination Diagram

The Activation Outlier Bottleneck in Low-Bit Serving

As language models scale past 6.7 billion parameters, their internal representations undergo an emergent phase transition. As documented by Dettmers et al. (2022), a tiny fraction of hidden channels (typically less than 0.1%) begin exhibiting magnitude spikes that are up to 100 times larger than the median activation values. These outliers are not random noise; they carry critical semantic and syntactic information, consistently appearing in the same coordinate dimensions across different sequence positions.

When quantizing an activation tensor $X \in \mathbb{R}^{B \times d}$ into a uniform low-bit grid (such as unsigned or signed 4-bit integers with $2^4 = 16$ discrete levels), the quantization step size $\Delta$ must span the entire dynamic range:

$\Delta = \frac{\max(X) - \min(X)}{2^b - 1}$

Because the dynamic range is dictated by the extreme outlier spikes, the vast majority of non-outlier features (which reside in a narrow band around zero) are collapsed into just one or two quantization bins. The resulting roundoff error destroys the representational capacity of the model.

Prior techniques attempted to mitigate this with channel-wise scaling. For instance, SmoothQuant (Xiao et al., 2023) applies a diagonal scaling matrix $S$ to divide out activation spikes and multiply them into the corresponding weight columns:

$Y = (X \cdot S^{-1}) (S \cdot W)$

While SmoothQuant works well for 8-bit quantization (W8A8), it breaks down at 4 bits (W4A4). Smoothing migrates difficulty from activations to weights, and when the target precision is 4 bits, the scaled weight matrix itself develops severe quantization distortion. Furthermore, diagonal scaling cannot resolve activation outliers that occur inside attention key-value projections or across intermediate non-linear states.

The Principle of Computational Invariance

Rotational quantization operates on the principle of computational invariance in neural networks, formalized in SliceGPT (Ashkboos et al., 2024) and QuaRot (Ashkboos et al., 2024).

Consider a standard linear projection $Y = X W$ , where $X \in \mathbb{R}^{B \times d}$ is an input activation matrix and $W \in \mathbb{R}^{d \times k}$ is the weight matrix. Let $R \in \mathbb{R}^{d \times d}$ be an orthogonal matrix satisfying $R^T R = R R^T = I$ , where $I$ is the identity matrix. Because $R R^T = I$ , we can insert this identity transformation directly between the activations and the weights:

$Y = X W = X (R R^T) W = (X R) (R^T W) = \tilde{X} \tilde{W}$

Here, $\tilde{X} = X R$ represents the rotated activation tensor, and $\tilde{W} = R^T W$ represents the rotated weight matrix. The output $Y$ is mathematically identical to the unrotated computation.

Because orthogonal transformations preserve Euclidean lengths and inner products ( $\|x R\|_2 = \|x\|_2$ for any vector $x$ ), the geometric norm of the hidden states remains invariant under rotation.

This property extends across normalization layers. Modern transformer architectures use Root Mean Square Normalization (RMSNorm):

$\text{RMSNorm}(x) = \frac{x}{\text{RMS}(x)} \odot \gamma, \quad \text{where } \text{RMS}(x) = \sqrt{\frac{1}{d} \sum_{i=1}^d x_i^2} = \frac{\|x\|_2}{\sqrt{d}}$

Because $\|x R\|_2 = \|x\|_2$ , the denominator $\text{RMS}(x R)$ equals $\text{RMS}(x)$ . Consequently, rotating the inputs to an RMSNorm layer rotates the normalized outputs by the exact same matrix $R$ , allowing rotations to be propagated through normalization layers without breaking the forward pass.

Incoherence Processing via Randomized Hadamard Transforms

The goal of rotational quantization is to select an orthogonal matrix $R$ that maximizes the incoherence of the activations. In compressive sensing and quantization theory, incoherence measures how uniformly distributed a vector's energy is across its basis coordinates.

If an activation vector has an isolated spike of magnitude $M$ along a single coordinate axis, its peak-to-average ratio is extremely high. By applying an orthogonal transform whose entries have uniform magnitude, that concentrated energy is redistributed equally across all $d$ dimensions.

The Walsh-Hadamard Transform (WHT) provides an optimal, multiplication-free orthogonal operator. A normalized Hadamard matrix $H_n \in \mathbb{R}^{n \times n}$ is defined recursively for powers of two:

$H_1 = [1], \quad H_{2k} = \frac{1}{\sqrt{2}} \begin{bmatrix} H_k & H_k \\ H_k & -H_k \end{bmatrix}$

Every element in $H_n$ is either $+1/\sqrt{n}$ or $-1/\sqrt{n}$ , and all columns are mutually orthogonal ( $H_n^T H_n = I$ ).

To prevent deterministic alignment between structured model features and the Hadamard basis, QuaRot and QuIP# (Tseng et al., 2024) employ the Randomized Hadamard Transform (RHT):

$R = H_d \cdot \text{diag}(s)$

where $s \in \{-1, +1\}^d$ is a random Rademacher vector with entries sampled uniformly from $\pm 1$ .

When an activation vector $x$ containing a coordinate outlier $M$ is multiplied by $R$ , the resulting rotated vector $\tilde{x} = x R$ exhibits coordinate values bounded by:

$\tilde{x}_j = \sum_{i=1}^d x_i R_{i,j} \approx O\left(\frac{M}{\sqrt{d}}\right)$

In a modern model with hidden dimension $d = 4096$ or $d = 8192$ , the factor $1/\sqrt{d}$ attenuates the outlier peak by a factor of 64 to 90. The coordinate distribution transitions from a heavy-tailed, spike-dominated profile into a spherically symmetric, Gaussian-like distribution. Because the dynamic range is compressed without truncating values, standard 4-bit uniform quantization bins cover the distribution with minimal quantization error.

Weight Absorption and Online Fast Walsh-Hadamard Transforms

Applying rotations at every layer could introduce unacceptable computational overhead if implemented naively. Rotational quantization frameworks divide transformations into two categories: offline fused rotations and online fast transforms.

1. Offline Weight Absorption

Wherever a rotation matrix directly precedes or follows a linear layer, it can be fused offline into the model weights during the quantization compilation phase:

For an input projection $W_{\text{in}}$ , the rotated weights are precomputed as $W_{\text{in, fused}} = R^T W_{\text{in}}$ .
For an output projection $W_{\text{out}}$ , the rotated weights are precomputed as $W_{\text{out, fused}} = W_{\text{out}} R$ .

Because these matrix multiplications are computed once during offline preparation, they introduce zero floating-point operations, zero memory overhead, and zero runtime latency during inference.

2. Online Fast Walsh-Hadamard Transform (FWHT)

Rotations cannot be absorbed across non-linear operations, such as the SwiGLU activation functions in modern feed-forward networks ($\text{SwiGLU}(x) = (x W_{\text{gate}} \odot \text{silu}(x W_{\text{up}})) W_{\text{down}}$) or the Softmax operation in multi-head attention.

In these locations, the model must apply an explicit rotation during runtime before quantizing the intermediate tensor (such as the input to the down-projection layer or the key-value states).

Instead of computing a general dense matrix multiplication ( $O(d^2)$ operations), the transformation is executed using the Fast Walsh-Hadamard Transform (FWHT). FWHT leverages the recursive Kronecker structure of Hadamard matrices via a butterfly algorithm, requiring only:

$O(d \log_2 d) \text{ operations}$

Crucially, the Walsh-Hadamard matrix contains only $\pm 1/\sqrt{d}$ entries. The butterfly computation requires exclusively addition and subtraction operations, with a single global scalar multiplication at the end. Highly optimized CUDA and Triton kernels execute the FWHT in memory-bandwidth-bound fashion, accounting for less than 1% to 2% of total per-layer inference time.

Learned Orthogonal Rotations: SpinQuant

While randomized Hadamard matrices successfully suppress outliers, their fixed structure possesses two notable constraints:

They require matrix dimensions to be powers of two (or products of small Sylvester-compatible integers), requiring padding for non-standard hidden dimensions.
They treat all feature dimensions uniformly, ignoring the non-isotropic curvature of the loss landscape.

To address this, SpinQuant (Liu et al., 2024) introduced learned orthogonal rotations. Rather than relying solely on fixed Hadamard transformations, SpinQuant parameterizes continuous rotation matrices $R$ directly on the Special Orthogonal group $SO(d)$ and optimizes them using gradient descent.

To guarantee that $R$ remains strictly orthogonal throughout gradient-based optimization, SpinQuant employs the Cayley transform:

$R = (I - S)(I + S)^{-1}$

where $S \in \mathbb{R}^{d \times d}$ is an unconstrained skew-symmetric matrix ( $S^T = -S$ ). For any real skew-symmetric matrix $S$ , the Cayley transform produces a strictly orthogonal matrix with determinant $+1$ .

During a lightweight calibration phase on a small dataset (such as 128 sequences from WikiText-2), SpinQuant optimizes the skew-symmetric parameters $S$ alongside quantization step sizes $\Delta$ to minimize the layer-wise Mean Squared Error (MSE) reconstruction loss:

$\mathcal{L}(S) = \mathbb{E}_{X} \left[ \| X W - Q(X R(S)) \cdot Q(R(S)^T W) \|_F^2 \right]$

where $Q(\cdot)$ denotes the uniform 4-bit quantization operator.

By optimizing the rotation matrices to align with the empirical Hessian of the activations, SpinQuant concentrates quantization precision in the directions that most heavily impact downstream task accuracy. On LLaMA-2-70B under full 4-bit weight, activation, and KV cache quantization (W4A4KV4), SpinQuant improves zero-shot benchmark accuracy by up to 16 percentage points compared to unoptimized round-to-nearest baselines, closing the remaining performance gap to unquantized FP16 models.

Hardware Execution and Serving Economics

The ability to maintain activation representations in 4-bit formats reshapes the economics of large language model serving across three fronts:

1. True INT4 Tensor Core Utilization

On modern GPU architectures (such as Nvidia Ada Lovelace, Hopper, and Blackwell), INT4 Tensor Core operations provide double the raw arithmetic throughput (TFLOPS/TOPS) of FP16 or BF16 operations. Weight-only quantization (W4A16) cannot leverage these integer units because inputs must be upcast to FP16 prior to matrix multiplication. Rotated W4A4 models execute end-to-end integer GEMMs directly on Tensor Cores, accelerating prompt prefill latency by over $2.1\times$ .

2. End-to-End KV Cache Footprint Reduction

During generation, the memory required by the KV cache scales linearly with context length and batch size:

$\text{Memory}_{\text{KV}} = 2 \times n_{\text{layers}} \times n_{\text{heads}} \times d_{\text{head}} \times L_{\text{seq}} \times B \times \text{precision}$

For a 70-billion-parameter model serving a 128k context window, a 16-bit KV cache requires tens of gigabytes per concurrent user. By rotating key and value projections before writing to cache memory, QuaRot and SpinQuant enable 4-bit KV caching (KV4) with zero outlier-induced perplexity blowup, reducing KV memory overhead by 75% and quadrupling maximum serving concurrency on a single node.

3. Outlier-Free Weight Quantization

Beyond activations, applying randomized Hadamard transformations to weight matrices (known as incoherence processing in QuIP#) removes weight outliers as well. This homogenizes the weight distribution into a sphere-shaped Gaussian distribution, allowing extreme sub-4-bit compression (including 2-bit vector quantization with $E_8$ lattice codebooks) to maintain downstream task performance.

Rotational quantization demonstrates that activation outliers are not an immutable property of large language models, but an artifact of standard Cartesian coordinate representations. By rotating internal representations into an incoherent coordinate space, modern inference runtimes achieve uniform 4-bit quantization across weights, activations, and KV caches without sacrificing model accuracy.