In multi-model AI workflows and long-horizon agent loops, routing tasks between models of varying parameter counts incurs substantial computational overhead. When a smaller model escalates a complex reasoning task to a larger model, or when a larger model offloads multi-turn dialogue to a smaller model, the receiving model must execute a full prefill pass over the entire conversation history. Because the computational cost of prefilling scales with sequence length and model dimensions, this recomputation introduces significant latency and hardware costs.
Researchers at Nvidia have introduced a technique for cross-model Key-Value (KV) cache transfer that projects prefilled memory representations directly between models without rerunning the prefill phase. By demonstrating that internal KV cache representations share a strongly linear relationship across model checkpoints, the researchers designed a closed-form ridge regression mapper that accelerates cross-model memory handoffs by 2.7x to 25x while retaining up to 98% of target model accuracy.
The Prefill Bottleneck in Multi-Model Routing
Autoregressive transformer inference divides into two computational phases: prefill and decode. During prefill, the model computes key and value projection tensors for all tokens in the input context and stores them in GPU memory as the KV cache. In subsequent decode steps, the model generates tokens one by one by attending over this stored cache.
In production agent systems, context windows expand across multi-turn interactions, tool invocations, and document ingestion. When an agent swaps models dynamically, such as dispatching routine validation to an 8B model while reserving a 70B model for intricate code generation, the architectural mismatch between source and target networks invalidates the cache. The target model is forced to recompute all key-value matrices from scratch, introducing high time-to-first-token (TTFT) latency and consuming redundant compute cycles.
Closed-Form Linear Ridge Mapping
Nvidia's research indicates that the underlying geometry of KV caches across model sizes within the same architectural family is largely linear. In empirical tests mapping a 14B parameter Qwen3 model to a 32B parameter variant, a single-layer linear regression recovered 56% of the variance in target keys and 32% in target values. Aggregating multiple source layers raised variance recovery to 79% and 65% respectively.
To operationalize this linearity without requiring backpropagation or deep neural network training, the researchers developed a three-stage transfer pipeline:
- Content-Space Transformation: The mapper removes Rotary Position Embeddings (RoPE) before translation. Stripping position-dependent rotational frequencies allows the mapping weights to operate purely on semantic content, enabling the pipeline to generalize across sequence lengths beyond the calibration set.
- Cross-Layer Source Selection: Because source and target models differ in depth, the system evaluates and selects the most predictive subset of source layers for each target layer, ensuring optimal information flow across layer hierarchies.
- Per-Head Ridge Regression: The transformation uses closed-form ridge regression fit independently for each attention head using a lightweight calibration dataset of 500 sequences (1,024 tokens each).
Empirical Benchmarks and Accuracy Retention
The method was evaluated across six matched-KV model configurations spanning the Qwen3, Llama 3.1, and Ministral 3 families, with parameter jumps ranging from 3B to 70B. Evaluations spanned core reasoning benchmarks including MMLU, GSM8K, ARC-Challenge, HellaSwag, and WinoGrande, alongside CoQA multi-turn evaluation and WikiText-2 perplexity.
Across four of the six tested model pairs, the closed-form ridge mapper preserved between 73% and 98% of standalone baseline prefill accuracy. In an 8.8x parameter scale transition from Llama 3.1 8B to Llama 3.1 70B, the linear transfer maintained 72.8% of the target model's standalone performance.
Transfer latency demonstrated substantial speedups over standard prefilling. Translating a 32,768-token KV cache from Qwen3 14B to Qwen3 32B executed in 278 milliseconds, compared to approximately 7,000 milliseconds for a standard prefill pass. On multi-turn dialog evaluations, accuracy drift remained stable across 10 sequential conversational turns.
For configurations exhibiting non-linear feature divergence, such as select Ministral pairs, replacing the closed-form linear mapper with a shallow two-layer Multi-Layer Perceptron (MLP) restored benchmark accuracy above 90%, albeit with the requirement of lightweight supervised training.
