Model Merging in Large Language Models: How Task Arithmetic, TIES, and DARE Combine Checkpoints Without Training

Fine-tuning foundation models for specialized tasks typically produces isolated checkpoints. A model adapted for mathematical reasoning retains high numerical precision but often degrades in general dialogue or code generation. Traditionally, unifying these capabilities required multi-task training: gathering mixed datasets, re-running optimization across multiple GPUs, and managing gradient conflicts during backpropagation.

Model merging provides an alternative paradigm. By operating directly in weight space, model merging combines parameters from multiple fine-tuned checkpoints derived from a shared base model without running gradient descent or loading training data.

Weight Space Geometry and Mode Connectivity

Model merging relies on linear mode connectivity. In overparameterized neural networks, models fine-tuned from the same pre-trained initialization reside within the same loss basin. As demonstrated by Wortsman et al. (2022) in their work on Model Soups, interpolating between checkpoints fine-tuned on the same or related objectives often encounters no high-loss barrier, yielding performance comparable to or exceeding individual fine-tunes.

When multiple models originate from identical base weights $\theta_0$, each specialized model $\theta_t$ moves along a trajectory in parameter space. Because the initialization is shared, the displacement vectors represent task-specific adaptations that can be linearly combined.

Task Vectors and Task Arithmetic

The foundational primitive of modern parameter merging is the task vector, introduced by Ilharco et al. (2023). For a model fine-tuned on task $t$ with final weights $\theta_t$ and pre-trained base weights $\theta_0$, the task vector $\tau_t$ is defined as:

$$\tau_t = \theta_t - \theta_0$$

Task arithmetic treats these parameter differences as directional vectors in weight space:

• Multi-Task Addition: Combining multiple capabilities into a single model by summing scaled task vectors onto the base model:

$$\theta_{\text{merged}} = \theta_0 + \sum_{t=1}^{N} \lambda_t \tau_t$$

where $\lambda_t$ is a scaling hyperparameter controlling the strength of task $t$.

• Task Negation / Unlearning: Removing undesirable behaviors (such as toxic generation or copyrighted style) by subtracting a fine-tuned vector:

$$\theta_{\text{unlearned}} = \theta_0 - \lambda \tau_{\text{toxic}}$$

• Analogy Arithmetic: Transferring capabilities across domains by combining task vectors from complementary fine-tunes.

While naive task arithmetic works for low numbers of closely related tasks, scaling to many checkpoints introduces destructive interference: updates to the same parameter across tasks frequently conflict in magnitude and direction.

Geometric Preservation: SLERP

When combining two models, standard linear interpolation (LERP) averages weights coordinate-by-coordinate:

$$\theta(t) = (1 - t)\theta_A + t\theta_B$$

Linear interpolation cuts directly through the high-dimensional parameter space, reducing the vector norm in intermediate regions. This magnitude shrinkage alters layer normalization scaling and weakens activation magnitudes.

Spherical Linear Interpolation (SLERP) solves this by interpolating along the spherical arc between parameter vectors on a unit hypersphere:

$$\theta(t) = \frac{\sin((1 - t)\Omega)}{\sin \Omega} \theta_A + \frac{\sin(t\Omega)}{\sin \Omega} \theta_B$$

where $\Omega = \arccos\left(\frac{\theta_A \cdot \theta_B}{\|\theta_A\| \|\theta_B\|}\right)$ is the angle between the two weight vectors. SLERP preserves geometric norm and rotational dynamics, making it the standard approach for fusing two high-performing general checkpoints.

Resolving Parameter Interference: TIES-Merging

When merging three or more models, naive addition causes parameter interference: small noisy updates accumulate into significant perturbations, and opposing updates cancel out. To address this, Yadav et al. (2023) introduced TIES-Merging (TRIM, ELECT SIGN, and MERGE).

TIES executes a three-step protocol on the set of task vectors $\{\tau_1, \tau_2, \dots, \tau_N\}$:

1. Trim: For each task vector $\tau_t$, keep only the top-$k\%$ parameters with the largest absolute magnitude, setting the remaining $(100-k)\%$ to zero. This eliminates low-magnitude parameter drift accumulated during fine-tuning.
2. Elect Sign: For each parameter index $j$, compute the consensus sign across all trimmed task vectors based on total parameter mass:

$$\gamma_j = \text{sgn}\left(\sum_{t=1}^N \tau_{t,j}\right)$$

1. Disjoint Merge: For each parameter index $j$, average only the task vectors whose sign matches the elected consensus sign $\gamma_j$, ignoring opposing updates:

$$\tau_{\text{merged},j} = \frac{1}{|A_j|} \sum_{t \in A_j} \tau_{t,j} \quad \text{where} \quad A_j = \{t \mid \text{sgn}(\tau_{t,j}) = \gamma_j\}$$

The final model weights are formed by adding the scaled merged vector back to the base initialization:

$$\theta_{\text{TIES}} = \theta_0 + \lambda \tau_{\text{merged}}$$

By resolving directional conflicts and trimming parameter noise, TIES preserves distinct capabilities across specialized models.

Extreme Sparsification: DARE

Building on the observation that fine-tuning updates are highly redundant, Yu et al. (2023) introduced DARE (Drop And REscale). DARE demonstrates that up to 90% to 99% of delta parameters in fine-tuned LLMs can be dropped entirely without performance degradation.

DARE applies a random Bernoulli mask to each task vector and rescales the surviving parameters to preserve expected magnitude:

$$\tilde{\tau}_{t,j} = \frac{m_{t,j}}{1 - p} \tau_{t,j}, \quad m_{t,j} \sim \text{Bernoulli}(1 - p)$$

where $p \in [0.9, 0.99]$ is the drop probability.

Because extreme random dropout reduces parameter density to 1% to 10%, multiple DARE-processed task vectors can be merged via simple averaging or TIES (DARE-TIES) with near-zero coordinate collision. DARE provides an effective mechanism for combining large numbers of specialized checkpoints without mutual capacity cancellation.

Layer Splicing and Depth Upscaling

Beyond weight interpolation within identical architectures, model merging extends to structural manipulation:

• Passthrough / Frankenmerging: Assembling layers from different checkpoints or duplicating layers to expand parameter count. For example, splicing layers 0 to 24 of Model A and layers 16 to 32 of Model B.
• Solar Depth Upscaling (DUS): Introduced by Kim et al. (2023), DUS expands a 32-layer 7B model to a 48-layer 10.7B model by duplicating intermediate transformer blocks and applying continued pre-training to smooth transitional layer representations.

Practical Engineering and Tooling

Model merging is supported by open-source tooling, notably MergeKit (Goddard et al., 2024). MergeKit executes out-of-core matrix operations, allowing engineers to merge 70B parameter models on CPU memory without allocating GPU clusters.

Key trade-offs and operational realities include:

• Homology Requirement: Parameter-level merging (Task Arithmetic, SLERP, TIES, DARE) requires identical architecture, vocabulary size, and shared pre-trained initialization. Merging models from different base lineages (such as Llama 3 and Mistral) requires cross-architecture translation or layer concatenation.
• Evaluation Dilution: While merged checkpoints often achieve higher composite benchmark scores, specific edge capabilities can suffer from subtle alignment degradation. Thorough evaluation across target task distributions remains necessary.
• Zero Compute Cost: Model merging executes in minutes on standard workstations, democratizing multi-task model synthesis for teams without large-scale pre-training or fine-tuning infrastructure.

Sources

• Wortsman, M., et al. (2022). Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. arXiv:2203.05482
• Ilharco, G., et al. (2023). Editing Models with Task Arithmetic. arXiv:2212.04089
• Yadav, P., et al. (2023). TIES-Merging: Resolving Interference When Merging Models. arXiv:2306.01708
• Yu, L., et al. (2023). Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. arXiv:2311.03099
• Kim, D., et al. (2023). SOLAR 10.7B: Building Large Language Models with Up-Scaling. arXiv:2312.15166
• Goddard, C., et al. (2024). mergekit: Tools for Merging Pretrained Large Language Models. arXiv:2403.13257