Model Merging in Large Language Models: How Task Arithmetic, TIES, and DARE Combine Checkpoints Without Training
Fine-tuning foundation models for specialized tasks typically produces isolated checkpoints. A model adapted for mathematical reasoning retains high numerical precision but often degrades in general dialogue or code generation. Traditionally, unifying these capabilities required multi-task training: gathering mixed datasets, re-running optimization across multiple GPUs, and managing gradient conflicts during backpropagation. Model merging provides an alternative paradigm. By operating directly
1 min