llms.blog
explainers14 articles

explainers

Articles

  • Model Merging and Task Arithmetic: Mathematical Foundations, Task Vectors, TIES-Merging, DARE, and Spherical Linear Interpolation (SLERP)

    Model Merging and Task Arithmetic: Mathematical Foundations, Task Vectors, TIES-Merging, DARE, and Spherical Linear Interpolation (SLERP) Model merging enables the combination of multiple specialized large language models into a unified checkpoint without retraining, backpropagation, or access to the original training datasets. Rather than running costly multi-task supervised fine-tuning across disparate domains, practitioners can directly manipulate parameter tensors in weight space to fuse ca

    1 min

  • Kahneman-Tversky Optimization (KTO): Mathematical Foundations, Prospect Theory, and Binary Signal Alignment in Large Language Models

    Post-training alignment of large language models has long relied on pairwise comparison datasets. Methods such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) assume access to curated pairs $(x, y_w, y_l)$, where a human or automated judge explicitly marks completion $y_w$ as superior to $y_l$ for a given prompt $x$. In production environments, however, collecting paired preferences is logistically complex, expensive, and artificial. Real-world telem

    1 min

  • Curriculum Learning in Large Language Models: How Difficulty Pacing, Competence Progression, and Task Scheduling Shape Training Dynamics

    In standard large language model pre-training and fine-tuning pipelines, training batches are almost universally sampled uniformly and independently at random from a static corpus: $$\mathcal{D} = \{z_i = (x_i, y_i)\}_{i=1}^N$$ While this independent and identically distributed (i.i.d.) sampling paradigm aligns with empirical risk minimization (ERM), it ignores the non-convex geometry of deep transformer loss surfaces. Early in training, when network parameters are randomly initialized or unal

    1 min

  • Kolmogorov-Arnold Networks: How Learnable Edge Splines Challenge the Multi-Layer Perceptron

    Kolmogorov-Arnold Networks: How Learnable Edge Splines Challenge the Multi-Layer Perceptron For over three decades, the foundational building block of deep neural networks has remained essentially unchanged: the Multi-Layer Perceptron (MLP). In standard transformer feed-forward blocks, convolutional networks, and deep classifiers, MLPs alternate linear matrix multiplications ($\mathbf{W}\mathbf{x}$) with fixed, point-wise non-linear activation functions (such as ReLU, GELU, or SwiGLU). Under th

    1 min

  • Forward KL vs. Reverse KL Divergence: Mode Covering, Mode Seeking, and the Alignment Dynamics of Large Language Models

    Forward KL vs. Reverse KL Divergence: Mode Covering, Mode Seeking, and the Alignment Dynamics of Large Language Models Every phase of modern large language model development, from multi-trillion token pre-training to reinforcement learning from human feedback (RLHF) and student model distillation, fundamentally revolves around minimizing statistical distance between probability distributions. The primary mathematical tool utilized for this purpose is the Kullback-Leibler (KL) divergence, introd

    1 min

  • The Information Bottleneck in Deep Learning: How Mutual Information Compression Shapes Generalization and Neural Representations

    The Information Bottleneck in Deep Learning: How Mutual Information Compression Shapes Generalization and Neural Representations Classical statistical learning theory struggles to explain why overparameterized deep neural networks generalize well to unseen test data. Traditional complexity measures such as Vapnik-Chervonenkis (VC) dimension and Rademacher complexity scale with the raw count of tunable weights, predicting severe overfitting when parameter counts exceed dataset sample sizes. Yet

    1 min

  • Feed-Forward Networks as Key-Value Memories: How Transformers Store, Retrieve, and Edit Parametric Knowledge

    Feed-Forward Networks as Key-Value Memories: How Transformers Store, Retrieve, and Edit Parametric Knowledge In transformer language models, multi-head self-attention and feed-forward networks (FFNs) perform distinct computational roles. While self-attention mechanisms route contextual information dynamically across sequence positions, feed-forward layers account for roughly two-thirds of total model parameters in standard architectures. Despite early interpretations characterizing FFNs simply

    1 min

  • The Softmax Bottleneck in Large Language Models: Mathematical Foundations, Matrix Rank Limits, and Mixture of Softmaxes

    title: "The Softmax Bottleneck in Large Language Models: Mathematical Foundations, Matrix Rank Limits, and Mixture of Softmaxes" slug: "the-softmax-bottleneck-in-large-language-models-mathematical-foundations-matrix-rank-limits-and-mixture-of-softmaxes" status: "published" feature_image: "https://cms.llms.blog/content/images/2026/08/softmax-bottleneck-cover-1.png" excerpt: "A standard linear projection followed by Softmax caps the rank of predicted log-probability distributions to the hidden dim

    1 min

  • Reward Model Overoptimization in Large Language Models: How Goodhart's Law, Proxy Exploitation, and KL Drift Degrade Alignment

    Post-training alignment of large language models relies on optimizing a policy toward objectives defined by human intent and preferences. Because querying human evaluators during every step of continuous reinforcement learning or high-throughput rejection sampling is computationally and logistically infeasible, alignment workflows construct a parameterised proxy reward model. Trained on pairwise preference datasets through formulations such as the Bradley-Terry model, this proxy acts as a surrog

    1 min

  • Transformer Feed-Forward Networks as Key-Value Memories: How First-Layer Keys and Second-Layer Values Store Knowledge

    Transformer Feed-Forward Networks as Key-Value Memories: How First-Layer Keys and Second-Layer Values Store Knowledge In modern autoregressive Transformers, the division of labor between attention heads and multi-layer perceptron (MLP) blocks is often summarized through a clean functional split: attention routes information across sequence positions, while feed-forward networks (FFNs) process information per position. Yet for years, the exact mechanism by which FFNs process that information rem

    1 min

  • ReAct in Large Language Models: How Interleaving Reasoning and Action Traces Built the Foundation of AI Agents

    ReAct in Large Language Models: How Interleaving Reasoning and Action Traces Built the Foundation of AI Agents Before autonomous agents could interact reliably with APIs, search engines, and bash environments, large language models (LLMs) operated in one of two disconnected paradigms: internal reasoning without external interaction, or external action generation without internal deliberation. In pure reasoning paradigms such as Chain-of-Thought (CoT) prompting, models generate intermediate natu

    1 min

  • Data Filtering and Deduplication in Large Language Models: How Heuristics, MinHash LSH, SemDeDup, and Quality Classifiers Curate Pre-Training Corpora

    Data Filtering and Deduplication in Large Language Models: How Heuristics, MinHash LSH, SemDeDup, and Quality Classifiers Curate Pre-Training Corpora Pre-training data quality governs the downstream reasoning, factual accuracy, and sample efficiency of foundation large language models (LLMs). While early pre-training efforts relied on uncurated or lightly processed dumps from web repositories like Common Crawl, modern state-of-the-art models depend on multi-stage data curation pipelines that fi

    1 min

  • Process Reward Models: How Step-by-Step Supervision and Search Drive LLM Reasoning

    When large language models tackle complex multi-step reasoning (such as formal mathematics, algorithm synthesis, or multi-hop logic), evaluating only the final answer creates a severe credit assignment bottleneck. An outcome-based verifier can confirm whether a final numerical result is correct, but it cannot determine whether the underlying derivation was logically sound or reached the right answer through compounding hallucinations and lucky cancellations. Process Reward Models (PRMs) resolve

    1 min

  • What Is a KV Cache? The Memory Bottleneck Behind LLM Serving

    What Is a KV Cache? The Memory Bottleneck Behind LLM Serving Every token a large language model generates costs more than compute. It costs memory. The mechanism that stores the model's working state across a generation is called the KV cache, and managing it well is what separates a serving system that feels fast from one that stalls. This explainer covers what the KV cache is, why it wastes memory, and how the PagedAttention algorithm from Berkeley's vLLM project reshaped how it is handled.

    1 min