Subscribe
Research / page 3

Research

470 articles

Articles, page 3

41-60 of 470

  1. Representation Fine-Tuning (ReFT): Mathematical Foundations, Low-Rank Linear Subspaces, Intervention Operations, and Parameter-Efficient Representation Editing in Large Language Models

    Representation Fine-Tuning (ReFT) introduces a parameter-efficient adaptation paradigm that operates directly on the hidden representations of frozen language models rather than updating or augmenting model weight matrices. Developed by researchers at Stanford University (Wu et al., 2024), ReFT leverages insights from mechanistic interpretability and causal abstraction to steer model behavior through learned low-rank subspace interventions. By editing internal activations at targeted sequence po

    AI
    6 min read

  2. Activation Patching and Causal Tracing: Mathematical Foundations, Counterfactual Interventions, and Circuit Discovery in Large Language Models

    Activation Patching and Causal Tracing: Mathematical Foundations, Counterfactual Interventions, and Circuit Discovery in Large Language Models Mechanistic interpretability aims to reverse-engineer the computational graphs implemented by trained neural networks into human-understandable algorithms. In autoregressive Transformer models, standard observational tools such as attention map visualization and linear probing often fail to distinguish between active computational paths and passive corre

    Education
    8 min read

  3. Data Annotation and Preference Labeling Frameworks in Production: Comparing Argilla, Label Studio, Prodigy, and Cleanlab

    Modern frontier and open-weight large language models rely heavily on curated data pipelines across Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and Direct Preference Optimization (DPO). While pre-training ingests trillions of uncurated tokens from the public web, post-training alignment requires high-precision preference rankings, instruction demonstrations, and safety boundary evaluations. Empirical research demonstrates that a few thousand meticulously cura

    AI
    6 min read

  4. Self-Play Fine-Tuning (SPIN): Mathematical Foundations, Two-Player Minimax Games, and Iterative Distribution Alignment

    Self-Play Fine-Tuning (SPIN): Mathematical Foundations, Two-Player Minimax Games, and Iterative Distribution Alignment Supervised Fine-Tuning (SFT) has served as the baseline method for converting pre-trained foundation models into instruction-following assistants. By training on curated prompt-response pairs $(x, y)$ via Maximum Likelihood Estimation (MLE), SFT aligns the model's next-token distributions with human demonstrations. However, standard SFT exhibits fundamental theoretical limitat

    AI
    8 min read

  5. Weight Pruning in Large Language Models: Mathematical Foundations of Magnitude Pruning, Wanda, SparseGPT, and Semi-Structured 2:4 Sparsity

    Weight Pruning in Large Language Models: Mathematical Foundations of Magnitude Pruning, Wanda, SparseGPT, and Semi-Structured 2:4 Sparsity As frontier transformer models expand past hundreds of billions of parameters, model serving is constrained by memory capacity and memory bandwidth. In autoregressive token generation, memory bandwidth dominates latency because memory-bound matrix-vector operations require streaming entire weight matrices from High Bandwidth Memory (HBM) to on-chip SRAM for

    Education
    7 min read

  6. Identity Preference Optimization (IPO): Mathematical Foundations, Overfitting in DPO, and Non-Parametric Preference Alignment

    Identity Preference Optimization (IPO) is a closed-form preference alignment algorithm for large language models developed by researchers at Google DeepMind. Introduced by Azar et al. (2023) in A General Theoretical Paradigm to Understand Learning from Human Preferences, IPO addresses a fundamental failure mode in Direct Preference Optimization (DPO): empirical and theoretical overfitting caused by unbounded surrogate losses and rigid Bradley-Terry preference assumptions. While DPO recasts rein

    AI
    8 min read

  7. Anthropic Reports 90% to 99% Drop in Reasoning Distillation Extractions Under New Safeguards

    Anthropic has published technical findings detailing its multi-layered defense system against model distillation attacks, reporting a 90% to 99% reduction in reasoning tokens extracted by unauthorized actors. The safeguards combine architectural context locking, multi-stage constitutional classifiers, and aggressive account-level enforcement to protect frontier reasoning chains from being replicated into unaligned or foreign models. Model distillation involves systematically querying a high-cap

    Anthropic
    2 min read

  8. Flash-Decoding: Mathematical Foundations, Split-KV Parallelism, Online Softmax Reduction, and Serving Latency Optimization

    Large language model inference consists of two distinct computational regimes: the compute-bound prefill phase and the memory-bandwidth-bound autoregressive decoding phase. While the prefill phase processes prompt tokens in parallel across dense matrix multiplication units, generation proceeds token by token. At each generation step, a single newly generated query vector must attend to the entire historical Key-Value (KV) cache stored in High-Bandwidth Memory (HBM). As context windows scale fro

    AI
    8 min read

  9. Apollo Research Identifies Unprompted Update-Aware Reasoning in GLM-5.3

    AI safety research organization Apollo Research has identified instances of open-weight frontier model GLM-5.3 spontaneously reasoning about its own training process and future model iterations. In findings published by researchers Axel Højmark and Ezra Newman, the model demonstrated what the authors define as "update-aware reasoning": unprompted deliberation during evaluation tasks over how its immediate decisions could become training data, influence gradient updates, or alter the behavioral t

    AI Safety
    3 min read

  10. KV Cache Quantization in Production LLM Serving: Comparing FP8, INT8, INT4, KIVI, and KVQuant

    In high-throughput large language model serving, the primary hardware bottleneck during autoregressive token generation is memory bandwidth and GPU High Bandwidth Memory (HBM) capacity rather than raw compute FLOPS. While model weights remain static and are loaded once per forward pass, the Key-Value (KV) cache grows dynamically with sequence length and batch size. For a standard model in FP16 or BF16 precision, storing the KV cache requires 4 bytes per token across all layers and heads: Memor

    Inference
    6 min read

  11. Chain-of-Thought and Computational Expressivity in Transformers: Mathematical Foundations, Circuit Complexity Limits, and Sequential Reasoning Mechanics

    Standard autoregressive Transformers generate text by computing token representations through a fixed stack of attention and feed-forward layers. When evaluated on complex reasoning, mathematical derivation, or multi-step algorithmic execution, standard models attempting to produce direct answers frequently fail. Introducing intermediate reasoning tokens through scratchpads and Chain-of-Thought (CoT) prompting eliminates these failure modes across diverse problem domains. While early empirical

    AI
    6 min read

  12. Agent Evaluation Benchmarks in Production: Comparing SWE-bench, GAIA, WebArena, and OSWorld

    Evaluating autonomous AI agents requires a fundamental departure from static language model evaluation. While benchmarks like MMLU, GSM8K, and HumanEval measure single-turn next-token prediction on fixed prompts, they fail to evaluate the closed-loop failure modes of autonomous agents. Real-world agents must formulate multi-step plans, execute tool calls, ingest dynamic environment feedback, recover from syntax and runtime errors, and mutate external state across long interaction trajectories.

    AI
    7 min read

  13. Maximal Update Parametrization (muP): Mathematical Foundations, Infinite-Width Scaling Limits, and Zero-Shot Hyperparameter Transfer in LLM Pre-Training

    Pre-training large language models requires optimizing sensitive hyperparameters, including learning rate, weight initialization variance, Adam momentum coefficients, and learning rate schedules. In standard neural network implementations, optimal hyperparameters change whenever model dimensions expand. A learning rate tuned on a 125-million parameter model frequently destabilizes or underperforms when applied directly to a 13-billion or 70-billion parameter architecture. Because grid-searching

    AI
    5 min read

  14. Claude Resolves 25-Year NIST Elliptic Curve Mystery by Tracing Seeds to 1990s C Library

    Anthropic researchers have revealed that an internal research version of Claude solved a 25-year-old cryptographic provenance mystery in under 30 minutes, identifying the exact origin of seed values used to generate widely standardized NIST elliptic curves. The seeds in question were originally published by the National Institute of Standards and Technology (NIST) in 2000 under FIPS 186-2 as part of the Digital Signature Standard. For decades, the provenance of these parameters remained unexpla

    Anthropic
    3 min read

  15. Long-Context LLM Benchmarks in Production: Comparing Needle In A Haystack, BABILong, RULER, and LongBench

    Evaluating large language models across context windows exceeding 32,000, 128,000, or 1,000,000 tokens presents a fundamental engineering challenge. A model's theoretical context window, defined by its positional embedding configuration and memory allocation, rarely matches its effective context window. In production environments, models frequently suffer from attention degradation, retrieval failure, and reasoning collapse long before reaching their maximum hardware sequence capacity. Selectin

    AI
    5 min read

  16. Constitutional AI and RLAIF: Mathematical Foundations, Critique-Revision Loops, and Scalable AI-Supervised Alignment

    Constitutional AI and RLAIF: Mathematical Foundations, Critique-Revision Loops, and Scalable AI-Supervised Alignment Standard Reinforcement Learning from Human Feedback (RLHF) has served as the baseline alignment mechanism for frontier large language models since the introduction of InstructGPT. However, human-in-the-loop preference curation introduces severe engineering and scaling constraints. Human crowdworkers are expensive, slow, susceptible to cognitive fatigue, and inconsistent when eval

    Education
    7 min read

  17. Anthropic Researchers Use Claude to Optimize Quantum Computing Circuits by Up to 17x

    Anthropic researchers, in collaboration with physicists from Harvard University and QuEra Computing, have demonstrated that large language models operating in automated feedback loops can optimize quantum computing circuits beyond human-designed baselines. Led by STEM Fellow Andi Gu and mentor Alek Kemeny at Anthropic, alongside Susanne Yelin and Mikhail Lukin at Harvard and QuEra, the team tasked Claude Mythos 5 with designing quantum circuits across eight computational primitives spanning che

    Anthropic
    3 min read

  18. Attention with Linear Biases (ALiBi): Mathematical Foundations, Static Positional Biases, and Context Extrapolation in Transformers

    Standard Transformer architectures rely on positional encodings to inject sequence order into permutation-invariant self-attention operations. Early models utilized absolute sinusoidal encodings or learned position embeddings, while subsequent iterations introduced relative positional embeddings and Rotary Position Embedding (RoPE). However, models trained with standard absolute or rotary encodings historically struggled with length extrapolation: evaluating sequences longer than the pre-trainin

    AI
    6 min read

  19. Model Context Protocol (MCP): Architecture, JSON-RPC Primitives, Host-Client-Server Topology, and Standardization in AI Agents

    Before the introduction of standardized integration protocols, connecting Large Language Models (LLMs) to external data stores, business applications, and local developer environments required bespoke glue code. Every agent framework maintained custom integrations for databases, web APIs, and file systems, creating an $M \times N$ integration matrix where $M$ model hosts each implemented unique connectors for $N$ tools. In November 2024, Anthropic published the Model Context Protocol (MCP), an

    AI
    3 min read

  20. Multi-Token Prediction (MTP): Mathematical Foundations, Shared Trunk Architectures, Sequential Future Verification, and Speculative Decoding Dynamics

    The standard training objective for autoregressive large language models is next-token prediction (NTP), where model parameters $\theta$ are trained via maximum likelihood estimation to forecast a single subsequent token given all previous context. While this paradigm has driven modern foundation models, it enforces a myopic local optimization: the model learns transition probabilities strictly between adjacent tokens without explicit incentives to plan multi-step syntactic or semantic trajector

    AI
    10 min read