Mechanistic Interpretability10 articles

Mechanistic Interpretability

Articles

Sparse Autoencoders (SAEs) and Mechanistic Interpretability: Mathematical Foundations, Dictionary Learning, Top-K Sparsity, Feature Steering, and Monosemanticity
Sparse Autoencoders (SAEs) and Mechanistic Interpretability: Mathematical Foundations, Dictionary Learning, Top-K Sparsity, Feature Steering, and Monosemanticity Modern autoregressive large language models represent a vast catalog of world concepts, syntactic rules, and abstract reasoning heuristics. However, inspecting the raw weight matrices and internal activation states of transformer networks reveals an obstinate barrier to mechanistic interpretability: individual neurons are notoriously p
August 29, 20261 min
Activation Addition and Representation Engineering: Mathematical Foundations, Linear Subspace Projections, and Inference-Time Steering in Large Language Models
Large language models encode vast linguistic, factual, and behavioral properties within their internal hidden representations. While traditional alignment and behavioral modification rely on Supervised Fine-Tuning (SFT) or Reinforcement Learning from Human Feedback (RLHF), these gradient-based techniques modify billions of parameter weights, require substantial compute, and frequently suffer from catastrophic forgetting or alignment tax. An alternative paradigm grounded in mechanistic interpret
August 26, 20261 min
Induction Heads: Mathematical Foundations, Two-Layer Circuit Composition, and the Emergence of In-Context Learning in Transformers
Induction Heads: Mathematical Foundations, Two-Layer Circuit Composition, and the Emergence of In-Context Learning in Transformers One of the defining capabilities of modern autoregressive large language models is in-context learning: the ability to infer rules, adapt to task formats, and execute complex few-shot instructions purely from prompt context without updating neural network weights. For years, in-context learning was treated as an enigmatic, emergent property of large-scale autoregres
August 25, 20261 min
Superposition and Sparse Autoencoders: Mathematical Foundations, the Polysemanticity Bottleneck, and Dictionary Learning in Language Models
For years, attempts to interpret transformer neural networks by inspecting individual neurons encountered an obstinate barrier: polysemanticity. A single neuron in an intermediate multi-layer perceptron (MLP) or residual stream layer rarely corresponds to a single human-interpretable concept. Instead, the same neuron frequently fires on a disparate mixture of inputs, such as Python syntax errors, discussions of Renaissance art, and Spanish verbs. This phenomenon prevents mechanistic interpretabi
August 25, 20261 min
The QK and OV Circuits in Transformers: How Bilinear Attention Routing and Subspace Projections Move Information
The QK and OV Circuits in Transformers: How Bilinear Attention Routing and Subspace Projections Move Information The standard mathematical presentation of multi-head self-attention, introduced in Vaswani et al. (2017), describes the layer as a sequence of matrix projections followed by scaled dot-product operations, concatenation, and an output projection. While computationally efficient for parallel GPU hardware, this formulation obscures the fundamental linear mechanics governing how transfor
August 23, 20261 min
Feed-Forward Networks as Key-Value Memories: How Transformers Store, Retrieve, and Edit Parametric Knowledge
Feed-Forward Networks as Key-Value Memories: How Transformers Store, Retrieve, and Edit Parametric Knowledge In transformer language models, multi-head self-attention and feed-forward networks (FFNs) perform distinct computational roles. While self-attention mechanisms route contextual information dynamically across sequence positions, feed-forward layers account for roughly two-thirds of total model parameters in standard architectures. Despite early interpretations characterizing FFNs simply
August 22, 20261 min
Superposition and Polysemanticity in Neural Networks: How High-Dimensional Geometry and Sparsity Allow LLMs to Store More Features Than Dimensions
Superposition and Polysemanticity in Neural Networks: How High-Dimensional Geometry and Sparsity Allow LLMs to Store More Features Than Dimensions In neural network interpretability, one of the most persistent puzzles has been the phenomenon of polysemantic neurons: individual neurons in large language models that activate for seemingly unrelated concepts. A single hidden unit in an MLP layer might fire in response to Python syntax errors, academic citations, and geographic place names. For ye
August 21, 20261 min
The Linear Representation Hypothesis in Large Language Models: How Neural Geometry Organizes Concepts, Latent Truth, and Steerable Features
The Linear Representation Hypothesis in Large Language Models: How Neural Geometry Organizes Concepts, Latent Truth, and Steerable Features Deep neural networks are composed of alternating non-linear activation functions, multi-head attention operations, and high-dimensional matrix projections. Despite this architectural non-linearity, empirical research across mechanistic interpretability reveals a striking geometric regularity: within the intermediate representation spaces of large language m
August 21, 20261 min
Activation Patching and Circuit Discovery in Large Language Models: How Causal Mediation Maps Transformer Subgraphs
Understanding how large language models perform complex reasoning requires moving beyond passive behavioral evaluation. While behavioral benchmarks measure model outputs on specific datasets, they treat the underlying neural network as an inscrutable black box. Mechanistic interpretability aims to reverse-engineer transformer weights and intermediate representations into human-understandable algorithms and computational graphs. At the core of modern mechanistic interpretability is causal mediat
August 21, 20261 min
The Logit Lens: How Direct Unembedding Decodes Intermediate Representations in Large Language Models
Autoregressive large language models process input tokens through deep stacks of transformer blocks, generating a final probability distribution over the vocabulary at the final layer. For years, the intermediate computations occurring within these hidden layers were treated as opaque black boxes. Mechanistic interpretability research has demonstrated that modern transformer architectures operate through a structured, iterative refinement process across their residual stream. The logit lens and
August 19, 20261 min