Sparse Autoencoders in Large Language Models: How Dictionary Learning Unpacks Superposition and Neural Monosemanticity
Deep neural networks have long been treated as uninterpretable black boxes. In transformer language models, individual neurons in the residual stream and multilayer perceptron (MLP) layers rarely map to singular, human-understandable concepts. Instead, individual neurons exhibit polysemanticity: a single neuron might fire for Python syntax, medical terminology, and Korean dialogue without an obvious shared semantic foundation. Mechanistic interpretability research explains this phenomenon throu
1 min