Most of the time, a neural network learns the way the loss curves in a tutorial suggest: training accuracy climbs, test accuracy follows a beat or two behind, and the gap between them never gets dramatic. Grokking breaks that script. A model drives its training error to zero, sits at chance-level accuracy on held-out data for what can be thousands of additional steps, and then abruptly jumps to near-perfect generalization. The training loss says the model is done. The test set says it has not started. Then, without any change to the data or the hyperparameters, it generalizes.
The term comes from Robert Heinlein's 1961 novel Stranger in a Strange Land, where "to grok" means to understand something so deeply that the knower and the known merge. The research community borrowed it in 2022, and it stuck because there was no better word for a model that appears to understand a rule thousands of steps after it appears to have finished learning the examples.

The paper that named it
Grokking was first documented and named by Alethea Power and colleagues at OpenAI in a January 2022 paper, "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets." The setup was deliberately minimal. Think of a multiplication-style table for a mathematical operation, with most entries blanked out: the model sees filled cells and must predict the rest. In the headline experiment, the operation was modular addition, where a plus b is computed modulo a prime like 97, and every mathematical element was replaced with an unrelated abstract symbol. The network could not read any numerical structure off its inputs. It had to infer the rule from how the symbols interact.
Small transformers trained on this task memorized their training pairs quickly. Then, after continued training with strong weight decay, test accuracy improved from random chance level to perfect generalization, and that improvement came well past the point of overfitting. The authors also found that smaller training sets required increasing amounts of optimization before generalization arrived: the less data, the longer the model sat in its memorization plateau.
Two details from the original recipe shaped every follow-up study. Grokking depended on a training regime, not just an architecture: full-batch gradient descent with a weight decay strong enough to matter (values around 1.0 are commonly reported) made the difference. And the phenomenon lived in a Goldilocks band of data size: enough examples that a generalizing solution exists, but not so many that the network generalizes immediately.
What the network is doing while you watch the plateau
The most concrete answer came from mechanistic interpretability. In 2023, Neel Nanda and colleagues reverse-engineered a small transformer that had grokked on modular addition. What they found was, by the standards of neural networks, almost embarrassing in its elegance: the model implements the task with discrete Fourier transforms and trigonometric identities, converting addition into rotation about a circle. To add a and b modulo 97, it maps each number to a rotation on a clock face, composes the rotations, and reads off the answer by rotating back. The network invents its own clock arithmetic without ever being told what modulo means.
Crucially, the authors showed that grokking is not a true discontinuity.

Decomposing the model into circuits and measuring each circuit's progress over training split the process into three continuous phases: memorization, circuit formation, and cleanup. The structured Fourier circuit is built gradually, under the hood, while test accuracy is still flat. The visible "sudden" jump happens later, when the model removes the memorization lookup components that were masking the circuit's contribution. As the authors put it, grokking arises from the gradual amplification of structured mechanisms followed by the removal of memorizing parts. The plateau was never a plateau. It was construction.
A study by Ziming Liu and colleagues, reported in a 2024 Quanta Magazine account, found the same flavor of solution across many runs. About 40% of the algorithms the networks discovered were variants of "clock" or "pizza slicing" geometry, solutions humans can interpret. For the rest, the researchers could not yet say what the networks were computing.
An artifact of the optimizer, or a law of deep learning?
Whether grokking is a quirky product of weight decay or something deeper became a live debate, and the honest summary is that several mechanisms have real evidence behind them.
One camp explains grokking through regularization and norm pressure. The Omnigrok paper (ICLR 2023) showed the phenomenon extends beyond algorithmic data, appearing in image classification when networks start with large-norm weights and receive weak regularization, and disappearing entirely once the dataset grows past a critical size. Follow-up work reported similar dynamics for weight decay acting as compression: the memorizing solution fits the data with larger weight norms, and prolonged decay pushes the network toward smaller, more general minimum-norm solutions.
Another camp disputes the norm story. A 2024 ICLR paper by Henry Lyth and Orion King recast grokking as the transition from "lazy" training dynamics, where the network stays near its initialization, to "rich" dynamics, where learned features actually move. In their examples, grokking occurred with weight norms increasing, not decreasing, which plain norm-based accounts cannot explain.
Statistical physicists added their own readings: one 2023 line of work modeled grokking as a first-order phase transition in sample complexity, and a NeurIPS 2025 paper proposed treating training as glass relaxation, where memorization is rapid cooling into a disordered, jammed state and generalization is the slow annealing toward a stable configuration. A 2026 ACM Computing Surveys review by Francesco Bertolotti and Walter Cazzola consolidated the field, identifying eleven partially overlapping mechanisms, each with experimental support. There is no settled single explanation, and posts claiming otherwise are selling something.
Does grokking happen in real language models?
This is the question practitioners actually care about, and the evidence has shifted from "toy task only" toward "probably yes, but it looks different."
Humayun, Balestriero and Baraniuk argued in 2024 that the phenomenon is widespread rather than exotic. They demonstrated delayed generalization in ordinary settings, a convolutional network on CIFAR-10 and a ResNet on Imagenette, and introduced "delayed robustness": networks also grok adversarial examples, becoming robust to attacks long after they interpolate the data.
The first study of grokking in practical LLM pretraining appeared in work accepted at ICLR 2026. Ziyue Li, Chenrui Fan and Tianyi Zhou trained mixture-of-experts language models for one epoch on a cross-domain corpus and watched memorization and downstream generalization separately, across math, commonsense, code and retrieval benchmarks. Their finding: a memorization-to-generalization lag persists in pretraining. Different data domains enter their generalization phases asynchronously, and expert-routing patterns in the model keep reorganizing after the pretraining loss has already converged, shifting from random, instance-specific paths toward structured, transferable ones. Their blunt framing of the practical implication: training loss is an unreliable proxy for generalization, diverging from it in both timing and scale.
No one has yet shown that a frontier lab left a model running two epochs longer and watched a capability wake up. The lag is measured on benchmarks, not announced in launch blogs. But the mechanism that made the original modular-arithmetic transformer click, continued reorganization of internal structure after the loss curve flattens, now has evidence at language-model scale.
Why you should care if you train or fine-tune anything
Grokking reorganizes three routine assumptions.
Early stopping is a bet. If validation performance plateaus while training loss is zero, the standard playbook says stop, you are overfitting. In grokking regimes, that plateau is where circuit formation happens. Stopping there means shipping the lookup table, not the clock.
Training loss is not progress. A converged loss curve says the model has memorized. It says nothing about whether its internal representations have become transferable. The ICLR 2026 work proposes cheap internal metrics, like routing stability across layers, as substitutes for test-time evaluation.
Held-out evaluation is the only honest instrument. In a phenomenon defined by the gap between train and test performance, anything you believe about a model's capability that is not measured on data it never touched is a guess. The same logic extends to distillation and small-data fine-tuning, where delayed generalization during student training has been observed anecdotally in the grokking literature but remains hard to predict.
The open problem, stated plainly in the survey literature: predicting in advance whether a given run will grok, and when. Today that answer arrives on the validation curve, which is precisely the problem.
Sources
- Power, Burda, Edwards, Babuschkin, Misra. "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets." arXiv:2201.02177, January 2022. https://arxiv.org/abs/2201.02177
- Nanda, Chan, Lieberum, Smith, Steinhardt. "Progress Measures for Grokking via Mechanistic Interpretability." ICLR 2023. arXiv:2301.05217. https://arxiv.org/abs/2301.05217
- Liu, Kitouni, Nolte, Michaud, Tegmark, Williams. "Towards Understanding Grokking." arXiv:2205.10343, 2022. https://arxiv.org/abs/2205.10343
- Nandakumar, Halas, Sanghavi, Navlakha. "Omnigrok: Grokking Beyond Algorithmic Data." ICLR 2023. arXiv:2210.01117. https://arxiv.org/abs/2210.01117
- Quanta Magazine. "How Do Machines 'Grok' Data?" April 12, 2024. https://www.quantamagazine.org/how-do-machines-grok-data-20240412
- Lyth, King. "Grokking as the Transition from Lazy to Rich Training Dynamics." ICLR 2024. https://proceedings.iclr.cc/paper_files/paper/2024/file/63ed15a46a143ff57484b38cd6b85d91-Paper-Conference.pdf
- Humayun, Balestriero, Baraniuk. "Deep Networks Always Grok and Here is Why." arXiv:2402.15555, 2024. https://arxiv.org/abs/2402.15555
- Li, Fan, Zhou. "Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Test." ICLR 2026. arXiv:2506.21551. https://arxiv.org/abs/2506.21551
- Bertolotti, Cazzola. "A Survey on Grokking." ACM Computing Surveys 58(13), 2026. https://doi.org/10.1145/3814603
- "Is Grokking a Computational Glass Relaxation?" NeurIPS 2025 poster. https://neurips.cc/virtual/2025/poster/117824
- Wikipedia. "Grokking (machine learning)." https://en.wikipedia.org/wiki/Grokking_(machine_learning)
