OpenAI's chief scientist published the company's internal pretraining loss numbers on November 22 and, in the same post, asked frontier labs to agree on development limits tied to that same metric. The proposal gives the industry its first concrete candidate for a shared speedometer. It also makes OpenAI the first lab to disclose where its own models sit on it.
In a blog post titled "Perplexity and Intelligence," Jakub Pachocki argues that perplexity, the standard measure of how well a language model predicts held-out text, is the metric worth pacing against. His framing is that modern AI systems are compressors trained on large corpora, and that most capability techniques, from prompting to fine-tuning to reinforcement learning, are elicitation layers sitting on top of that one substrate. OpenAI has tracked log-perplexity on an internal held-out dataset since GPT-3.5 and is now publishing the curve.

The thresholds Pachocki cites
The post ties specific capability jumps to specific loss levels, a retrospective mapping rather than a predictive table:
- GPT-2, which set a then-new state of the art on older datasets, was arguably the first model to consistently write coherent paragraphs.
- GPT-3 could learn in context from a few examples.
- GPT-3.5, at roughly 0.99 log-perplexity on OpenAI's main evaluation dataset, supported fluent conversation and became the basis of the original ChatGPT.
- GPT-4, at about 0.91, got good enough at chain-of-thought reasoning to motivate the modern class of reasoning models.
- Below 0.8, OpenAI says it has seen "transformative capabilities" in fields like cybersecurity and mathematics.
The sharpest anecdote concerns math. Pachocki writes that OpenAI has posed open problems to its models as an evaluation for over a year, and that once the company "upgraded to a model of log-perplexity ~0.76, it started producing groundbreaking results," which he describes as long-considered questions resolving "one after another in a single Slack thread."
From there comes the 2027 warning. If AI can produce step changes in mathematical research, Pachocki argues, a similar avalanche in deep learning research itself is plausible, specifically algorithmic progress on pretraining. He extrapolates that transformative capabilities should be expected again in the 0.68 to 0.70 log-perplexity window, and notes that the steep 2026 acceleration on OpenAI's own curve is attributed mostly to long-term research and compute investments, with "only in a smaller part" to automation, though the most recent models have already received meaningful design contributions from AI.
What the limits proposal asks for

Pachocki proposes three things: shared norms for evaluating and red-teaming models as they cross thresholds, common transparency standards, and "common limits, based on perplexity and other measurements, that developers are committed to not crossing until our ability to understand and monitor AI behavior achieves a certain concrete milestone." He offers one example milestone: a successor to chain-of-thought monitors. The post links to the shared safety measures statement that OpenAI employees backed earlier this year, and the publication of the loss curve is framed as a step toward a common scientific basis for pacing.
He also lists the metric's weaknesses himself: numbers are not comparable across labs because evaluation datasets and training mixtures differ; perplexity is detached from any task anyone actually cares about; it misses advances outside pretraining, since chain-of-thought raises capabilities without lowering perplexity; and the floor is unknown, because nobody knows the inherent entropy of the data, meaning that near the floor, small perplexity gains may correspond to large intelligence leaps.
Early reactions
Sam Altman amplified the post on X, and the Hacker News thread filled within hours. Researchers engaged with the substance, not the diplomacy. Elie Bakouch of Prime Intellect called raw perplexity "a very good measure for progress in pretraining" and said he had been pushing for exactly this kind of reporting, while noting a limitation Pachocki does not flag: perplexity numbers depend heavily on the compute spent, so efficiency gains would be a cleaner metric. Other respondents raised the Goodhart objection, that tying a formal limit to a single number gives labs an incentive to game the number rather than the capability, and asked what makes the 0.68 to 0.70 window special if the underlying capability thresholds have historically been identified only after the fact.
One editorial observation, clearly labeled as ours: the disclosure is real but narrow. The numbers are measured on OpenAI's internal dataset, which the company says it chose because those are the figures it relies on most. Until other labs publish curves of their own, or the community builds transferable comparisons, the cross-lab column of the proposal is empty. A speedometer that only one driver has installed is, for now, mostly a statement about that driver.
Sources
- Jakub Pachocki, "Perplexity and Intelligence," OpenAI, November 22, 2026: https://openai.com/index/perplexity-and-intelligence
- Techmeme coverage: https://www.techmeme.com/261122/p9
- Hacker News discussion: https://news.ycombinator.com/item?id=50601525
- Elie Bakouch on X: https://x.com/eliebakouch/status/2124566777740882027
