Subscribe
Opinion
4 min read

Opinion: Anthropic's AI R&D Alarm Rests on a Forecast

Anthropic's November 10 S-1 amendment says it met its AI R&D safety threshold based on what it expects from future models, not what it has seen in current ones. My view: a forecast trigger is the right design, but the promised reports must show the evidence.

Olivetti-style cover reading Anthropic AI R&D Alarm on a Forecast, Opinion, beside a charcoal panel with three rising teal, mustard and red bars on cream paper
Olivetti-style cover reading Anthropic AI R&D Alarm on a Forecast, Opinion, beside a charcoal panel with three rising teal, mustard and red bars on cream paper

This is an opinion piece. The facts come from Anthropic's SEC filings and its Responsible Scaling Policy, linked below. The argument is mine.

Anthropic now says it has met the AI research and development threshold in its own safety rulebook. The line sits in the risk factors of Amendment No. 2 to its Form S-1, filed with the SEC on November 10, 2026. Our news desk covered the filing in Anthropic Says It Met Its AI R&D Safety Threshold.

The easy headline is "AI is now improving itself." The filing does not say that. It says Anthropic made the call "based on what we expect from future models and our overall rate of progress, rather than on what we have observed from current models."

My view: triggering a safety rule on a forecast is the right design. But a forecast that only Anthropic can see is not yet evidence for anyone else. The test of this decision is the report that comes next, not the sentence in the S-1.

What the filing actually says

The passage is short. Anthropic says "our use of AI systems has already been accelerating our research and development" and that its "internal estimates indicate that we may experience a further increase in the rate of progress in AI capabilities."

The filing explains that the threshold "applies where we “observe or expect” a specified degree of accelerated progress." On that basis, it says "we made a determination that we have met this threshold." It then adds: "We continue to assess the underlying data."

Eight days earlier, Amendment No. 1 used softer words. It said Anthropic "may imminently meet" the definition, and "We have not designated any of our current models as exceeding this threshold." Both versions rest on expectations about future models. The change is in the verdict, not in the kind of evidence.

Two-column panel comparing what the November 10 S-1 amendment says Anthropic based its call on (what it expects from future models and its overall rate of progress) with what it says it did not base the call on (what it has observed from current models)

Why a forecast trigger is the right call

The policy was written this way on purpose. In RSP version 3.4, the second route to this threshold covers cases where "we observe or expect" double the rate of progress in aggregate AI capabilities. If you only act on what you have already seen, a safety rule about fast progress fires too late by design.

The filing makes the same point against itself, and I think this is the most underrated line in it. It says "much of our evidence regarding the pace of such AI-driven acceleration is derived from lagging indicators," and that the company has "in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred."

That is a lab saying its own speedometer reads late. Given that admission, waiting for proof would mean acting after the fact. Pulling the trigger early is the consistent choice. Other labs with similar rules should take note: a threshold you can only confirm afterward is a threshold you will always miss.

Why "met" still needs evidence

The bar here is high, and that is why the word matters. The RSP defines "double the rate of progress" as "as much progress in one year as one would see in two years at baseline." Its own example: if baseline progress were a 3x compute scaleup and a 3x gain in algorithmic efficiency, a 9x "effective scaleup," then double the rate "would entail something like an 81× effective scaleup."

The policy also says this "is not the same idea as 'doubling researchers' productivity,' since doubling inputs does not necessarily double the rate of progress." The test is the rate of progress in aggregate AI capabilities, compared with both the rate Anthropic would expect and the fastest extended rate it has seen without significant AI help, over at least three model generations.

Anthropic is candid that this is hard to measure. The RSP changelog says the threshold "is intended to capture the onset of dramatic recursive self-improvement, and has proven difficult to operationalize." For more on how these levels work, see our Responsible Scaling Policy explainer.

Put those together and you get a high bar, measured with lagging data, met on internal estimates that have not been published. I do not think that makes the call wrong. I think it makes the call impossible to check today. Outside readers are asked to trust the forecast, the same way they are asked to trust a model's stated reasoning, which is a clue, not a confession.

Who wins, and what to watch

If the follow-up is real, safety-minded readers win twice. They get a lab that acts on a forecast, and a public record of what the forecast was. If the follow-up is thin, the main effect of the sentence is to tell investors about a risk.

The filing gives a few hints about what comes next. Anthropic says it "recently chose to delay pre-training a new frontier base model for more than a month" to improve security against model weight theft. It says it is considering RSP updates "to require human review and approval of certain decisions in our research and development processes as our models perform an increasing share of that work." And it says it does "not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding."

Those are reasonable steps. The filing says the company "will release further reports in the future." When it does, I will judge the decision on three things:

  • The data. Which estimates drove the call, and against what baseline.
  • The safeguards. Which new controls now apply, and to which work.
  • The human check. Whether "human review and approval" becomes a written rule, and what it covers.
Checklist panel of three things the promised Anthropic reports should show: the data behind the call, the new safeguards, and the human review rule

Anthropic made a hard call early, in public, in a legal document. That deserves credit. Now it owes the evidence.

Sources

OpinionAnthropicAI Safety