Google announced Gemini 4 Carbon on October 28, 2026, and the second paragraph of its launch post opens with a ranking: "Carbon ranks #1 on the recently updated v5.0 of Artificial Analysis Intelligence Index at 59." The same post says Carbon goes first to "a subset of cyber defenders," and only later to developers, businesses and consumers.
So the day's lead claim was a #1 ranking for a model the public cannot use yet, scored on an index that had started, the day before, to base 60% of its score on private test sets or held-out answers. Neither decision is wrong on its own. Together, I think they change what "the best model" means for a normal user, and the honest reading is: provisional, until you can test it yourself.
Two decisions, one day apart
The first decision is Google's. In its launch post, Google says it is "taking a deliberate approach to rollout" and is "actively engaged in the U.S. government's voluntary process for pre-release model testing." It lays out the order: "Starting this week, we will make Carbon available to a subset of cyber defenders and will gradually phase access to additional trusted partners through our Fairwind Program." Wider access comes after Google continues "to gather feedback and iterate on guardrails." Google gives no date for that step, only "as soon as possible." Our Gemini 4 Carbon launch post covers what Google released and who can get it.
The second decision is Artificial Analysis's. On October 27, 2026, it announced Intelligence Index v5 and wrote: "With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers." The post's subheading reads: "For the first time, the majority of the Index is built on private datasets." At launch, Claude Opus 5.5 (max) led at 58. Our post on Artificial Analysis Index v5 explains the new tests and weights.
One day later, Artificial Analysis put Carbon first at 59, one point ahead of Opus 5.5. In the same post it wrote that "Gemini 4 Carbon is being rolled out to selected users and is not yet publicly available." It said this was also the case at the launch of Gemini 4 Argon.
Why each choice makes sense
I do not think Google is wrong to hold Carbon back. The launch post says Carbon has "state-of-the-art cybersecurity" capabilities, and Google's own Fairwind Program page says of such tools: "But in the wrong hands, the same capabilities can become an equally powerful threat." Giving defenders a head start on a model that Google says can find vulnerabilities and check which ones are exploitable is a defensible call.
Private tests also have a real reason behind them. Artificial Analysis says the tasks in FrontierCode 2.0, Cognition's new private coding test in the index, "stay private to avoid contamination." Contamination here means a model has seen the test questions during training, so it can score well without being able to do the work. Nathan Lambert, writing in Interconnects, calls the shift "a change that directionally I think is defensible, but it's not unanimous."
What a normal user loses
The problem is what the two choices do together. A user reading "#1" today cannot check it in either direction.
You cannot try the model. Google says it will reach developers, businesses and consumers only after the cyber defender and Fairwind stages. The scores that matter most to Google's story were also measured on a version you may never see: the launch post says Carbon's scores on three external security benchmarks were "evaluated without guardrails," and that Google will "iterate on guardrails" before a wider release. My view is that the model consumers eventually get may not behave exactly like the one that was scored.
You cannot read most of the test, either. That is the trade Lambert points to: "you're putting complete trust in the private benchmark builder, as you no longer get the benefit of the community finding issues in the benchmark and pushing the benchmark to improve itself," which, he adds, happens a lot. He also notes that "Artificial Analysis's business model is to make money from the labs to verify their results," and writes: "I have a solid amount of trust for Artificial Analysis, but not the most I do for anyone in the ecosystem." He adds: "There is enough money on the line, where I'd rather see transparency in benchmarking and enabling many eyes to find bugs."
Some of Google's other claims are harder still to check. The launch post says "the Chrome team has reported that Carbon performed more than 10X better than another lab's large frontier model when generating high quality patches." Google does not name the other lab or the model.
Readers noticed the access gap. Our roundup of reactions to Gemini 4 Carbon collected Hacker News and X posts complaining that the model cannot be used yet, and others questioning the timing of the index update. That roundup notes that none of the timing skeptics offered evidence of coordination, and I have seen none either.
The one-point lead is the least useful number
Even taken at face value, the index says less than "best" implies. Artificial Analysis writes that Carbon has "the #1 score in 4 of the 9 Index evaluations" and "is not leading in coding and knowledge work." At v5's launch, it wrote that "no single model leads them all."
Settings and price also move the ranking more than one point does. Artificial Analysis says Claude Opus 5.5 scores 35 at low effort and 58 at max, "at 10x the cost per Intelligence Index task ($0.98 vs $10.04)." It says GPT-6.1 Sol (max) "scores within 4 points of Claude Opus 5.5 (max) at about an eighth of the cost per task." For someone paying per task, a model within four points at about one eighth of the price may be the better pick. A one-point lead at the top of a private index is not a reason to switch tools. Our comparison of Gemini 4 Carbon, GPT-6 Astra and Claude Opus 5.5 lists cost per task and effort settings next to each score.
How I would judge "best" now
Here is the rule I would use, and it is my view, not a source's:
- Treat a "#1" for a gated model as a preview. It tells you where a lab thinks it stands, not what you can buy.
- Read the per-test results, not the composite. Artificial Analysis publishes which model leads each of the nine evaluations. Pick the ones that look like your work.
- Check the effort setting and the cost per task next to every score. Labels such as (max) and (high) are effort settings, and the same model can score and cost very differently at each.
- When the model reaches you, run your own prompts on it. Your tasks are the one private test set you control.
Google can launch to defenders first, and Artificial Analysis can keep its tests private. Both have stated reasons. But when a lab leads with a leaderboard rank for a model the public cannot touch, on a test the public cannot read, the claim asks for more trust than it earns. The fix is not to stop reading leaderboards. It is to read them as a starting point and wait to judge until you can try the model.
Sources
- Google: Gemini 4 Carbon, our next frontier model
- Google DeepMind: Fairwind Program
- Artificial Analysis: Announcing Artificial Analysis Intelligence Index v5
- Artificial Analysis: Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5
- Interconnects (Nathan Lambert): Artificial Analysis v5 and the benchmarker's dilemma
