In late 2023, Chatbot Arena was a research project at UC Berkeley. Users typed a question, two anonymous models answered, the user picked a winner, and an Elo rating moved. It was a way to sidestep contaminated test sets and marketing claims, and it cost almost nothing to run.
In October 2026, the company built on that project, Arena (it was LMArena until a January 2026 rebrand), closed a $200 million Series B at a $3.1 billion valuation. The round was co-led by Lightspeed and Khosla Ventures, with Salesforce Ventures and Dell Technologies Capital joining. The company says it exceeded $100 million in annualized revenue, up from a $30 million run rate when it raised its Series A in January. Grading AI models is no longer a public service. It is an industry, and the industry now has funding rounds, subscription tiers, and the same conflicts of interest as every other business that audits things.

Why the demand is real
Three forces turned independent evaluation from a nice-to-have into a market.
Saturation. Stanford's 2026 AI Index Report notes that frontier models gained 30 percentage points in a single year on Humanity's Last Exam, a benchmark built to stay hard for AI. When six or eight frontier models score within one percentage point of each other on a test designed for a world where 75 to 85 percent was a capability shift, the test stops being a decision tool. The same Stanford report found the top four labs clustered within 25 Elo points of each other on the Arena leaderboard as of March 2026. Buyers cannot choose between products using numbers that no longer separate them, so the evaluation firms keep being paid to build harder tests.
Distrust of lab-reported numbers. In April 2025, The Verge reported that Meta had "got caught gaming AI benchmarks": Llama 4 Maverick posted a strong ELO of 1417 and a number-two Arena ranking from a crafted submission, while the build that shipped to the public performed noticeably worse. Meta denied training on test sets, and the incident became the standard reference point for why vendor-run numbers belong in the screening pile, not the business case.
Measurement itself is under audit. Stanford's report puts error rates on some widely used evaluations as high as 42 percent. Epoch AI, a nonprofit, now runs a Benchmark Reviews program that inspects a benchmark's tasks, scorers, harness, and model settings before certifying it. Across 21 published reviews, 10 benchmarks are rated Flawed, including SWE-bench Verified, Humanity's Last Exam, and Terminal-Bench 4.0. The message to buyers is that a leaderboard score, even an independent one, is a claim about a particular version run at particular settings, not a fact.
Three ways to monetize a scoreboard
Arena sells data generated by the public and consumed by the labs. The free platform is the data engine: the company says the platform has logged 350 million sessions and 62 million votes from tens of millions of monthly visitors in more than 150 countries. The paid product, an AI Evaluations service launched in September 2025, sells analyses of that preference data to model providers and enterprises. Arena works with OpenAI, Google, and xAI as paying customers, and has open-sourced 375,000 data points including its leaderboard methodology. The pitch and the problem are the same: labs want the preferences of millions of real users, and the labs are also the customers. Contrary Research's company analysis puts it bluntly: the easiest path to profitability is selling evaluation tools and premium services to the same labs whose models Arena ranks.
Artificial Analysis sells independence, and increasingly sells privacy. The project started in a Sydney basement in 2024 by George Cameron and Micah Hill-Smith, took seed funding from Nat Friedman and Daniel Gross through the AI Grant program, and incorporated into a company in early 2026. Its Intelligence Index became one of the most-cited composite scores in the industry. In November 2025 it overhauled the index, dropping the staples labs loved to quote (MMLU-Pro, AIME 2025, LiveCodeBench) in favor of economically framed tests: GDPval-AA, built on OpenAI's dataset of real work products across 44 occupations, CritPt research physics (the leading model scored 11.5 percent), and AA-Omniscience, which penalizes guessing instead of abstaining. The November 2026 Index v5 goes further: for the first time, Artificial Analysis says, the majority of the index is built on private datasets no model has ever seen. Revenue, per third-party buyer's guides, mixes free leaderboards with a Pro subscription reported at around $417 per seat per month and private benchmarking sold to labs.

Epoch AI stays nonprofit and spends its credibility on audits and open data. Its Benchmarking Hub publishes results under a permissive license, its Epoch Capabilities Index fits a Bayesian model over many benchmark scores (top value 173, set by Claude Fable 5.2 in November 2026, on pre-launch testing), and it builds genuinely hard tests: FrontierMath Erdős hands AI systems 68 unsolved problems posed by Paul Erdős, and the best model has solved three. Epoch funds research rather than selling rankings, which is precisely why its Flawed verdicts carry weight, including on benchmarks other outlets quote uncritically.
The structural problem nobody has priced in
Every business model here has the same fault line. The referee needs customers, and only two things in AI have money: the labs and the enterprises buying from the labs. Sell to labs and you rank your paymasters in public. Sell private test sets to enterprises and the public cannot verify your numbers. Retire famous benchmarks to stay ahead of contamination and you burn the comparability that made you cited. Artificial Analysis's v5 pivot to private data is the cleanest example: it trades the one thing a scoreboard sells, auditability, for the one thing a scoreboard cannot keep from models, secrecy.
The 2026 equilibrium is a stack of partial trust: Arena for what humans prefer, Artificial Analysis for what models can do on private work-like tests, Epoch for what benchmarks are actually worth reading, and labs' own system cards for what labs claim. None of these numbers transfer cleanly to your workload. Which is why the enterprise buyer's guides that emerged this year all converge on the same advice: treat third-party scores as screening, then run 50 or more tasks from your own failure cases at the configuration you will actually pay for.
The bottom line
Independent evaluation grew up in three years from a Berkeley spreadsheet to a $3.1 billion valuation because model buyers needed a neutral party and labs could not provide one. It stayed neutral only as long as it stayed small and public. The money arrived, the tests went private, and the graders started charging for the answer key. The machines have not changed. The grading industry now sells what a benchmark never could: defensible cover for a procurement decision.
Sources
- Arena raises $150 million Series A at $1.7 billion valuation (PRNewswire, Jan 6, 2026): https://www.prnewswire.com/news-releases/lmarena-raises-150-million-to-build-the-worlds-most-trusted-ai-evaluation-platform-302653012.html
- LMArena raises $100M at $600M valuation (SiliconANGLE, May 21, 2025): https://siliconangle.com/2025-05-21/lmarena-raises-100m-600m-valuation-expand-ai-benchmarking-platform
- LMArena business breakdown and conflict-of-interest analysis (Contrary Research): https://research.contrary.com/company/lmarena
- Arena.ai history, rebrand, and Series B citations (Wikipedia): https://en.wikipedia.org/wiki/Arena.ai
- Arena Raises $200 Million at $3.1 Billion Valuation, revenue and usage figures (MLQ.ai, Oct 2026): https://mlq.ai/news/arena-raises-200-million-at-31-billion-valuation-in-series-b
- Arena Series B details and platform metrics (WOWTALE, Oct 10, 2026): https://en.wowtale.net/2026-10-10/235432
- Arena Raises $200M Series B (The SaaS News, Oct 2026): https://www.thesaasnews.com/news/arena-raises-200m-series-b
- Meta got caught gaming AI benchmarks (The Verge, Apr 2025): https://www.theverge.com/meta/645012/meta-llama-4-maverick-benchmarks-gaming
- Meta accused of Llama 4 bait-and-switch to juice LMArena rank (The Register, Apr 8, 2025): https://www.theregister.com/software/2025-04-08/meta-accused-of-llama-4-bait-n-switch-to-juice-lmarena-rank/1470166
- Artificial Analysis overhauls its Intelligence Index (VentureBeat): https://venturebeat.com/technology/artificial-analysis-overhauls-its-ai-intelligence-index-replacing-popular
- Artificial Analysis company profile, founding and funding (DataLearnerAI): https://www.datalearner.com/en/ai-organizations/artificial-analysis
- Intelligence Index v5 announcement, private datasets (Artificial Analysis on LinkedIn): https://www.linkedin.com/pulse/announcing-artificial-analysis-intelligence-index-nevqc
- Intelligence Index methodology (Artificial Analysis): https://artificialanalysis.ai/methodology/intelligence-benchmarking
- Model leaderboard and current index results (Artificial Analysis): https://artificialanalysis.ai/leaderboards/models
- Vendor AI benchmarks buyer's guide, Pro pricing and replication advice (The Daily Brief): https://www.beri.net/article/vendor-ai-benchmark-claims-buyers-guide-independent-evals-contamination-50-task-replication
- Epoch AI benchmarks and FrontierMath Erdős launch (Epoch AI): https://epoch.ai/benchmarks
- Epoch finds half of 21 major AI benchmarks are flawed (AlphaSignal): https://alphasignal.ai/news/epoch-finds-half-of-21-major-ai-benchmarks-are-flawed
- Epoch Capabilities Index (Epoch AI): https://epoch.ai/eci
- Stanford HAI 2026 AI Index, technical performance (saturation, 42% error rates, Elo clustering): https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance
