For a decade the recipe for a better model was more text. Scrape the internet, predict the next token, scale. That recipe has hit two walls: the usable internet is largely consumed, and next-token prediction on frozen text produces something that can talk but not necessarily something that can finish a 40-step task in a terminal. Since OpenAI's o1 showed that reinforcement learning against verifiable math and coding problems produces capabilities that look a lot like reasoning, the frontier has moved on to a less glamorous input. Labs now pay for training grounds: simulated applications, sandboxes, and graders where an agent attempts real work, gets scored, and tries again. These are reinforcement learning environments, and by late 2026 they are the most contested input in AI training after compute itself.
What an environment actually is
In current practice an RL environment is the set of actions a model can take (run code, click buttons, search files) plus the state those actions affect, wrapped around a task and an automated grader. The grader is the point: a unit test suite, a rubric checked by another model, a diff compared against a reference solution. Epoch AI interviewed 18 people across environment startups, neolabs, and frontier labs, and their examples are concrete:
- A git repository at a specific commit with a failing test suite, SWE-bench style, where the grader reruns the tests.
- An Airbnb clone with fake listings and prices, where the agent navigates a DOM and the grader checks the final answer.
- A Bloomberg terminal clone for financial lookup tasks, or an Excel clone where the grader compares a pivot table against a reference.
Deliveries are typically Docker containers, though not always. The terminology is fuzzy: one shop's "environment" is a single container with one task, another's is a collection with hundreds.
The price list
Pricing in this market is mostly private, but the Epoch interviews put numbers on it:
- Tasks run $200 to $2,000 each as a rule of thumb. The rare, gnarly software engineering task can list at $20,000.
- SemiAnalysis reported basic website replicas ("UI gyms") at roughly $20,000; a faithful clone of a complex product like Slack runs closer to $300,000.
- Contracts are commonly six figures and often seven figures per quarter, with $300k-$500k mentioned as a smaller deal range.
- Exclusivity is a real lever: two environment founders independently said exclusive deals price about 4-5x above non-exclusive ones, since the same vendor clone can otherwise be resold to a competing lab.
The demand side backs it up. The Information reported in September 2025 that Anthropic had discussed spending over $1 billion on RL environments in the following year, and TechCrunch noted labs were considering comparable commitments.
Why pay $2,000 for a single task? Because of what the task rides on. Mechanize estimated that with frontier-length transcripts (around 500,000 tokens per rollout per task), a group size of 64 as in DeepSeek-R1-style training, and roughly five reuses per task, each RL task carries on the order of $2,400 of compute. Buy a $100 junk task and you burn $2,400 of GPU time learning noise. Cheap tasks waste compute, so labs bid up task quality instead.

Who sells them
The market has settled into four layers:
- Environment-native startups. Mechanize, which has worked with Anthropic, plus HUD, Fleet AI, Bespoke Labs, Plato and Veris AI, as catalogued by CleverX.
- Open platform plays. Prime Intellect runs an Environments Hub and raised a $130M Series A in July 2026 led by Radical Ventures, total funding above $150M, with reported revenue run-rate above $100M and more than 6,000 customers.
- Legacy data vendors pivoting. Mercor ($10 billion valuation after its $350M Series C, and now acquiring environment shops like Deeptune), Scale AI, Surge, Turing and Handshake all sell environments alongside labeling, per Epoch.
- In-house teams and product partnerships. xAI and Anthropic post environment engineer roles; labs partner with software vendors who know their own interfaces, like Benchling with Anthropic or Shopify and Stripe integrations with OpenAI.

What separates a good environment from shelfware
The operators interviewed agree on the quality bar, and it is mostly about the grader.
Reward-hacking resistance is table stakes. A policy under RL pressure eventually finds whatever seam the container leaves open: editing the test file, checking out future commits that contain the fix, reading grading material that shipped inside the image. Prime Intellect found this in its own runs and now withholds test patches and expected outputs until scoring time; some public datasets, like R2E-Gym, still leave tests readable inside the container, which is fine for eval and not fine for training.
Difficulty calibration matters more than difficulty. If every rollout of a task scores the same, all pass or all fail, there is no learning signal. Labs want at least a 2-3% pass rate out of 64 or 128 rollouts, a spread of tasks across 0%, 5% and 30% pass rates, and they discard tasks once the model solves them around 70% of the time.
Raw task counts are not the moat. Prime Intellect advertises around 365,000 tasks across 23 integrated tasksets, and the honest number is smaller: of 32,079 raw SWE-rebench-V2 rows, only 6,275 survived gold-patch validation. Broken images, network-dependent tests and tasks solvable with zero edits are common in mined data. A lab researcher summed up the low end of the market bluntly: everyone can vibe-code a buggy website clone, and there is a large amount of useless environments out there.
Where the spend is going
Math environments, the original staple since o1, may be shrinking; one founder said they are easy to write but transfer poorly. Coding stays the largest single market and is drifting from toy SWE-bench instances toward production-shaped environments with GitHub, issue trackers and IDEs wired in. The fastest growth is in enterprise workflows: expense reports, CRM updates, spreadsheet reconciliation, ERP navigation. These are exactly the tasks customers ask agents to do, and they score cleanly. Long-horizon, multi-application tasks are the stated frontier for 2027.
The operational constraint is not supply of experts. Finding domain experts is a marketplace problem that Mercor and its peers have solved. Managing them is the bottleneck: keeping contractors from generating tasks with LLMs, iterating graders until they survive attack, shipping fast enough to stay ahead of model capability. One founder said the cap on their own revenue is the difficulty of scaling task creation at required quality. The skills that matter, per the same interviews, are domain knowledge and expert-level prompting rather than machine learning: the best task authors are described as heavy Claude Code users, not researchers.
Why this matters if you are not a frontier lab
Three practical reads. First, evaluations and environments are converging: an eval harness is an environment used for measurement instead of training, so work you do to score agents on your own product doubles as RL substrate. Second, your product logs are a task stream. Will Brown of Prime Intellect frames deployed product usage as an environment: tasks, harness and reward arriving from real users. Companies with workflows but no grader have raw material they have not priced. Third, treat the environment market as a hiring and procurement signal. If a vendor cannot show grader iteration history, difficulty distributions or reward-hacking postmortems, they are selling containers, not training signal.
The internet stopped being the training corpus. Little scripted worlds where models attempt, fail and get graded have taken its place, and someone now has to build, verify and maintain every one of them. That is the job the last decade of data labeling companies trained for, priced 20x higher.
Sources
- Epoch AI: An FAQ on Reinforcement Learning Environments
- Mechanize: Cheap RL tasks will waste compute
- TechCrunch: Silicon Valley bets big on environments to train AI agents
- The Information: Anthropic and OpenAI developing AI co-workers
- Prime Intellect: Scaling Agentic RL, 365,000+ environments
- Prime Intellect: Environments Hub announcement
- Intel Capital: Prime Intellect raises $130M Series A
- CleverX: Companies building RL environments in 2026
- New Market Pitch: RL environments funding tracker
- SemiAnalysis: RL environments and RL for science
- Will Brown, Prime Intellect: RL environments at scale (talk)
