Modal, a cloud company that rents computing power to AI developers, said on October 19 that it plans to add Nvidia's Vera CPU to the machines behind Modal Sandboxes, which runs code for uses such as background agents and reinforcement learning. In a Modal engineering post, Erik Dunteman says early benchmarks showed "up to double the task throughput on Vera compared with an equivalent x86 GNR machine." Reuters reported that Modal has until now run this work only on Intel and AMD chips.
Modal says Nvidia hand-delivered its first Vera CPU server to its San Francisco office last week. Reuters calls the win "an early sign" that Nvidia's push into a CPU market long controlled by Intel and AMD has traction.
What Vera is, and what changes for Modal
Modal describes Vera as Nvidia's "new Arm-compatible CPU, built for the Vera Rubin GPU platform but also available as a standalone CPU server." Modal has a "long history exclusively running x86 machines," so support for Arm means changes "across our infrastructure and image build system," the post says. Reuters describes these as "deep software changes."
Modal is not dropping its current suppliers. Chief executive Erik Bernhardsson told Reuters that Modal will run the Nvidia chips alongside its Intel and AMD offerings in a mixed fleet and put each job on the hardware that runs it best. He said the tests were against Intel and AMD chips already on the market, and that newer models from both are in development. "We'll test them when they're available," he said.
Neither the Modal post nor Reuters gives a date for the rollout.
Why a CPU matters for agents
Modal runs "hundreds of concurrent containers and VMs on each cloud host," and sandbox load is spiky. A sandbox can sit mostly idle while it waits on an LLM, then surge to 10x its baseline CPU use during an npm install or an expensive grep.
Modal sees two prevalent use cases, and in both, CPU time costs more than the CPU:
- Background agents, such as Ramp Inspect or agents on Niteshift. Faster agents mean faster answers and lower CPU time.
- Reinforcement learning rollouts. Most training setups are still synchronous, so GPUs sit idle until a full set of sandboxed rollouts completes. "Any time saved on CPU is time saved on GPU," the post says.
For background on how sandbox providers isolate agent code, see our comparison of E2B, Modal, Daytona and Docker with gVisor.
How Modal tested it
Modal built its own benchmark, Agentless Agent Bench. It is a set of 50 Terminal-Bench oracle solutions and verifiers with all network calls removed, prewarmed in concurrent Docker containers. It keeps only the work agents and RL do on the CPU: compilation, file grepping, git operations, device emulation and more.
The first test ran tasks one at a time on an unloaded machine. Across the 50 tasks, "Vera often performed better than the other CPUs, but not always, and not by a particularly large margin," Dunteman writes.
The second test raised the number of concurrent containers and counted task completions in a fixed time. Modal's chart compares six setups: Nvidia Vera, AMD Zen 5, Intel GNR, Intel GNR (VM), AWS Graviton 4 (VM) and AWS Graviton 5 (VM), at 50, 100 and 150 concurrent containers.
At 150 concurrent containers with 79 logical CPUs, the chart's data shows these verified completions per 60 seconds:
- Nvidia Vera: 5,702
- Intel GNR: 3,027
- AWS Graviton 5 (VM): 2,865
- AWS Graviton 4 (VM): 2,596
- Intel GNR (VM): 1,726
- AMD Zen 5: 1,047
The chart notes different physical core counts: Intel GNR 128, Intel GNR (VM) and Graviton 96, and Vera 88. Modal says per-task latency on Vera measured "equal to, or sometimes even lower than" the x86 machine. It credits Vera's high memory bandwidth, which "allows for significantly more container workloads to pack per core, without CPU throttling." As CPU counts rose, the other chips "hit early throughput plateaus, while Vera continues to improve," according to the post.
For Modal, the post says, this "unlocks greater packing density and more forgiving burst protection." Modal lets sandbox users reserve capacity for P50 usage rather than the worst-case P100.
Caveats
These are Modal's own early results, on a benchmark Modal designed, from one Vera server node. The post says the results suggest Vera could go "even further than the node we had access to." The Intel and AMD parts tested are current products, and Bernhardsson told Reuters both rivals have newer chips coming.
The bigger picture
Modal is growing fast. On October 14 it announced a $750M Series D at a $15.75B post-money valuation, led by Accel, and said it had crossed $1B in annualized run rate. Sandboxes alone are a $300M run rate product, and Modal says it has spun up close to 3 billion sandboxes.
For Nvidia, Reuters reports the company has told investors it expects at least $20 billion from Vera chip sales this fiscal year, in a market it values at roughly $200 billion. Nvidia has said OpenAI, Anthropic, Oracle and Perplexity also plan to use the chips, per Reuters.
Faster code execution is also the aim of doubleAI's Maxwell, a code execution model, while Intel is pitching its own inference hardware with the Data Center GPU D-Series.
