OpenAI Publishes Benchmark Results for Jalapeño Custom Inference Silicon
OpenAI has shared its first empirical performance data for Jalapeño, the custom inference processor co-developed with Broadcom. Presented at the Hot Chips conference and published in an engineering report, the benchmark results evaluate the 700-watt accelerator across open-weight frontier models against Nvidia GB200 and GB300 systems using SemiAnalysis's InferenceX test suite.
The published metrics focus on serving efficiency, token generation latency, and power consumption across varying request batch configurations.
Benchmark Results Across Standard Models
OpenAI evaluated Jalapeño against commercial Nvidia Blackwell-generation hardware across three representative open architectures: GPT-OSS 120B, DeepSeek R1 670B, and Moonshot AI's Kimi K2.5 1T. All tests measured end-to-end request latency and throughput per kilowatt of power draw.
On GPT-OSS 120B, Jalapeño reached peak throughput of 85,448 mixed tokens per second per kilowatt (TPS/kW), compared to 44,960 mixed TPS/kW for the Nvidia GB200 baseline (package TDP 1,200W), representing an approximate 1.9x throughput-per-watt advantage. End-to-end latency dropped from 1.80 seconds to 1.03 seconds (1.7x lower), while minimum time between tokens (TBT) fell from 1.87 ms (535 tokens/second per user) to 0.69 ms (1,459 tokens/second per user).
On DeepSeek R1 670B (MXFP4 precision), Jalapeño achieved 19,641 mixed TPS/kW versus 11,781 mixed TPS/kW on the GB300 baseline (package TDP 1,400W), marking a 1.7x efficiency gain. End-to-end latency decreased by 3.6x, from 5.99 seconds to 1.65 seconds, with minimum TBT dropping from 5.90 ms (169 tokens/second per user) to 1.43 ms (700 tokens/second per user).
On Kimi K2.5 1T (MXFP4 precision), the chip recorded 18,195 mixed TPS/kW against 11,862 mixed TPS/kW for the GB300 system (1.5x higher). Latency fell 3.4x, from 5.31 seconds to 1.56 seconds, while minimum TBT declined from 5.48 ms (182 tokens/second per user) to 1.44 ms (694 tokens/second per user).
While the package thermal design power is rated at 700 watts, OpenAI stated that measured sustained power remained at or below 550 watts during the benchmark runs.
Architectural Focus and AI-Assisted Kernel Optimization
Jalapeño is structured specifically around the bifurcated requirements of large language model serving: the compute-bound prompt prefill phase and the memory-bandwidth-bound autoregressive token generation (decode) phase.
According to OpenAI, data movement between chips and memory subsystems is the primary bottleneck in multi-agent and long-context inference. The Jalapeño architecture utilizes an integrated, large-domain interconnect network that allows model state, including dynamic key-value (KV) caches, to be pinned locally and routed without cross-system memory synchronization penalties.
OpenAI also reported using its own AI models during chip design and software bring-up. The hardware development cycle from architectural planning to initial tapeout took nine months, aided by AI-driven circuit exploration and verification tooling. On the software stack, OpenAI used Codex paired with GPT-Astra to generate low-level compute kernels for open-weight models not included in the original silicon schedule. For targeted GPT-OSS attention and mixture-of-experts (MoE) operations, the AI-generated kernels executed 1.5x to 1.8x faster than human-authored baseline implementations.
Deployment Timeline and Infrastructure Strategy
OpenAI hardware vice president Richard Ho confirmed that initial production deployment of Jalapeño in data centers is scheduled to begin in late 2026 at low volumes, before expanding to broader capacity throughout 2027. Successive iterations, designated Gen 2 and Gen 3, are currently in design stages.
OpenAI noted that proprietary silicon will not replace merchant hardware entirely. The company plans to continue large-scale deployment of GPUs and accelerators from Nvidia and third-party vendors for both training and serving workloads alongside Jalapeño clusters.
