OpenAI has revealed architectural specifications and benchmark data for its first in-house artificial intelligence accelerator, code-named Jalapeño. Presented by hardware lead Richard Ho at the Hot Chips conference at Stanford University, the application-specific integrated circuit (ASIC) is engineered specifically for large language model inference rather than model training.
Developed over an 18-month partnership with Broadcom and manufactured by TSMC, the chip targets large-scale token generation workloads across ChatGPT, OpenAI Codex, and API endpoints. OpenAI designed the processor from scratch to optimize memory movement, execution kernels, and interconnect bandwidth tailored to transformer autoregression.
Architecture and Efficiency Metrics
Jalapeño operates within a 700-watt thermal design power (TDP) envelope, with sustained operational consumption measured at or below 550 watts during testing. The architecture prioritizes low per-token serving costs and low time-between-tokens latency, aiming to bridge the gap between high-throughput general accelerators and ultra-low-latency custom silicon.
OpenAI benchmarked the ASIC against Nvidia GB200 and GB300 systems using SemiAnalysis's public InferenceX benchmark suite across three open-weight architectures:
- GPT-OSS 120B: OpenAI's open model architecture.
- DeepSeek R1 670B: Mixture-of-experts model evaluated across variable batch sizes.
- Moonshot AI Kimi K2.5: One-trillion-parameter model architecture.
In published test runs, Jalapeño delivered 1.5x to 1.9x higher throughput per kilowatt compared to Nvidia GB300 rack systems. On end-to-end response latency, the chip registered 1.7x to 3.6x lower latency at comparable operational batch sizes. The largest efficiency deltas emerged at low-latency operating points, where specialized memory subsystem bandwidth reduced token generation wait times.
Roadmap and Infrastructure Strategy
OpenAI indicated that initial production deployments of Jalapeño will begin later this year to offload high-volume commercial inference. A second-generation silicon revision is scheduled to tape out in the coming months, with a third generation currently in conceptual design.
The deployment of custom silicon reflects a broader industry shift among frontier AI labs to mitigate data center power constraints and reduce reliance on single-vendor supply chains. While OpenAI continues to expand deployments of Nvidia GPUs and multi-gigawatt third-party cloud agreements, specialized ASICs provide margin relief on high-frequency API and consumer chat traffic.
