OpenAI's custom inference chip Jalapeño delivered 1.5x–1.9x more AI work per watt and 1.7x–3.6x lower end-to-end latency than Nvidia Corp. Blackwell systems across three large models, the company said at the Hot Chips conference on Tuesday1,2,4. On highly interactive workloads, Jalapeño was 2.1x–4.1x faster than the systems it was compared against.
The results come from SemiAnalysis's InferenceX benchmark suite, run with OpenAI engineers in the lab on working silicon10. The tested models were GPT-OSS 120B, DeepSeek-R1 670B, and Kimi-k2.5-1T-A32B11. SemiAnalysis said Jalapeño beat every Nvidia Corp., AMD, and Google chip it has tested on multiple top open-weight models9. Jalapeño hit over 700 tokens per second per user at concurrency 1 on DeepSeek-R1 and approximately 1,400 tokens per second per user across all three models.
"The bottom line is that the results show a very, very significant performance advance over state of the art," said Richard Ho, OpenAI's head of hardware. Ho said Jalapeño offers "the best of both worlds" with lower latency and higher throughput, whereas existing hardware systems "often require a trade-off between the two"12.
Jalapeño is an application-specific integrated circuit developed in partnership with Broadcom, built from scratch exclusively for LLM inference. Design work began in mid-2024, and OpenAI went from initial team hiring to manufacturing tape-out in approximately 16 months, with tape-out occurring in November 2025. The chip's B0 stepping is currently in fabrication and delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized compute die manufactured on TSMC's N3P process. Jalapeño is rated at 700W TDP but reportedly stayed at or below 550W during the tested runs. The chip will ship with HBM4 memory.
At the system level, OpenAI's rack design, done in partnership with Celestica, pairs 16 host CPU trays ("Katsu") with 16 ASIC trays ("Vindaloo"), each containing eight Jalapeño ASICs for 128 per rack. Up to 2,048 Jalapeño XPUs can connect within a single scale-up network.
SemiAnalysis noted that Nvidia Corp.'s upcoming Rubin platform, which also uses HBM4, is a more apt like-for-like comparison, and that Vera Rubin systems are starting to ship to customers now5. Ho estimated Jalapeño would deploy in "very small volumes" by end of 2026, with more significant deployment in 2027. OpenAI said Gen 2 is already deep in development and Gen 3 is underway.
OpenAI used its own models throughout Jalapeño's development. AI-generated implementations of selected GPT-OSS attention and mixture-of-experts blocks ran 1.5x–1.8x faster than versions written by OpenAI's own experts. Using Codex with GPT-Astra, the hardware team brought three open-weight models that were not part of Jalapeño's original production plan to high performance within two months.
Analysts told CNBC that Jalapeño could put pressure on Nvidia Corp. in the inference market and reduce OpenAI's reliance on Nvidia Corp. for some workloads. Adrien Sanchez said Jalapeño "raises the stakes for Nvidia's largest customer relationship specifically". Omdia expects custom ASIC chips like Jalapeño to exceed GPUs in volume by 2028. OpenAI said it expects to continue widely deploying accelerators from Nvidia Corp. and other partners for both training and inference8. Nvidia Corp. said it would provide up to $105 billion in credit support for OpenAI's data center project in Ohio, which will exclusively use Nvidia Corp. compute.
ANALYSIS The benchmark results position Jalapeño as a credible first-generation ASIC, but the competitive window is narrow: Vera Rubin systems that use the same HBM4 memory generation are already shipping, and Jalapeño production is not scheduled to ramp until 2027.