OpenAI released the first measured performance numbers for Jalapeño, the custom inference accelerator it has been building with Broadcom since mid-2024, and the results are considerably stronger than first-generation silicon usually is. Running SemiAnalysis's public InferenceX suite on GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T, Jalapeño delivered 1.5 to 1.9 times more work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison Nvidia systems, and 2.1 to 4.1 times higher performance in the highly interactive regime. On DeepSeek R1 the part reached 19,641 mixed tokens per second per kilowatt against 11,781 for GB300, with end-to-end latency of 1.65 seconds versus 5.99, and minimum time-between-tokens of 1.43 milliseconds versus 5.90. On GPT-OSS the peak figure was 85,448 mixed tokens per second per kilowatt against 44,960 for GB200. All of this was achieved with single-token prediction, no speculative decoding and no prefill-decode disaggregation, while the Nvidia configurations used multi-token prediction.
The architecture explains a lot of the gap. Jalapeño is a weight-stationary systolic matrix engine using MXFP formats, but with support for small matrix shapes so it avoids the tiling cliffs that punish TPU-class arrays on awkward dimensions. Cores and HBM are divided into slices, each core slice holding a low-latency local view of its own HBM slice, with synchronization confined to a dedicated high-bandwidth collective network. Cores are out-of-order with an L1 cache rather than the software-managed scratchpad every other accelerator uses, which removes barrier and launch latencies that GPUs must amortize over larger batches. The package carries HBM4 at roughly 10 Gbps pin speed for 15.4 TB/s of bandwidth, ahead of the 9.6 Gbps Nvidia is getting in Rubin, and off-package I/O runs through an N3E chiplet with 32 lanes of 800G SerDes feeding a 2,048-accelerator scale-up domain. The B0 stepping now in the fab delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized N3P die at 700 W TDP, against 17.5 PFLOPs of dense NVFP4 for a similarly sized Rubin die at 900 to 1,150 W.
The timeline is the part that should unsettle the merchant silicon vendors. Design work began in the middle of 2024, tapeout of the full CoWoS design landed in November 2025, and these numbers come from A0 stepping with only three months of bring-up on real silicon and a software stack started from zero. OpenAI says it used its own models to explore implementations and optimize arithmetic circuits, claiming an eight percent reduction in SIMD area and ten percent in matrix-engine area, and that Codex with GPT-Astra brought three open-weight models that were never in the original production plan up to high performance in two months, with AI-generated attention and mixture-of-experts kernels running 1.5 to 1.8 times faster than the human-expert versions on selected blocks. SemiAnalysis, who verified the InferenceX runs in the lab, raise the obvious caveats: all numbers were supplied by OpenAI, the workloads are 8k-in, 1k-out single-turn rather than the long-context multi-turn AgentX suite that stresses routers and prefix caching, and the fair comparison is Rubin rather than Blackwell since Vera Rubin systems are shipping now while Jalapeño has only engineering samples. Even against Rubin's July figures, though, Jalapeño's single-token-prediction throughput per megawatt exceeds Rubin's multi-token-prediction results, and on performance per total cost of ownership the two are roughly level before speculative decoding is added to Jalapeño. Deployment inside OpenAI's own infrastructure is scheduled to begin by the end of the year, with volume ramp through 2027 and second and third generations already in development.
- SemiAnalysis verified the InferenceX runs in person but flags that all numbers came from OpenAI and no AgentX long-context results exist yet.
- SemiAnalysis argues the honest comparison is Vera Rubin, not Blackwell, since both use HBM4 and Rubin is already shipping.
- OpenAI frames the result as full-stack advantage; SemiAnalysis reads it as evidence the CUDA moat is thinner than assumed.
- Hacker News discussion centred on the nine-month design-to-tapeout cycle as the real signal about AI-assisted chip design.