What Was Actually Published
OpenAI and Broadcom unveiled Jalapeno on June 24, 2026. What happened this week at Hot Chips 2026 is different and more interesting: for the first time, the chip was measured on a benchmark that OpenAI does not own, and the results were made public.
The framework is InferenceX, a public benchmark from SemiAnalysis. Its distinguishing feature is that it scores the full path of serving a request, not raw chip throughput. That matters more than it sounds. A processor can win every synthetic kernel test and still lose on the thing users actually experience, because real inference is memory-bound, latency-sensitive, and spread across a network of accelerators rather than confined to one die.
Three workloads were run: GPT-OSS 120B, DeepSeek R1 at 670 billion parameters, and Kimi K2.5 at a trillion. All three are open-weight models, which is the correct choice for a benchmark anyone else is expected to reproduce.
The Numbers, Stated Precisely
A 700-watt Jalapeno part delivered up to 1.9 times the throughput per kilowatt of a 1,400-watt Nvidia flagship, with latency as much as 3.6 times lower. On DeepSeek R1 with a single user, it produced roughly 700 tokens per second.
Two details in that sentence deserve emphasis, because they are usually dropped in the retelling.
The first is up to. These are peak figures at the favourable end of a measured range, not an average across the workload set. The second is the power asymmetry: a 700-watt part against a 1,400-watt system. Normalising by the watt is the honest way to compare parts of different sizes, and it is also the framing that flatters the smaller, newer, more specialised device.
The Footnote Almost Nobody Quoted
Jalapeno runs HBM4 memory at 15.4 terabytes per second of bandwidth. The Blackwell systems it was measured against do not. They use the previous memory generation.
SemiAnalysis, whose own framework produced the result, called the comparison "somewhat incomplete and unfair." That is an unusual thing for a benchmark provider to say about its own headline, and it should carry weight. Its point is straightforward: the like-for-like matchup is against Nvidia's Rubin, which is also built around HBM4. Comparing a new HBM4 part to an older HBM3-class system measures the memory generation at least as much as it measures the architecture.
Two further caveats sit underneath that one:
- A0 stepping. The silicon tested is the first working revision of the design. A B0 revision is already in fabrication. A0 parts typically carry conservative clocks and unresolved errata, which cuts both ways: the numbers may improve, but they are also not the numbers of a shipping product.
- Single-turn workloads only. The tests measured one-shot requests. Multi-turn conversations, long contexts and agentic workloads with large accumulated KV caches stress memory capacity and interconnect in ways a single turn does not, and that is where a meaningful share of production traffic now lives.
Specialisation Is Supposed to Win This Comparison
Jalapeno is an inference ASIC. It does not train models, it does not run graphics, it does not need to be good at anything except serving tokens from transformer weights. Nvidia's parts are general-purpose accelerators that must serve training, inference, scientific computing and a decade of accumulated CUDA software. A fixed-function design beating a general-purpose one on the single function it was built for is the expected outcome, not the surprising one. The genuinely notable result is not that Jalapeno won on perf-per-watt. It is that a first-silicon part from a company that had never taped out a chip got close enough to be measured on the same axis at all.
What Jalapeno Actually Is
Strip the benchmark argument away and the engineering underneath is substantial.
The Partnership
OpenAI owns the architecture. Broadcom handles physical implementation, networking and manufacturing logistics. Fabrication is at TSMC. This is the same division of labour Google used to get TPUs into production, and it is the only realistic path for a company that wants custom silicon without spending a decade building a semiconductor organisation first.
The Schedule
Roughly nine months from initial RTL to tapeout. For a first custom accelerator, that is fast to the point of being the most impressive number in the entire disclosure, and it says more about what OpenAI can execute than any throughput figure does.
The Topology
A local Jalapeno domain reaches 128 ASICs with low core-to-core latency. Above that sits a half-flattened two-level Clos network built on Broadcom Tomahawk 6 switches, spanning a global domain of 2,048 chips, with higher bandwidth allocated to tensor-parallel traffic and lower bandwidth to expert-parallel traffic.
That asymmetry is the design telling you what it is for. Mixture-of-experts models route each token to a small subset of experts, so expert-parallel communication is sparse and bursty, while tensor-parallel communication is dense and constant. Spending your interconnect budget accordingly is what you do when you know exactly which models you will be serving, because you wrote them.
The Economics
OpenAI has claimed roughly 50% lower inference cost versus GPUs. Take that as a directional target rather than a settled result, since it depends on assumptions about utilisation, power price and amortisation that nobody outside the company can check. But the direction is the point, and the direction is the entire reason the project exists.
When Any of This Ships
This is where enthusiasm should meet the calendar. Volume stays very small through the end of 2026. Meaningful deployment arrives in 2027. Full operation is targeted for the first half of 2028. The 10-gigawatt plan with Broadcom runs through the end of 2029.
So the honest framing is that OpenAI has demonstrated a credible first-generation part on a benchmark, with a production ramp that is still eighteen months away from mattering. Between now and then, Nvidia ships Rubin.
Nvidia Is Helping Finance Its Own Replacement
At the same time as these benchmarks were published, Nvidia is reported to be financing OpenAI infrastructure, including a commitment tied to a $105 billion Ohio data centre. The company being displaced is helping fund the buildings that will house the chips built to displace it. This is not irrationality, and it is not a contradiction. It is what a market looks like when one supplier is so far ahead that even its challengers have to route capital, capacity and credibility through it on the way to competing. The same pattern shows up in every phase of the AI buildout: the incumbent's balance sheet is so large relative to the challengers that funding them is cheaper than losing the ecosystem they would otherwise build somewhere else.
Why Every Lab Is Now a Chip Company
Jalapeno is not an outlier. It is the fifth entry in a pattern that has been running for a decade.
- Google started TPUs in 2015 and is now several generations deep, with a credible external cloud business built on them.
- Amazon has Trainium and Inferentia, positioned explicitly on cost-per-token rather than peak performance.
- Meta has MTIA, aimed first at ranking and recommendation inference, its largest and most predictable internal workload.
- Microsoft has Maia, developed alongside its own network and cooling stack.
- OpenAI now has Jalapeno.
The common logic is not that any of them expects to beat Nvidia outright. None of them will, and none of them needs to. What they need is a credible second source, because the existence of an internal alternative is what changes the price of the external one. A buyer with no fallback negotiates on the supplier's terms. A buyer with working silicon in its own racks negotiates on its own.
The number that will actually settle this. Not a benchmark. The price Nvidia quotes its largest customers in 2027, once Rubin is shipping and Jalapeno, TPU, Trainium, MTIA and Maia are all in volume production at the same time. Custom silicon programmes are usually judged on whether they beat the incumbent. They should be judged on what they do to the incumbent's pricing power, and that result shows up in procurement contracts, not conference slides.
What This Means If You Build on These Models
1. Inference Cost Is About to Stop Being a Constant
For three years, API pricing has been set by what the market would bear on top of a cost base nobody could audit. As custom silicon comes online across multiple providers between 2027 and 2028, the cost floor for serving open-weight-class models drops meaningfully. Expect that to show up first in price cuts on smaller and mid-tier models, where competition is most direct, and last in frontier flagships, where it is weakest.
2. Read Perf-Per-Watt Claims With the Memory Generation Attached
This will not be the last benchmark published against a conveniently chosen comparison point. The three questions worth asking of any of them: which memory generation is on each side, is the silicon a shipping revision or an early stepping, and were the workloads single-turn or multi-turn. If a claim does not answer those, it has not told you what it measured.
3. Portability Is Now a Real Architectural Decision
A world with five serious inference back-ends is a world where the model you deploy and the hardware it runs on are decoupled in ways CUDA lock-in previously prevented. If you are building anything with a multi-year horizon, keeping your inference layer abstracted behind an interface you control is no longer a purity exercise. It is the thing that will let you move when the price curve moves.
4. Power, Not Chips, Is the Constraint to Watch
The 10-gigawatt figure is the one to keep in view. Ten gigawatts is roughly the continuous output of ten large nuclear reactors, committed by one company for one workload. Whatever happens between Jalapeno and Blackwell and Rubin, the binding limit on AI capacity over the next five years is grid interconnection and generation, and that constraint does not care which logo is on the die.
Frequently Asked Questions
My Take
The instinct when a company publishes a benchmark of its own chip is to discount it entirely, and that instinct is wrong here. OpenAI used a third-party framework, tested open-weight models anyone can rerun, and published rather than briefed. That is a better standard of evidence than most of what passes for silicon news.
What I would not do is read the result as a verdict on Nvidia. It is a first stepping, on newer memory, against an older comparison point, on single-turn workloads, eighteen months from volume. Every one of those qualifiers moves the number in the same direction, and the fact that the benchmark provider volunteered as much is the most credible thing in the disclosure.
The part I keep coming back to is the nine months from RTL to tapeout. Companies do not accidentally build a working accelerator in nine months. That schedule implies a hardware organisation that was staffed, funded and executing long before any of this was public, and it reframes what OpenAI has been doing with its capital. The chip is not a hedge against Nvidia pricing. It is a statement that the company intends to own the cost structure of inference the way Google owns it, and that it started early enough to have a real chance.
Whether it succeeds will not be visible in a benchmark. It will be visible in gross margin, two years from now, in a quarterly filing.
Do you read the vendor's number, or the benchmark's footnote?
Related Articles:
