TechFlow Logo
Login/ Sign up
ETH Gas
Gwei
Fear
gas
SemiAnalysis: Can OpenAI’s In-House Chips Actually Outperform NVIDIA’s Flagships in Real-World Tests? The Scenario Jensen Huang Feared Most Has Finally Happened.

SemiAnalysis: Can OpenAI’s In-House Chips Actually Outperform NVIDIA’s Flagships in Real-World Tests? The Scenario Jensen Huang Feared Most Has Finally Happened.

2026.08.26
Share
This paper will delve into the architectural details, software details, and performance results of Jalapeño on InferenceX.

Author: SemiAnalysis

Translated by: TechFlow

TechFlow Summary: Real-world benchmark data for OpenAI’s first self-developed inference chip, Jalapeño, has been revealed. Its energy efficiency directly surpasses NVIDIA's current flagship, Blackwell, and closely approaches the next-generation Rubin. For AI companies constrained by data center power bottlenecks, this chip could reshape the compute market landscape. However, whether a first-generation chip can truly shake up NVIDIA’s CUDA ecosystem remains an open question.

Total Cost of Ownership, Throughput per Megawatt, and Intricate Details: Comparing the Self-Developed ASIC (Jalapeño) Against Rubin

Over the past two years, OpenAI has been quietly developing "Jalapeño," an inference chip recently unveiled at Hot Chips. Rumors of its successful tape-out have circulated for a while. Now we have the details. OpenAI invited us to examine the chip, enter their lab to verify its actual capabilities, and run benchmarks using our InferenceX suite.

In June this year, OpenAI announced a chip project in partnership with Broadcom, designed from scratch specifically for LLM inference. Design work began in mid-2024, and it took only about 16 months from initial team recruitment to manufacturing tape-out—a remarkably fast ASIC development cycle.

Typically, first-generation chips lack competitiveness, but OpenAI did the opposite. Across multiple top-tier open-source models, it outperformed all NVIDIA, AMD, and Google chips we were able to test, placing it at the forefront of the industry. OpenAI achieved this through extreme hardware-software co-design. Surprisingly, OpenAI did not overly focus on a specific segment of model inference; instead, it focused on creating a general-purpose chip that delivers high performance across all scenarios.

This article delves into Jalapeño’s architectural details, software details, and performance results on InferenceX.

A General-Purpose Inference Chip

Everyone claims OpenAI’s chip is customized exclusively for OpenAI models, but that’s incorrect. OpenAI built a general-purpose chip for AI inference.

The timeline is absolutely insane. This proves that the saying "using AI to accelerate chip design" is real. Despite the tight schedule, OpenAI invested heavily, made pragmatic design decisions, and boasts a highly capable team, which makes this outcome unsurprising.

Just looking at the specifications, it immediately becomes a strong contender:

And the use of HBM4 makes it competitive enough to stand alongside NVIDIA and AMD’s flagship GPUs:

Many media reports on this chip followed OpenAI’s offhand remarks, claiming it would optimize its own models in ways other chips cannot. That is wrong. Jalapeño is a general-purpose inference chip capable of running various models and workloads, including our InferenceX benchmark—which we ran in the lab alongside OpenAI engineers. As a joke, OpenAI even showed us it running Doom, ported to their chip using nothing but Codex prompts.

Below are our most critical performance-per-watt results, measuring token throughput under total power consumption per megawatt. Jalapeño completely wins against all other chips. All of this was achieved without using multi-token prediction (MTP), whereas the other chips in the chart represent their best SKU configurations with MTP fully enabled.

In nearly all scenarios, Jalapeño’s performance-per-watt exceeds Blackwell’s, and it was not tuned for a specific point on the curve. It performs excellently not only in low-latency scenarios but also in high-throughput scenarios. A fairer comparison looks at single-token prediction results, where Jalapeño leaves every competitor far behind. Under low-concurrency conditions, Jalapeño demonstrates remarkable interactivity; on the DeepSeek R1 model, it delivers over 700 tokens per user per second at a concurrency of 1.

Incredibly, all of this is achieved using single-token prediction (STP), without speculative decoding or prefill-decode disaggregation. Besides DeepSeek R1, we also observed other models, including Kimi-K2.5 and GPT-OSS, reaching approximately 1,400 tokens per user per second. For all models tested, we confirmed that Jalapeño’s GSM8K evaluation results match those of NVIDIA chips.

Here are a few caveats. First, all data was provided by OpenAI. We personally verified InferenceX's operation in the lab, but did not run the full InferenceX benchmark suite, nor did we see the AgentX results. AgentX is our preferred suite for comparing chip performance because its dataset's ultra-long context and multi-turn nature reflect caching behavior under real production workloads. Frameworks that perform well on 8k1k tests may perform worse on AgentX, because real production loads stress components like routers, prefix caching mechanisms, cache management, and offloading infrastructure. Single-turn 8k1k tests do not cover these aspects. For more details, please read our AgentX article.

AgentX - InferenceXv3: Can the CUDA Moat Withstand Agentic Inference?

Second, we believe the comparison with Blackwell is somewhat incomplete and unfair. Jalapeño’s true competitor is Rubin and similar chips that also utilize HBM4. Vera Rubin systems have already begun shipping to customers, while OpenAI’s Jalapeño is still only available as an engineering sample and needs some time before mass production.

Therefore, performance should genuinely be compared against Rubin, not Blackwell. In a sense, we initially expected custom chips like Jalapeño to surpass Blackwell. As we noted in last month’s analysis of NVIDIA and CoreWeave’s performance announcements, the performance-per-watt of Vera Rubin NVL72 is 5.4 times that of GB200 NVL72. We will later compare Jalapeño against Vera Rubin’s performance data from July.

Vera Rubin NVL72 vs. GB200 NVL72? Inference TCO and Architectural Analysis

Third, the tested models are not at the absolute cutting edge of open-source releases. NVIDIA and AMD have already published larger model results using AgentX, such as DeepSeek V4 Pro and Kimi K3. The larger the model and the newer its release, the more complex it is to get it running on new chips. That said, the models OpenAI has successfully run on Jalapeño are hardly small either.

Performance Analysis

OpenAI’s design target is performance-per-watt. The reason is straightforward: OpenAI is currently constrained by data center power availability, rather than budget or rack space, making tokens per megawatt critically important. At Computex 2026, Jensen Huang stated that performance-per-watt, reliability, and long lifespan are the core characteristics of future GPUs. His exact words were: “If you have one gigawatt of power, then throughput per watt is revenue.” He also mentioned that choosing the wrong architecture simply because a chip is cheaper makes no sense.

NVIDIA emphasized the same point in its Vera presentation at Hot Chips 2026, displaying the identical revenue chart: “Data centers today are power-constrained.” Power is paramount and drives revenue.

Operators cannot easily acquire additional megawatts. The timescales for adding GPUs versus expanding grid capacity differ vastly. Data center power caps are subject to numerous constraints, such as utility interconnection, infrastructure, cooling capacity, and UPS/backup power design. Grid delays frequently exceed hardware and construction progress, driving demand for behind-the-meter power capacity. This means building gas turbines and on-site generators locally at the data center. These capacities sit behind the utility meter and do not rely on the public grid. This allows operators to power facilities without waiting for grid interconnection and utility upgrades. This is precisely why xAI’s Colossus 2 relies heavily on behind-the-meter power, while its actual grid connection lags significantly further behind. For more information, refer to our energy modeling piece.

As we wrote in our X post, tok/s/MW simplifies to tokens per joule. Since a watt is a joule per second, tok/s/MW represents system efficiency and the ability to convert energy into tokens.

In this regard, even when compared to Rubin, Jalapeño emerges victorious. OpenAI’s Jalapeño achieves higher STP token throughput per megawatt than Vera Rubin’s MTP results. Those results were released by NVIDIA and CoreWeave in July. It also far exceeds GB200’s 2025 MTP results. As noted in our Vera Rubin analysis, comparing VR against 2025 GB200 results works because both are in similar early bootstrapping phases. Meanwhile, comparing against the 2025 GB200 keeps software maturity consistent. Following this logic, we compare three result sets: Vera Rubin’s latest July 2026 figures, GB200’s 2025 figures, and current Jalapeño results. This comparison is highly reasonable since these represent the best publicly available Rubin data. Furthermore, OpenAI taped out its own chip after Rubin. Both OpenAI and Rubin remain immature, meaning performance will continue to improve.

Regarding performance relative to Total Cost of Ownership (TCO), Vera Rubin and Jalapeño are neck-and-neck, yielding nearly identical output tokens per dollar. However, as previously mentioned, Jalapeño’s results do not use speculative decoding, whereas Vera Rubin’s results do. Speculative decoding can reduce per-token costs by roughly 3-5x. Once Jalapeño implements speculative decoding, its cost-effectiveness in delivering token services will increase. Of course, part of the TCO advantage comes from bypassing NVIDIA’s high profit margins in favor of Broadcom’s lower (though still substantial) margins. But this isn’t the whole story. For instance, Meta and Microsoft’s AI ASIC projects have invested longer without successful deployment. This shows that cost is only one variable in the equation. For a complete TCO breakdown of Jalapeño, refer to the SemiAnalysis AI Cloud TCO Model.

Architecturally, OpenAI chose not to split prefill and decode into different chip pools. Draft models and base models share the same silicon and interconnect structure. This design philosophy sacrifices some theoretical efficiency in exchange for practical operational convenience. The motivation is that workload composition changes over time. For example, the ratio of inputs, cache writes, cache reads, and output tokens has shifted significantly. This evolution occurred as we progressed through three model eras—knowledge, reasoning, and agents—as discussed in our recent articles. Therefore, predetermining fixed quantities for heterogeneous prefill silicon and decode silicon would lead to inefficiencies over time. In this architecture, OpenAI opted for a homogeneous pool and strives to make the chip perform well across all tasks.

And it indeed succeeded. On Kimi K2.5 (upon which Cursor Composer 2.5 is based), Jalapeño reaches nearly 700 tok/s/user. This speed is over nine times faster than the runner-up chip’s 100 tok/s/user.

On GPT-OSS, it’s another blowout. Jalapeño’s tokens-per-second-per-megawatt for equal-interaction throughput is nearly twice that of GB200’s highest throughput point, and over 50 times its concurrency-1 point. The higher-concurrency Jalapeño data points utilize EP8.

These results are impressive! However, we must nitpick: they are only on 8k1k configurations, which are much easier to tune, and there is no AgentX runtime data yet. As noted in our AgentX article, multi-turn and long-context workloads stress more aspects of the inference stack, such as routers and prefix caching. To excel in agentic workloads, further optimizations are required. See our AgentX article for more details.

AgentX - InferenceXv3: Can the CUDA Moat Still Hold in Agentic Inference?

Deep Dive into Specifications and Architecture

All these results come from Jalapeño’s A0 stepping, merely 9 months into the project. But the B0 stepping is already in the fab! Optimizations in B0 boost per-watt performance by approximately 25% over the earlier A0 silicon. Specifically, the B0 stepping achieves 13.4 PFLOPs of MXFP4 on a single reticle-limited compute die. The chip is manufactured on TSMC’s N3P node. By comparison, a single Rubin compute die achieves 17.5 PFLOPs of dense Rubin NVFP4 at a similar size and same process node.

Given Jalapeño’s TDP is only 700W, while each Rubin compute die runs at 900-1150W, this achievement is even more commendable. Because Jalapeño targets inference rather than training, OpenAI did not need to push TDP to maximize FLOPs. This is understandable. Regardless, the above figures indicate that Jalapeño delivers substantial peak theoretical FLOPs.

When directly compared to other accelerators, Jalapeño boasts the highest HBM bandwidth per watt and the highest FLOPs per watt. This level is comparable to the 1800W Rubin Max-Q configuration.

Jalapeño will feature HBM4, positioning it among the earlier adopters of this technology following NVIDIA and AMD, and even ahead of existing TPU and Trainium programs. One key architectural principle of Jalapeño is to fully leverage HBM bandwidth, so compromising with non-top-tier HBM would contradict this goal. This yields an onboard memory bandwidth of 15.4TB/s, surpassing all other commercially available accelerators using HBM3E. A bandwidth of 15.4TB/s indicates its HBM4 supports 10Gbps pin speeds, slightly higher than NVIDIA’s 9.6Gbps for HBM4 in Rubin. Samsung likely supplies the HBM.

OpenAI completed Jalapeño’s tape-out in November 2025, more accurately referring to the CoWoS design tape-out rather than just the top-level bare die. Within 9 months of that November tape-out—and only 3 months of debugging on actual silicon—OpenAI has already achieved excellent results with Jalapeño. Considering the team started the software stack from scratch, this is even more impressive.

Meanwhile, Rubin’s CoWoS tape-out was completed in October 2025, a month earlier. However, the only early results we have seen so far come from CoreWeave’s engineering samples. Unlike OpenAI, NVIDIA did not allow us to test and publish benchmarks, indicating its chip software remains immature. Given how quickly OpenAI can get new models running on its own silicon, the CUDA moat may have already evaporated.

They are still far from optimized, but we can clearly see that overall Jalapeño provides superior data. We do not believe NVIDIA’s hardware is inferior; rather, Jalapeño’s software debugging progressed faster than NVIDIA’s. This demonstrates the power of hardware/software co-design, which is precisely how top-tier frontier lab ASIC teams can surpass more mature commercial chip vendors. Counterintuitively, starting from scratch may have actually benefited OpenAI, as it allows for entirely new architectural decisions without being constrained by backward compatibility or legacy software versions.

Although OpenAI already possesses Jalapeño engineering samples, mass production is currently planned to ramp up gradually in 2027, with the majority of output scheduled for late next year. For more details on unit volumes and average selling prices, refer to the SemiAnalysis Accelerator Model.

It can be said that the OpenAI Jalapeño is a truly large-scale ASIC.

Compared to Rubin’s timeline, Jalapeño’s speed is astonishing. As shown earlier, despite Rubin getting a head start, Jalapeño’s results still surpass Rubin’s.

Jalapeño Architecture

Diving into the architecture, the chip’s matrix engine uses the MXFP numerical format and a weight-stationary systolic array, similar to TPUs. However, compared directly to TPUs, it supports smaller shapes/dimensions, meaning it avoids the strange performance cliffs associated with shape-mismatched matrix multiplications on larger systolic arrays.

It also features 64-bit scalar cores and FP32/INT32 vector cores. OpenAI incorporated redundant design at the tray level and embedded yield recovery at the core and channel levels. They claim that AI-assisted chip design reduced SIMD area by 8% and matrix engine area by 10%. While they did not specify exact Process/Voltage/Temperature (PVT) conditions, they noted that AI-assisted modules improved timing and power consumption compared to initial implementations.

Jalapeño’s architecture focuses on eliminating memory movement for KV caches and weights, as well as fixed latency and overhead, allowing it to approach raw peak compute/bandwidth even under small batch sizes or shapes, outperforming other accelerators in these regimes.

Cores and HBM are partitioned into multiple slices, with each core slice maintaining a low-latency local view of its own HBM slice. Synchronization between slices occurs via a high-bandwidth dedicated collect network. This minimalist memory hierarchy gives Jalapeño a massive potential advantage over GPUs, because GPU memory access must traverse complex memory systems, generating significant latency that must be amortized or hidden across large shapes.

This approach works because, through careful placement of weights and KVs, synchronization between cores can be confined to limited and known high-bandwidth communications, such as tensor parallel communication that can overlap with computation.

Additionally, there is an extra general-purpose NoC (Network-on-Chip) for general communication and accessing the scale-out fabric. Overall, by simplifying the NoC and memory subsystem, OpenAI saves significant power and achieves notable performance gains compared to NVIDIA and Google.

At the core level, OpenAI describes an out-of-order (OoO) core with an L1 cache. This starkly contrasts with patterns seen in other accelerators, which typically rely on software-managed registers, often paired with asynchronous DMA support. The argument here is that this allows Jalapeño to avoid fixed overheads like barrier latency. On other accelerators like GPUs, such overheads must be hidden or amortized by increasing the workload per core, making it harder to approach raw peak bandwidth/compute.

The trade-off is that Jalapeño thus relies on robust prefetching to ensure memory requests arrive on time, which is harder to predict and reason about. However, leveraging Codex to capture detailed traces within a solid framework makes finding optimal kernels with the best prefetching for given shapes possibly require almost zero human intervention. We believe this is exactly why OpenAI could get DeepSeek R1, Kimi K2.5, and GPT-OSS running so rapidly.

The cores also support “smaller” matrix dimensions, which (depending on how small) should make it more versatile across varying model and batch dimensions, becoming less sensitive to matrix dimension alignment, padding overhead, and tiling inefficiency. For instance, TPU, Trainium, and Etched chips possess very large systolic arrays and may require large batches or perfectly divisible model dimensions to avoid tiling inefficiencies.

With Jalapeño, OpenAI focuses on eliminating fixed latency in the system to approach roofline performance as closely as possible across all regions of the Pareto curve. Theoretically, this could yield relative advantages over GPUs at multiple operating points:

Peak performance for low-latency/small-batch inference significantly outperforms GPUs. GPUs are constrained by fixed overheads such as startup latency, barrier latency, and memory system latency.

Even with large batches or long contexts, there is potential to approach the hardware roofline more closely.

This comes with a caveat: even if peak performance exists theoretically, actual kernels might struggle to reach it. Thus, their approach seems to be:

Design for maximum peak performance across all workload shapes

Let Codex handle the heavy lifting to find kernels that achieve that peak

The OpenAI team extremely fast-tracks launching InferenceX workloads onto Jalapeño.

We are therefore optimistic about this methodology.

If Jalapeño succeeds, it will send a powerful signal.

The industry’s obsession with programming models and perfect universal compilers will be shattered by frontier AI models.

OpenAI writes Jalapeño kernels akin to writing assembly.

Each kernel contains manually tuned code, with some portions reaching around 3,000 lines.

It also includes correctness checks and custom sanitizer support.

Early kernel work involved humans-in-the-loop rather than being fully automated.

Later efforts shifted toward a more scaled internal version of Codex.

OpenAI plans to pitch this version to enterprise clients.

The internal serving engine is called "Teacup."

Fascinatingly, OpenAI previously lacked an internal MLA kernel implementation.

Not until they benchmarked DeepSeek using InferenceX.

Codex managed to quickly write usable, efficient kernels without requiring direct intervention from the kernel engineering team.

This showcases the development capability of the software pipeline.

OpenAI programs Jalapeño using Gluon.

Gluon is OpenAI’s kernel programming language.

Gluon is built atop Triton, preserving Triton’s SPMD (Single Program, Multiple Data) programming model.

Yet it exposes underlying programming abstractions.

For instance, targeting NVIDIA GPUs, it provides APIs mapped directly to PTX instructions.

Including MMA instructions, TMA instructions, mbarrier mechanisms, and more.

The most unique abstraction Gluon provides is layout.

In general, layout defines the mapping between hardware resources and tensor elements.

For example, register 5 of warp 9 corresponds to the tensor element at row 6, column 7.

Gluon’s layout abstraction is based on Linear Layouts.

This is a layout algebra invented by OpenAI.

Linear Layouts mathematically formalize what a layout is.

And provide tools to manipulate layouts.

This enables numerous features, such as formally provable-correct layout transformations.

As well as optimal memory swizzling.

In Jalapeño’s programming model, each Gluon program maps to persistent threads.

We interpret this to mean Jalapeño is suited for a persistent kernel programming paradigm.

Each program executes across multiple tiles, with work distribution handled by the programmer rather than the hardware scheduler.

OpenAI mentions TensorInfo.

It is an abstraction that explicitly encodes layout.

This is likely a curated set of layouts designed specifically for Jalapeño.

It will be driven by Linear Layouts.

Finally, each core offers data prefetching and decoupled out-of-order units.

For example, users can program waits for prefetched data.

The data is gated by semaphores.

Ironically, OpenAI models like GPT 5.6 Sol currently run on NVIDIA GPUs.

They are being used to design chips, posing a genuine threat to the CUDA moat.

NVIDIA’s own GPUs are actively powering its potential successors in real time.

Across temporal comparisons, we can also witness Jalapeño’s development velocity.

In under two weeks, throughput for certain interactive scenarios increased by over 2x.

Every tarball we receive from the Jalapeño team contains a wondrous world.

Not only did kernel performance improve, but the Jalapeño team enabled TP32 within just 8 days.

Expanding upon a prior TP8 configuration beyond a single system to achieve a full rack-level setup for running large models.

This development velocity is truly impressive.

To validate performance before deploying on real hardware, OpenAI also employs a simulator named "chilisim."

Its accuracy stays within 5% of actual hardware, utilizing a fixed-width trace bus.

Trace capabilities are limited on A0 but significantly improved on B0.

This improvement likely stems from actual runtime data collected on the A0 silicon.

Engineers demonstrated a Codex CLI running internal models.

It goes by the nickname "Raiku" or "5.3 Codex Spark," featuring a TPOT of 1.2ms.

The team also showcased demos written by Codex running directly on the chip.

Including 36 FPS Doom and FP32 fluid dynamics simulations.

Plus a "Liquid Light" mouse drag visualization.

Regarding model execution, OpenAI’s internal megakernel scheme is nicknamed "gigakernel."

It is built around a single megakernel that loops on-device to minimize CPU overhead and startup time.

The team continues pushing forward on compile-time scheduling strategies.

Internally, special attention is focused on how to coordinate the usage of million-rollout executions.

To Disagg or Not to Disagg: That Is the Question

We previously mentioned that OpenAI does not employ prefill-decode disaggregation on these chips.

This surprised us, given that NVIDIA and AMD GPUs derive significant performance benefits from Prefill-Decode Disaggregation (PDD).

Even when operating on homogeneous hardware.

Let’s dig deeper into why the Jalapeño team made this choice.

When workloads are static, prefill-decode separation appears highly attractive.

Prefill and decode place different stresses on hardware.

Assigning each phase to separately tuned pools can improve efficiency under selected input-output ratios.

But production traffic rarely maintains that ratio.

Input/output sequence lengths, concurrency, cache hit rates, speculation acceptance rates, and latency targets fluctuate throughout the day.

Once devices are split into prefill and decode pools, excess prefill demand leaves decode chips idle while requests queue up.

Conversely, excessive decode demand causes the reverse problem.

Operators must constantly predict the correct split, reserving spare capacity on both sides.

And continuously rebalance a system where the ideal ratio is constantly shifting.

In a unified system, certain resources may be underutilized during specific phases.

But every device remains available to serve the next request.

In a disaggregated system, entire chips might sit idle solely because they belong to the wrong pool.

Local utilization looks better, but global utilization could be poor.

Disaggregation also breaks locality.

Prefill workers generate massive KV caches that decode workers require immediately.

The system must transmit this state across the network for generation to proceed.

This increases bandwidth consumption, synchronization overhead, queuing latency, and introduces another failure domain.

Costs also rise with input sequence length as the KV cache expands.

However, avoiding KV movement is primarily a power and latency optimization.

Willingness to move some KV improves hardware utilization at the cost of increased power consumption and per-request latency.

Interchangeable clusters can shift capacity between latency-sensitive requests and throughput-oriented batches.

Fixed splits leave hardware stranded when traffic composition shifts.

Furthermore, context length alters the balance between attention and FFN compute.

Any fixed hardware ratio is only efficient near its design point.

The same limitations apply to speculative decoding. Draft models must deliver candidate tokens to validators with extremely low latency. Splitting them into separate dedicated pools turns a tightly coupled decoding loop into a distributed protocol. Extra communication and coordination overhead could easily erase the latency savings gained from drafting. Placing both models on the same device and low-latency fabric is necessary to preserve the locality that makes speculation worthwhile.

Nevertheless, in scenarios where demand is sufficiently large, stable, and predictable, decoupling can still win out. Especially when traditional GPUs require large batches and staged processing to achieve decent throughput. But it’s never a free lunch.

From Japanese to Indian (Mild to Extra Spicy): How Katsu, Vindaloo, and Chana Curry Dishes Assemble a Rack System?

The Jalapeño system at the rack unit level consists of CPU host racks and ASIC racks. The host rack houses 16 host CPU trays named "Katsu." Each Katsu corresponds to one of the 16 ASIC trays on the right side, named "Vindaloo." Each host is equipped with dual Turin-class AMD EPYC CPUs, 1.5TB DRAM per rack, 2x E1.S, and 2x M.2 SSDs per rack. Each tray also features 400G (2x200G) front-end networking. Every Katsu tray connects to each Vindaloo tray via 8 external PCIe DAC cables. These cables route horizontally across the front of the rack. The system-level design was developed in collaboration with Celestica.

The ASIC rack comprises 16 Vindaloo trays and 8 scale-out switch trays (6 local + 2 global) named "Chana." Each Vindaloo tray holds 8 Jalapeño ASICs, totaling 128 Jalapeño ASICs per rack. The ASICs connect to each Chana switch tray via copper backplane cables, similar to NVIDIA’s Oberon architecture. The scale-out topology divides into a local domain and a global domain. The local domain covers the 128 ASICs within the rack. The global domain connects up to 16 racks or 2,048 ASICs. We will explain the bandwidth and topology in greater detail below.

The side-mounted host rack draws approximately 50kW (31kW in production), while the ASIC rack consumes 130kW. The entire dual-rack system totals roughly 160kW. In terms of power draw, this essentially matches a dual-width GB300 rack.

OpenAI can connect up to 2,048 Jalapeño XPUs within a single scale-out network. The scale-out network consists of two domains. The local domain connects all 128 XPUs within the rack via the backplane. The global domain uses a hybrid of copper and optical interconnects to link 2,048 XPUs across 16 racks. Each rack contains 8 Chana switch trays. The middle 6 Chana switches handle the local domain. Each carries a single 102.4T Tomahawk 6 switching ASIC. The top and bottom 2 Chana switches handle the global domain. We believe these are likely 2x 102.4T Tomahawk 6 switches per tray, peaking at 204.8T per switch tray.

In the local domain, the unidirectional bandwidth per XPU across the 128 Jalapeño chips is 4.8Tb/s. They connect in a fully connected manner to six 102.4Tb/s Tomahawk 6 ASICs. This equates to 48 pairs of differential pairs (DP) male/female connectors per XPU. In total, this requires 6,144 DP passive copper cables for local scale-out per rack.

In the global domain, 2,048 XPUs across 16 racks connect via copper backplanes, electrical 204.8T TH6 switches, 1.6T optical transceivers, and optical circuit switches. The unidirectional bandwidth of the global link per XPU is 1.6Tb/s. This means each XPU utilizes 16 pairs of DP male/female connectors for the backplate between the XPU and the global switch. Each global switch tray houses 2 ASICs, with their egress bandwidth divided between the backplane and front-panel optics.

Bridging the local and global domains, the backplane connector count per rack amounts to 64 pairs of DP per XPU. This totals 8,192 pairs of DP passive copper cables per rack.

The global domain employs a pure rail architecture consisting of 8 rails within the global domain. We speculate OpenAI routes optical links in the global domain via Optical Circuit Switches (OCS) installed in each rack. Each XPU’s 1.6Tb/s global bandwidth reaches the global switch tray via the copper backplane, then exits the switch through the front panel’s 1.6T optical modules. It first enters the passive optical switch before leaving the rack. This enables the scale-out domain to reach 2,048 XPUs, composed of 16 racks with 128 XPUs each.

Since the scale-out network accounts for only about 10% of total system costs, this flexibility buys valuable optionality for future 10T to 20T parameter models or 2M to 4M token context windows. Regarding deployment, OpenAI partners with neocloud. It is also gathering reliability data with data center partners before January while optimizing deployment timelines from dock to rack.

Next Steps

Next, we discuss the future of Jalapeño as its first production tokens approach. The next milestone is 100MW, with hurdles primarily in hardware: how many they can produce, how well they can deploy and operate data centers, and how they handle monitoring and resilience. Software validation is complete, and with internal models, any software first-mover advantage will be easily caught up. Behind the paywall, we discuss the impact on chip companies like NVIDIA, AMD, and Cerebras, which signed contracts with OpenAI for the coming years.

We also cover yield, volumes, and schedules for next-generation chips in our Accelerator Model.

Subsequent sections contain premium content that has not been further compiled due to permission restrictions. Nevertheless, the preceding text already provides substantial information reflecting OpenAI’s challenge to NVIDIA’s chips. For investors and analysts tracking the semiconductor sector and NVIDIA’s stock, these insights will offer fresh and solid reference points.