The Blackwell GPU

Chips are printed by projecting a pattern onto silicon, and there is a hard limit to how large a single pattern can be — the "reticle limit". Blackwell hit it. Rather than accept a smaller chip, NVIDIA printed two of the largest possible dies and joined them edge to edge with a link so fast that software cannot tell them apart.

The result behaves as one GPU with 208 billionTransistors per Blackwell GPU 104 B per die, two dies per package. NVIDIA transistors and 192 GBHBM3e per Blackwell GPU 180 GB usable after ECC. NVIDIA of very fast memory attached. You do not schedule work onto "die 0" or "die 1"; you just see one device.

The two dies are TSMC TSMC 4NPProcess node NVIDIA , 208 billionTransistors per Blackwell GPU 104 B per die, two dies per package. NVIDIA in total, joined by NV-HBINVLinkNVIDIA’s GPU-to-GPU interconnect — on NVL72 it carries 1.8 TB/s per GPU, about 14× PCIe Gen5. at 10 TB/sNV-HBI die-to-die link Makes the two reticle-limited dies present as one cache-coherent GPU. NVIDIA . That bandwidth is the whole trick: it is high enough, and low enough latency, that the pair present as a single cache-coherent device to CUDA. There is no NUMA domain to schedule around, no explicit die affinity, no partitioning of the L2 — 126 MBL2 cache per GPU NVIDIA of it is shared.^NVIDIA

Attached is 192 GBHBM3e per Blackwell GPU 180 GB usable after ECC. NVIDIA of HBM3eHBM3eHigh Bandwidth Memory — DRAM stacked vertically next to the GPU die, trading capacity for enormous bandwidth. at 8 TB/sMemory bandwidth per GPU NVIDIA , of which 180 GB is usable after ECC. Hold that bandwidth number next to the FP4 figure of ~20 PFLOPSPeak FP4 per GPU Sparse. NVIDIA and the tension in the design is immediately visible: about 20,000 TFLOPs against 8 TB/s is an arithmetic intensity requirement of roughly 2,500 FLOPs per byte before the tensor cores are the limit. Very few real kernels are anywhere near that.

Two consequences worth being precise about.

The NV-HBI link is a bandwidth solution to a reticle problem, not a latency-free one. Coherence across the die boundary is maintained in hardware, but a cache line sourced from the far die still costs more than a local one; kernels with pathological cross-die access patterns will see it. In practice the scheduler's locality heuristics absorb this, and it is not exposed as a tunable — which is a deliberate choice to keep the programming model simple, and a real constraint if you were hoping to control it.

The 192 GB / 180 GB distinction accounts for a chunk of the disagreement over the rack's pooled HBM total: 72 × 192 GB is 13.82 TB physical, 72 × 180 GB is 12.96 TB usable, and published figures land anywhere in between depending on which the source meant and how it rounded. The site shows ~13.5 TBPooled HBM3e per rackSources disagree: 13.4 – 13.8 TB Sources differ on rounding, on physical versus usable-after-ECC capacity, and on SKU. NVIDIA’s 192 GB per GPU gives 13.82 TB physical and 12.96 TB after ECC; Supermicro’s datasheet quotes up to 372 GB per Superchip — 186 GB per GPU — which is where the 13.4 TB figure comes from. NVIDIA · Supermicro · SemiAnalysis with the range attached rather than picking one.

The capacity column is the one that changes deployment decisions. A trillion-parameter model at BF16 needs 2 TB of weights and does not fit in a single rack's HBM alongside activations and KV cacheKV cacheThe stored keys and values from every previous token, re-read on each decode step.. The same model at FP4 needs 500 GB and leaves room to work in. NVFP4 — NVIDIA's block-scaled 4-bit format, which attaches a shared scale factor to small groups of values — exists to make that trade survivable numerically.

It is not free, and the honest framing is that FP4 is a quantisation decision with an accuracy budget, not a free doubling. NVIDIA's own MLPerf submissions are the evidence that the budget can be met on real models: up to 3.2×Blackwell NVFP4 training speedup vs Hopper FP8 MLPerf Training v5.1, Llama 3.1 405B, at the same GPU count. NVIDIA Technical Blog on Llama 3.1 405B training at equal GPU count.^{NVIDIA Technical Blog}