A rack that behaves like one GPU.

Normally, when you want more computing power than one machine can give you, you buy more machines and connect them with a network. The machines stay separate: each one has its own memory, and moving data between them is slow enough that you have to plan around it.

This rack does something different. All 72Blackwell GPUs per rack NVIDIA of its GPUs are joined by a dedicated interconnect that is fast enough that a program can treat another GPU's memory almost like its own. You do not send a message and wait; you just read the address. That difference — not the raw component count — is what the whole design is for.

The instinct on seeing 72 accelerators in one enclosure is to read it as a small cluster that happens to share a chassis. That reading gets the design backwards. In a cluster, the unit of composition is the node: each has private memory, and inter-node traffic crosses a network stack with its own protocol, its own congestion behaviour, and bandwidth an order of magnitude below local memory. You partition your model to avoid that boundary.

Inside an NVL72 there is no such boundary to avoid. Every GPU has 1.8 TB/sNVLink 5 bandwidth per GPU Bidirectional: 18 links × 100 GB/s. About 14× PCIe Gen5. NVIDIA of NVLinkNVLinkNVIDIA’s GPU-to-GPU interconnect — on NVL72 it carries 1.8 TB/s per GPU, about 14× PCIe Gen5. bandwidth to the fabric, and because each of its 18NVLink ports per GPU NVIDIA ports lands on a different switch ASIC, that bandwidth is available to any peer simultaneously. The fabric is non-blockingnon-blockingEvery port can run at full rate simultaneously — no traffic pattern can starve another.: no traffic pattern degrades another. Aggregate in-rack bandwidth is 130 TB/sAll-to-all NVLink bandwidth in-rack NVIDIA .^{NVIDIA,NVIDIA}

NVIDIA's own framing is that the rack "acts as a single, massive GPU".^NVIDIA Marketing language, but in this case it describes the programming model accurately: ~13.5 TBPooled HBM3e per rackSources disagree: 13.4 – 13.8 TB Sources differ on rounding, on physical versus usable-after-ECC capacity, and on SKU. NVIDIA’s 192 GB per GPU gives 13.82 TB physical and 12.96 TB after ECC; Supermicro’s datasheet quotes up to 372 GB per Superchip — 186 GB per GPU — which is where the 13.4 TB figure comes from. NVIDIA · Supermicro · SemiAnalysis of HBM3eHBM3eHigh Bandwidth Memory — DRAM stacked vertically next to the GPU die, trading capacity for enormous bandwidth. in one address space, coherent, reachable by ordinary load and store instructions.

The port arithmetic is worth doing explicitly, because it is the entire argument. Each Blackwell GPU exposes 18 NVLink 5 ports at 100 GB/s bidirectional. The rack contains 9NVSwitch trays ServeTheHome NVSwitch trays of two ASICs each — 18NVSwitch5 ASICs 2 per switch tray. NVIDIA ASICs, exactly matching the port count per GPU. Wire port n of every GPU to switch n and you get a single-layer fabric with no oversubscription and no second tier: 72 endpoints × 18 ports, against 18 switches × 72 ports. There is no topology to be clever about, no rail assignment, no bisection ratio to derate collectives against.

What that removes from the performance model matters more than what it adds. Tensor and expert parallel dimensions can be chosen for numerical and load-balance reasons rather than to keep chatty dimensions inside a node boundary. The classic "TP within the node, PP across nodes" heuristic has no node boundary to respect until you reach the rack edge.