People shopping multi-GPU dedicated servers focus on the cards and ignore the bus. The PCIe topology that connects those cards is often the silent bottleneck of a multi-GPU rig. A four-GPU server where every card runs at x4 lanes is slower than a two-GPU server where each card gets x16, even with twice the cards on paper.
Contents
Basics
A GPU connects to the motherboard through PCIe at a width (x16, x8, x4) and a generation (Gen 3, 4, 5). Bandwidth doubles with each generation at the same width. A Gen 4 x16 link gives ~32 GB/s per direction. Gen 5 x8 gives the same. Gen 4 x4 gives ~8 GB/s per direction – a quarter of full x16.
Common Topologies
| Chassis Shape | Per-GPU Bandwidth |
|---|---|
| Workstation, 2 GPUs | Often x16/x16 Gen 4 or 5 – ideal |
| Consumer board, 2 GPUs | Sometimes x16/x4 – one card starves |
| Server, 4 GPUs | x8 each Gen 4 or 5 – good |
| Server, 8 GPUs | x8 each Gen 5 – good if Gen 5, poor if Gen 4 |
| Rack with PLX switch | Shared upstream – fine for inference, contested for training |
Performance Impact
For single-GPU inference, PCIe barely matters – weights load once, math happens on-card. For tensor parallel inference, every forward pass all-reduces across cards and PCIe bandwidth directly affects tokens/sec. An x16/x16 Gen 4 setup delivers close to theoretical tensor-parallel speed. An x4/x4 setup on the same cards can be 30-50% slower on the same workload.
For training, PCIe is more often the limit. Gradient all-reduce during backprop hits PCIe hard. Gen 5 x8 is a meaningful upgrade over Gen 4 x8 for FSDP or DeepSpeed workloads.
Multi-GPU Chassis With Honest Topology
We publish the PCIe layout of every multi-GPU server – no buried x4 lanes starving your tensor parallel.
Browse GPU ServersWhat to Ask For
When ordering a multi-GPU server, ask three things:
- PCIe generation per slot (Gen 4 vs 5)
- Lane width per slot (x16, x8, x4)
- Whether the GPUs share a switch (PLX) or connect direct to CPU
Direct-to-CPU x8 Gen 5 is generally better than shared x16 Gen 4 through a switch for training. For inference-only deployments a PLX switch is often fine. See NCCL tuning and GPU interconnect options.
