I/O Interfaces & Buses
In AI systems with many GPUs, why are dedicated high-speed GPU-to-GPU links (such as NVIDIA NVLink) used in addition to PCI Express?
- APCI Express cannot connect even one GPU
- BA dedicated link makes GPU memory unnecessary
- CTraining large models exchanges huge amounts of data between GPUs, needing more bandwidth than PCI Express
Show answer and explanation
Answer: C. Training large models exchanges huge amounts of data between GPUs, needing more bandwidth than PCI Express
When multiple GPUs train a large AI model, they frequently exchange gradients, parameters and intermediate results mid-computation. If this communication is slow, the GPUs stall waiting. Dedicated GPU-to-GPU links bundle many fast SerDes lanes so GPUs can exchange data directly with several times the bandwidth of PCI Express.
- A. GPUs normally connect via PCI Express.
- B. GPU memory is still needed.
- C. Correct. Inter-GPU communication easily becomes the bottleneck.