A Beginner’s Guide to NVIDIA Spectrum and Spectrum-X Networking
Spectrum vs ordinary switching, fabric thinking, history, and AI motivation
- Part 1 — What NVIDIA Spectrum Is
- Part 2 — Why NVIDIA Spectrum
- Part 3 — AI Factory Fabric Architecture ← You are here
- Part 4 — RDMA and ROCE
- Part 5 — NOS, IaC, and Operations
- Part 6 — HA, Physical Layer Design, and Scaling
- Part 7 — Multi-Cloud, Security, and Lifecycle Management
- Part 8 — Design Scenarios, Glosary, and References
Large-Scale Fabric Architecture
Small leaf-spine
Figure 7. Small two-tier leaf-spine fabric
Figure 8. Pods connected through an additional tier or DCI layer
Pods are useful fault and scaling domains. However, adding a network tier also adds latency, cost, optics, and failure modes. Modern Spectrum-X Multiplane is specifically aimed at extending two-tier scale so that designers can avoid an extra tier for some very large AI fabrics.
Three-stage Clos
Figure 9. Three-stage Clos abstraction
Dual-homed servers
Figure 10. Dual-homed endpoint connectivity
A dual-homed server can use MLAG/bonding or independent Layer-3 paths depending on design. AI back-end networks increasingly favour explicit multi-rail or multiplane connectivity because separate rails can map more predictably to GPU/NIC topology and failure domains.
Border and service leaves
Figure 11. Border/service connectivity
Scaling questions to ask
- How many endpoints and NIC ports exist today, and at the next growth step?
- What host bandwidth must remain available after a link or spine failure?
- Is the fabric non-blocking, intentionally oversubscribed, or workload-aware?
- How many ECMP paths exist between any two racks?
- What is the largest failure domain: link, switch, rack, pod, or data centre?
- Can the cabling system physically support the required number and reach of links?
- Does the design preserve operational simplicity as it grows?
AI Factory and GPU Cluster Networking
Distributed AI converts the network from shared infrastructure into a performance-critical accelerator interconnect. During training, GPUs repeatedly exchange gradients, activations, parameters, or expert-routing data. During distributed inference, model parallelism and disaggregated components can also generate latency-sensitive east-west traffic.
Common traffic patterns
- All-reduce: workers combine values and redistribute the result. Common in data-parallel training.
- All-to-all: every participant sends data to many or all peers. Common in mixture-of-experts and some shuffling stages.
- Incast: many senders converge on one receiver or one output port simultaneously.
- Elephant flows: long, high-volume flows that can dominate links for sustained periods.
- Synchronized bursts: many GPUs begin the same communication phase at nearly the same moment.
Figure 12. Eight GPU servers through leaf-spine
Why collective communication is sensitive
Collectives behave like team sports: the phase completes when all required participants finish. A single hot path can therefore hold up many otherwise idle GPUs. The network metric that matters is not merely link rate; it is effective delivered bandwidth and consistency at the job level.
Figure 13. Simplified ring all-reduce
Figure 14. Congestion hot spot versus balanced paths
NCCL in one paragraph
NVIDIA Collective Communications Library (NCCL) implements GPU communication collectives and chooses transport/path strategies based on the system. NCCL does not replace the network; it exposes the network. Poor topology, congestion, NUMA placement, NIC configuration, or link health can become visible as lower NCCL bandwidth and GPU utilization.
Source note: See references [18], [19].
Routing and Control Plane
Why Layer 3 is the default mental model
Large fabrics usually prefer routed point-to-point links because Layer 3 limits broadcast domains, removes many spanning-tree dependencies, enables ECMP, and makes failure domains explicit. BGP is popular because it scales, is well understood, and fits a topology where every leaf peers with every spine.
Figure 20. L3 leaf-spine underlay
eBGP underlay
In an eBGP underlay, switches use external BGP sessions even though they may belong to one organization. Each node or tier can use distinct autonomous system numbers. BGP distributes loopbacks and endpoint prefixes while ECMP installs multiple equal-cost next hops. BFD can detect forwarding failures faster than waiting for standard routing timers.
EVPN/VXLAN overlay
Figure 21. EVPN/VXLAN overlay over an IP underlay
EVPN provides a BGP control plane for distributing MAC/IP and VTEP reachability information, while VXLAN encapsulates tenant traffic across the IP fabric. This is useful for multi-tenant cloud, virtualized networks, and workload mobility. A dedicated AI back-end network may not need an overlay at all; a pure routed fabric can be simpler and easier to troubleshoot.
Anycast gateway
Figure 22. Distributed anycast gateway
Source note: See references [22], [23].