A Beginner’s Guide to Modern Ethernet Fabrics for HPC and AI Environments
vendor-neutral fundamental, packet walks, AI/HPC design context , and troubleshooting
- Part 1 — The Foundations
- Part 2 — VXLAN and EVPN
- Part 3 — Distributed Networking Services
- Part 4 — How The Fabric Works
- Part 5 — HPC and AI Environments ← You are here
- Part 6 — Operations, Automation, and Troubleshooting
- Part 7 — Design Examples, Glosary, and References
EVPN/VXLAN in HPC and AI Environments
Why AI and HPC Networks Are Different
Enterprise applications often have diverse traffic patterns that average out statistically. Distributed AI training can be much more synchronized. Collective operations such as all-reduce cause many GPUs to exchange data in coordinated phases. Large flows can start together, fill queues rapidly, and create incast or congestion hotspots. A single slow path can delay an entire collective operation.
HPC has long had similar sensitivity to latency, jitter, and collective communication. AI adds extremely high accelerator density, multiple high-speed NICs per server, scale-out storage, checkpoint bursts, and clusters that can span thousands of accelerators. The network becomes part of the parallel computer rather than merely a way for servers to reach applications.
- Massive east-west traffic: GPU-to-GPU and node-to-node communication dominates many jobs.
- Synchronized bursts: many senders can transmit at once after the same compute phase.
- Elephant flows: a small number of huge flows can dominate path utilization.
- Incast: many sources converge on a receiver or link and build queues rapidly.
- Storage pressure: checkpoint and dataset traffic may compete with compute traffic if fabrics are converged.
- Failure amplification: one failed or degraded link can slow a large job even when ordinary application connectivity still works.
- Topology sensitivity: collective libraries can perform better when communication maps well to physical rails and switch hierarchy.
Why this matters
A “connected” AI fabric can still be a poor AI fabric. Throughput distribution, tail latency, congestion signals, loss behaviour, and path symmetry matter as much as reachability.
Where EVPN/VXLAN Fits in an AI/HPC Data Centre
An AI or HPC data centre usually contains several network roles. Not all of them need the same control plane or encapsulation. EVPN/VXLAN is especially useful where logical segmentation, tenant isolation, distributed gateways, workload mobility, or integration with a general private-cloud environment matters. The hottest GPU-to-GPU compute path may instead use a simpler routed RoCEv2 fabric or InfiniBand, depending on performance and operational requirements.
Figure 16 – One facility can use different network architectures for service, management, storage and accelerator traffic.
A common architectural split is: a management fabric for BMCs, provisioning and telemetry; a service or front-end fabric for user/API traffic and enterprise integration; a storage fabric for datasets and checkpoints; and a compute fabric optimized for accelerator communication. Some designs converge several roles on one high-performance Ethernet fabric, but convergence increases the importance of QoS, congestion management, failure isolation, and observability.
| Network role | Why EVPN/VXLAN may help | Why a simpler routed fabric may be preferred |
| Service/front-end | Multi-tenancy, VRFs, shared services, distributed gateways | If segmentation is simple and no overlay is needed |
| Management | Tenant/project separation and automation at scale | Management often values simplicity over overlay features |
| Storage | Segmentation and routed reachability across racks | Dedicated storage paths may prioritize deterministic performance |
| GPU compute | Can provide tenant/logical services in Ethernet AI fabrics | Lowest-complexity routed RoCEv2 or InfiniBand may be preferred for the hottest path |
| Kubernetes platform | Maps well to tenant/cluster network virtualization | CNI may implement its own overlay, creating unnecessary double encapsulation if design is careless |
EVPN/VXLAN and RoCEv2/RDMA
Remote Direct Memory Access (RDMA) allows one system to access memory on another system with very low CPU involvement and low latency compared with traditional socket data paths. RoCE (RDMA over Converged Ethernet) carries InfiniBand transport semantics over Ethernet. RoCEv2 adds UDP/IP encapsulation, making the traffic routable across Layer-3 IP networks. The standard IANA UDP destination port is 4791.
This solves a different problem from VXLAN. RoCEv2 is a transport used by applications and RDMA libraries. VXLAN is an overlay encapsulation used by network infrastructure. EVPN is a routing/control-plane information model. A network can carry RoCEv2 with plain routed IP and no VXLAN. A data centre can also run EVPN/VXLAN for service networks while keeping the GPU compute fabric as a dedicated routed RoCEv2 network.
| Concept | Primary purpose | Typical plane |
| EVPN | Distribute MAC/IP/prefix and service reachability | Control plane |
| VXLAN | Encapsulate overlay traffic across IP | Data-plane overlay |
| RoCEv2 | Carry RDMA transport over UDP/IP/Ethernet | Application/transport data path |
| PFC | Per-priority link-level pause mechanism used in some lossless designs | Ethernet QoS/flow control |
| ECN | Mark congestion without necessarily dropping packets | IP congestion signalling |
High-performance RoCEv2 designs pay close attention to MTU, queueing, QoS classification, ECN, and sometimes Priority Flow Control (PFC). PFC can pause a selected priority on a congested Ethernet link instead of stopping all traffic, but poor configuration can propagate congestion or create pause storms. Modern designs increasingly rely on careful end-to-end congestion control rather than assuming “enable PFC everywhere” is sufficient.
If RoCEv2 is carried inside VXLAN, the design must also preserve enough MTU and useful hashing entropy, and it must ensure QoS markings are mapped consistently between inner and outer headers. Some platforms explicitly support RoCE over VXLAN, but support and performance characteristics are vendor- and generation-specific. For the most latency-sensitive compute rail, avoiding unnecessary encapsulation can simplify performance engineering.
Common misconception
EVPN does not make Ethernet lossless. VXLAN does not make RoCE reliable. RoCEv2 does not require EVPN. These technologies can be combined, but each solves a separate problem.
AI Fabrics, Rail-Optimized Designs, and Scale
Large GPU systems frequently expose multiple high-speed NICs or SuperNICs per compute node. A rail-optimized topology connects corresponding NIC positions across nodes into consistent network planes or paths. The goal is to give collective communication predictable bandwidth and reduce cross-rail bottlenecks.
Figure 17 – Rail optimization tries to align multiple server NICs with repeatable network paths.
NCCL-style collective communication can exploit multiple NICs and paths, but the network must provide enough entropy and avoid persistent hotspots. BGP/ECMP gives the base fabric multipath reachability. Adaptive routing, congestion telemetry, and NIC-aware path selection can improve behaviour beyond static hashing. Current Ethernet AI reference architectures often use full or near-nonblocking leaf-spine/fat-tree designs and rail-oriented attachment for this reason.
EVPN can still provide valuable tenant and service segmentation around an AI cluster. The architectural question is whether the compute path benefits from the overlay features enough to justify extra encapsulation and state. In some AI factories the answer is yes, especially for converged or multi-tenant Ethernet. In others, the compute fabric is deliberately a simpler routed domain while EVPN/VXLAN serves the surrounding cloud, storage, or service networks.