Fast Workstations Need Fast Buses: Inside PCIe Gen 4

Advances in workstation performance seem to be as certain as the sunrise. The dynamic duo of faster and more powerful CPUs and GPUs always get the attention for such advances. Just as a star quarterback can’t win without a team of position players, the CPU/GPU duo can’t “win” without help from an overlooked component that makes all these advances possible—the PCIe interface. 

Officially known as peripheral component interconnect express, PCIe is the computer expansion bus standard, the successor to generations of expansion slots that have been inside personal computers since the Apple II. The current shipping model is PCIe 4.0. 

How important is PCIe? Legendary graphics expert and industry analyst Jon Peddie notes, “PCIe freed [graphics processing units] from the tyranny of buses with a scalable architecture that is still delighting and surprising us 17 years later and with no end in sight.”

The fourth generation of the PCIe standard started shipping in workstations in early 2021. For years, the upgrade pace was slow as a reflection of industry demand but more recently development has been faster. 

“It took us seven years to move from Gen 3 to Gen 4,” says Al Yanes, the president and chairman of PCI-SIG, the standards association in charge of PCIe. “In the last couple of years we have been on steroids … “we want to double the bandwidth every three years.” 

PCIe Gen 4 essentially runs twice as fast as PCIe Gen 3. PCIe Gen 4 offers 16 gigatransfers per second, double that of Gen 3. The doubling extends down to the individual lanes. 

Point-to-Point Connections

Originally PCI was a shared bus architecture; the PCI host (the slots) and all devices share a common set of addresses, data and control lines. PCIe advanced the standard by moving to a point-to-point topology. Separate serial links connect each device to the host, which allows for full-duplex communications not possible in the old standard. 

The link between the motherboard and the peripheral or component is expressed in lanes. Some devices only need one lane; some need several. PCIe Gen 4 supports link widths of x1, x2, x4, x8, x12, x16 and x32. 

This flexibility allows the standard to support low-bandwidth devices and such bandwidth devouring devices as the latest graphics processing units (GPUs) or Ethernet 10. Newer generation PCIe buses support devices built for previous generations, which encourages continuity of product use. 

Workstation vendors are now shipping a few models that support PCIe Gen 4. The move will take time, because not only must the motherboard support Gen 4, but all the necessary peripherals—including the CPU, memory and storage—must also support the standard. 

“To get the benefit of Gen 4, all the peripherals must be compatible,” says Matt Allard, director of strategic alliances at Dell Technologies. 

In the increasingly competitive race between AMD and Intel, AMD was first out of the gate with workstation-class CPUs that supported PCIe Gen 4, the Ryzen 3000 line. Intel was close behind with its Gen 11 CPUs that support the standard, making for an ample supply of CPU support. 

When Gen 4 Makes Sense

There is no reason to throw out all workstations built before PCIe Gen 4, Allard says. 

“We recognize that customers have budgets. Our new mobiles, like our new 17-in. thin and light, offer the choice of Gen 3 or Gen 4. Customers can make knowledgeable choices.” 

By assigning multiple lanes to devices as needed, devices that need the most bandwidth can get it. Mainstream workstation users may not need the extra speed boost. 

Where PCIe Gen 4 makes the most sense is for large data sets, like very large CAD models or complex CAE jobs. 

“3D CAD workflows using complex textures will benefit from Gen 4, as will real-time workflows for virtual reality and LiDAR point cloud scans,” says Mano Gialusis, a senior product marketing manager at Dell. “People who use large datasets love [PCIe Gen 4].” 

Team members creating visualizations from large models will also see a speed increase using Gen 4. 

“8K video is the top end today, [so] not that uncommon in engineering,” says Gialusis. 

If the user must create a dissolve between scenes, it requires two high-bandwidth streams running simultaneously. In a Gen 3 workstation, the software would have to lower resolution or frame rate to get top performance. Gen 4 allows the rendering to continue at full speed and full fidelity.

By moving up to PCIe Gen 4 Gialusis says Dell can now offer a 4 TB M.2 storage device in their mobile workstations. Each Gen 4 M.2 requires three lanes, making it possible for Dell to ship a mobile with 12 TB storage. PC World ran benchmarks on early PCIe Gen 4 peripherals. A Gigabyte Aorus M.2 solid-state drive (Gen 4 compatible) hit speeds 35% higher than the magazine had seen with Gen 3 devices. 

One more reason to like Gen 4 is its ability to run faster while consuming less energy than Gen 3. AMD emphasizes energy consumption as a competitive advantage over Intel, and has aligned its research and development strategy with the availability of PCIe Gen 4 to help lower system-wide energy use. 

What the Future Holds 

PCIe Gen 4 just started shipping in 2021, and already the next two generations are nipping at its heels. “Gen 4 is past us. The Gen 5 spec has been released for compliance review, and the spec for Gen 6 will be out by the end of the year,” says PCI-SIG’s Yanes.

Demand for bandwidth is rising rapidly, so the SIG is pacing its development work to match. In years past graphics seemed to drive the need for speed, but “now there are accelerators, machine language and artificial intelligence” driving the need for increased bandwidth, Yanes says. 

Gen 5 and Gen 6 PCIe will offer multiple choices for hardware developers, as PCIe extends into cellphones, low-power devices and automotive. Gen 6 will be of specific interest to the automotive industry; the twin titans of automotive R&D—electric propulsion and autonomous driving —need as much bandwidth as possible. 

“The need for speed, for information transfer, for calculation, all that stuff plays in our space,” says Yanes. “Years ago we were not low power. Now PCIe is fast and low power.”

More Dell Coverage

Artificial Intelligence for Design and Engineering Workflows

In this white paper, learn how artificial intelligence and machine learning can improve design and simulation.

More Jon Peddie Research Coverage