DE · Topics ·

Getting the Bigger Performance Picture

Professional benchmark software only tells part of a workstation’s performance story.

Professional benchmark software only tells part of a workstation’s performance story.

Holomark 2 is created by Rhino 3D users. The Cooper Mini model in the benchmark has been modified in hundreds of ways by Rhino users. Image courtesy of Robert McNeel and Associates.

Professional engineering workstations provide a high level of power and performance for engineering, simulation and design software. The trick is to make sure any workstation you consider for purchase is a good fit for your specific requirements.

A variety of benchmarking software tools are available to evaluate workstations. They test specific performance issues, but they don’t tell the whole story of how any model will run in all use cases. It is generally a good idea to use a combination of different benchmarking tools to get a comprehensive understanding of a workstation’s performance. There is a list of the most popular and valuable workstation benchmarks at the end of this article.

The workstation vendors we spoke with recommend these standard benchmarks but are quick to point out they can’t tell the whole story.

“Even the SPECapc benchmarks only test a subset of the available features in each application,” says Brian Bothwell, advisory performance engineer at Lenovo. “While a good approximation of performance can be made, I don’t think any premade benchmark will be able to cover entire workflows, even specific to one application.”

The various benchmark programs from SPEC are used by all major workstation vendors and thousands of customers. Image courtesy of SPEC.

There are other issues to consider beyond how a computer runs under any particular benchmark. “There are two main concerns for most customers,” says Scott Hamilton, a senior consultant for Precision workstations at Dell. “First, is the workstation configuration tested and certified? Second, what is the best configuration for the application and workflow used?”

Hamilton says Dell maintains a website to look up certification for the most common applications ( Software vendors provide similar offerings.

There are additional ways to test how a particular workstation will run using existing software. “If the user is already using the application, they can leverage their own testing by using Windows Performance Monitor to see what is happening while using different workflows within the app itself,” says Bill Martin-Otto, senior workstation performance manager at Lenovo. “This will allow them to determine if they are running out of memory or maxing out the CPU or [graphics processing unit (GPU)]. It can also tell them how much bandwidth they are using on the storage device.”

Fit for Multipurpose

Most workstations often serve more than one purpose, yet software engineers have differing requirements.

“Experienced users doing CAD modeling with complicated models understand the application processes are mostly single threaded, and if it is rendering, then it is a multi-threaded CPU process,” says Pradeep Ramineni, senior performance engineer at Lenovo.

CAE tools for simulation and engineering present a different set of challenges.

“It really depends on how complicated the user models are and what solvers are used,” Ramineni says. “Typically, the CAE application licenses are more expensive than the workstation hardware used to run these applications. Though CAE applications are heavily multithreaded processes and memory bandwidth/capacity dependent, a system configured for CAD would be good enough for simple models. However, users need to better understand their application licensing and models’ complexity and use a combination of [independent software vendor (ISV)] support, benchmark data and white papers in the public domain to configure a workstation that fits their workflow requirements.”

Most hardware vendors are members of the Standards Performance Evaluation Corporation (SPEC). It is a nonprofit that publishes and promotes the use of standardized benchmark testing to evaluate compute performance and energy efficiency. Membership provides funding for SPEC, but there are other reasons vendors participate.

“We gain a deeper understanding of workstation class workloads, including financial services, CPU and GPU rendering, compute, interactive graphics, and others,” Lenovo’s Martin-Otto says. “Our participation in SPEC also keeps us more up to date on changes with various applications and workloads that are used in the SPEC/Graphics and Workstation Performance Group benchmarks and allows us to provide better guidance for our customers. Since the benchmarks are developed with our competitors it provides results, which are vendor agnostic.”

SolidWorks 2020 is one of several SPEC benchmark routines that tests for one specific product. Images courtesy of SPEC.


Testing the Broader Workflow

Existing benchmark tools test specific performance issues inside the workstation. But no one works in isolation these days. Workstations are on local networks, and often work with remote cloud services. Now, there is no equivalent to a downloadable benchmark to test for such system-wide performance.

“Most benchmarks test one subsection of an entire workflow,” says Pete Seidel, senior performance engineer at Lenovo. “The best way to evaluate an entire workflow is to run it on a few different hardware configurations to determine which one performs the best for a given price range. This method would be difficult and expensive for a customer to do on their own. Lenovo offers customers the ability to try their specific workflows in our performance lab on numerous hardware configurations.” Other large vendors have similar performance labs.

Dell’s Hamilton offers an idea on how to use benchmark software to examine broader performance issues.

“Users can find a benchmark that will measure the specific application workload like those provided by SPEC, then test all the applications to determine maximum requirements and identify bottlenecks so they can specify a system that meets all of their needs,” Hamilton says. “Also, if they are running some applications simultaneously then they should run simultaneous benchmarks as well to represent their true working conditions most accurately.”

Special Consideration for Mobile Workstations

Mobile workstations have a unique set of performance settings and considerations due to power and thermal design limitations.

“Buyers need to understand more about how their mobile workstation is designed than they would when running a desktop,” Lenovo’s Martin-Otto says.

“When running benchmarks on mobile workstations, you need to make sure you are running it in the highest performance mode possible—which is also the loudest acoustics,” Martin-Otto says. Just make sure the workstation is plugged in, as operating on battery power will result in much lower performance.

Graphics is another area where mobile workstations pose issues. For example, some mobile workstations use hybrid graphics, mixing integrated and discrete graphics processors for power efficiency. In some situations, using an external monitor instead of the built-in display may force the dedicated GPU to render everything on the screen.

More Dell Coverage

Artificial Intelligence for Design and Engineering Workflows
In this white paper, learn how artificial intelligence and machine learning can improve design and simulation.
The AI Workstation
At Dell Technologies World, Dell and NVIDIA outline the future of PCs as more companies adopt on-prem AI.
New NVIDIA RTX GPUs Built for AI-Powered Design and Visualization
Based on the NVIDIA Ampere architecture, the new RTX A400 and RTX A1000 GPUs deliver real-time ray tracing and support for generative AI tools to every workstation.
Dell Technologies World: AI in Focus
Dell, NVIDIA showcase how their hardware supports innovative AI use cases.
Dell Boosts Enterprise AI Capabilities
New Dell Technologies workstations powered by NVIDIA AI solutions unlock more engineering potential.
Dell Company Profile

More Lenovo Coverage

Making the Case for Engineering Workstation Upgrades
In this Making the Case whitepaper, Lenovo outlines how upgrading to the latest generation of professional workstations can provide a return on investment through increased engineering efficiency and greater flexibility.
Lenovo Supercharges Copilot+ PCs
With this latest debut, Lenovo says it intends to offer new levels of personalization in personal computing across its PC portfolio.
View a Live XR Broadcast in Apple Vision Pro?
XR, VR apps emerge to augment live events.
A New Price/Performance Leader: Lenovo ThinkStation P5
A bold new design and great performance make this new Lenovo workstation a winner.
Lenovo Debuts Desktops Powered by AMD Processors
Next-generation business workstations feature AI capabilities.
Big, Powerful and Pricey: Lenovo ThinkStation PX
Lenovo delivers a new flagship workstation marked by new design and myriad options.
Lenovo Company Profile

Share This Article

Subscribe to our FREE magazine, FREE email newsletters or both!

Join over 90,000 engineering professionals who get fresh engineering news as soon as it is published.

About the Author

Randall  Newton's avatar
Randall Newton

Randall S. Newton is principal analyst at Consilia Vektor, covering engineering technology. He has been part of the computer graphics industry in a variety of roles since 1985.

  Follow DE