Holomark 2 is created by Rhino 3D users. The Cooper Mini model in the benchmark has been modified in hundreds of ways by Rhino users. Image courtesy of Robert McNeel and Associates.
March 8, 2023
Professional engineering workstations provide a high level of power and performance for engineering, simulation and design software. The trick is to make sure any workstation you consider for purchase is a good fit for your specific requirements.
A variety of benchmarking software tools are available to evaluate workstations. They test specific performance issues, but they don’t tell the whole story of how any model will run in all use cases. It is generally a good idea to use a combination of different benchmarking tools to get a comprehensive understanding of a workstation’s performance. There is a list of the most popular and valuable workstation benchmarks at the end of this article.
The workstation vendors we spoke with recommend these standard benchmarks but are quick to point out they can’t tell the whole story.
“Even the SPECapc benchmarks only test a subset of the available features in each application,” says Brian Bothwell, advisory performance engineer at Lenovo. “While a good approximation of performance can be made, I don’t think any premade benchmark will be able to cover entire workflows, even specific to one application.”
There are other issues to consider beyond how a computer runs under any particular benchmark. “There are two main concerns for most customers,” says Scott Hamilton, a senior consultant for Precision workstations at Dell. “First, is the workstation configuration tested and certified? Second, what is the best configuration for the application and workflow used?”
Hamilton says Dell maintains a website to look up certification for the most common applications (PrecisionWorkstations.com). Software vendors provide similar offerings.
There are additional ways to test how a particular workstation will run using existing software. “If the user is already using the application, they can leverage their own testing by using Windows Performance Monitor to see what is happening while using different workflows within the app itself,” says Bill Martin-Otto, senior workstation performance manager at Lenovo. “This will allow them to determine if they are running out of memory or maxing out the CPU or [graphics processing unit (GPU)]. It can also tell them how much bandwidth they are using on the storage device.”
Fit for Multipurpose
Most workstations often serve more than one purpose, yet software engineers have differing requirements.
“Experienced users doing CAD modeling with complicated models understand the application processes are mostly single threaded, and if it is rendering, then it is a multi-threaded CPU process,” says Pradeep Ramineni, senior performance engineer at Lenovo.
CAE tools for simulation and engineering present a different set of challenges.
“It really depends on how complicated the user models are and what solvers are used,” Ramineni says. “Typically, the CAE application licenses are more expensive than the workstation hardware used to run these applications. Though CAE applications are heavily multithreaded processes and memory bandwidth/capacity dependent, a system configured for CAD would be good enough for simple models. However, users need to better understand their application licensing and models’ complexity and use a combination of [independent software vendor (ISV)] support, benchmark data and white papers in the public domain to configure a workstation that fits their workflow requirements.”
Most hardware vendors are members of the Standards Performance Evaluation Corporation (SPEC). It is a nonprofit that publishes and promotes the use of standardized benchmark testing to evaluate compute performance and energy efficiency. Membership provides funding for SPEC, but there are other reasons vendors participate.
“We gain a deeper understanding of workstation class workloads, including financial services, CPU and GPU rendering, compute, interactive graphics, and others,” Lenovo’s Martin-Otto says. “Our participation in SPEC also keeps us more up to date on changes with various applications and workloads that are used in the SPEC/Graphics and Workstation Performance Group benchmarks and allows us to provide better guidance for our customers. Since the benchmarks are developed with our competitors it provides results, which are vendor agnostic.”
Testing the Broader Workflow
Existing benchmark tools test specific performance issues inside the workstation. But no one works in isolation these days. Workstations are on local networks, and often work with remote cloud services. Now, there is no equivalent to a downloadable benchmark to test for such system-wide performance.
“Most benchmarks test one subsection of an entire workflow,” says Pete Seidel, senior performance engineer at Lenovo. “The best way to evaluate an entire workflow is to run it on a few different hardware configurations to determine which one performs the best for a given price range. This method would be difficult and expensive for a customer to do on their own. Lenovo offers customers the ability to try their specific workflows in our performance lab on numerous hardware configurations.” Other large vendors have similar performance labs.
Dell’s Hamilton offers an idea on how to use benchmark software to examine broader performance issues.
“Users can find a benchmark that will measure the specific application workload like those provided by SPEC, then test all the applications to determine maximum requirements and identify bottlenecks so they can specify a system that meets all of their needs,” Hamilton says. “Also, if they are running some applications simultaneously then they should run simultaneous benchmarks as well to represent their true working conditions most accurately.”
Special Consideration for Mobile Workstations
Mobile workstations have a unique set of performance settings and considerations due to power and thermal design limitations.
“Buyers need to understand more about how their mobile workstation is designed than they would when running a desktop,” Lenovo’s Martin-Otto says.
“When running benchmarks on mobile workstations, you need to make sure you are running it in the highest performance mode possible—which is also the loudest acoustics,” Martin-Otto says. Just make sure the workstation is plugged in, as operating on battery power will result in much lower performance.
Graphics is another area where mobile workstations pose issues. For example, some mobile workstations use hybrid graphics, mixing integrated and discrete graphics processors for power efficiency. In some situations, using an external monitor instead of the built-in display may force the dedicated GPU to render everything on the screen.
More Dell Coverage
More Lenovo Coverage
Subscribe to our FREE magazine, FREE email newsletters or both!
About the Author
Randall S. Newton is principal analyst at Consilia Vektor, covering engineering technology. He has been part of the computer graphics industry in a variety of roles since 1985.
Follow DE
