The Rise of Data Science Workstations

Earlier this year, NVIDIA announced a reference architecture for a new class of professional workstation, the data science workstation. Almost immediately, leading workstation original equipment manufacturers (OEMs) announced workstations that conform to NVIDIA’s Data Science specification.

“Data science is one of the fastest growing fields of computer science and impacts every industry,” said NVIDIA founder and CEO Jensen Huang at the announcement. “Enterprises are eager to unlock the value of their business data using machine learning and are hiring—at an unprecedented rate—data scientists who require powerful workstations architected specifically for their needs.”

An obvious question comes to mind: When are specialized workstations needed? Why not just go with the utility of a typical professional workstation? Digital Engineering asked NVIDIA and several workstation vendors.

Their answers repeated a general theme, one we have been hearing from several sources, not just workstation vendors: There is strong and increasing demand for data science in every industry that now uses professional workstations. There are not enough data science specialists to meet employer demand, so engineers and programmers from other disciplines are being drafted as data scientists. This lack of expertise extends to the specifics of what software to run and what computer hardware is best suited for the task.

Specification Defined

A new workstation meeting NVIDIA’s standard will have several specific features not common to most existing workstations. The first is dual NVIDIA Quadro RTX graphics processing units (GPUs), based on the Turing GPU architecture. Each RTX 8000 has 48GB of GPU memory, required for large data sets typical of artificial intelligence (AI) training or deep learning and machine learning analysis. The new NVIDIA GV100, a Volta class GPU, also may be used in a data science workstation.

Both the RTX line and the GV100 use two new types of compute cores, RT cores and Tensor cores. RT is short for ray tracing but could also refer to real time; these cores are specialized for high-performance, local visualization.

Tensor Cores specialize in matrix math, common to deep learning and some applications in other fields that now run only on high-performance computing (HPC) clusters or cloud computing platforms. “[Tensor Cores] do the basics for workhorse calculating in deep learning,” says Michael Houston, a senior distinguished engineer at NVIDIA.

Tensor cores perform a fused multiply add, where two 4x4 FP16 matrices are multiplied, and the result added to a 4x4 FP16 or FP32 matrix. It sounds like high school math, but tensor cores do millions of these calculations every second, much faster than commodity CPU or GPU compute circuitry. There is also an advantage in the tensor core’s ability to accumulate everything in FP32. “Thirty-two bit accumulation tends to really matter for convergence of networks,” says Houston, “to make mixed precision really work.” Houston says the theoretical performance boost of using tensor cores is 8x. “On a lot of neural nets, NVIDIA sees a 4x speed increase end to end.” Data science models often take several days to run; a 4x speed increase would complete a four-day job in one day.

The NVIDIA Data Science Workstation specification calls for Ubuntu Linux 18.04, nicknamed Bionic Beaver, as the operating system. Along with Ubuntu comes a set of software libraries based on the NVIDIA CUDA-X AI protocol for AI research. The collection includes RAPIDS, TensorFlow, PyTorch and Caffe open source libraries and several NVIDIA-written acceleration libraries for machine learning, artificial intelligence and deep learning.

The department of aeronautics and astronautics at MIT is a pre-release user of the NVIDIA Data Science specification. “The NVIDIA-powered data science workstation provides significant capabilities for training deep neural networks for robot perception. With it, the MIT FAST Labs’ ability to train drones to see depth and avoid collisions from a single camera was significantly accelerated because we could process larger batch sizes,” says Sertac Karaman, an associate professor in the department.

Workstations meeting the NVIDIA Data Science specification go through testing and optimization tailored to the needs of data science users. The result is a local, single-user computer that NVIDIA says replaces the need for time on more expensive HPC or cloud computing platforms.

“The NVIDIA-powered data science workstation enables our data scientists to run end-to-end data processing pipelines on large datasets faster than ever,” said Mike Koelemay, chief data scientist at Lockheed Martin Rotary & Mission Systems. “Leveraging RAPIDS [software libraries] to push more of the data processing pipeline to the GPU reduces model development time, which leads to faster deployment and business insights.”

When Money is no Object

“AI is a big market and huge talking point, and it starts on a workstation,” says Mike Leach, the workstation portfolio manager for AI, AR and VR at Lenovo. By following the NVIDIA specification, Leach says Lenovo can give users “a certified solution and the right software tools out of the box.”

“We see a big shift in data scientists,” adds Leach. They start with “gigabytes of data, a data lake of images [or] financial data on the left. They want to move to a fully predictive AI on the right. The journey in-between is seeing the data, iterating on predictions and accuracy.”

While attending an AI conference recently, Leach observed “sometimes money is no object.” Data scientists are expensive employees to bring aboard, but companies “consider them must-have, and they will provide the right hardware.” As a result, Leach says, these scientists “can deliver massive cost savings based on the products they create.”

Recognizing the Software Stack

Dell, HP and Microway are also releasing workstations for data scientists that incorporate NVIDIA GPUs. Boutique workstation vendor Velocity Micro is another vendor building workstations to the NVIDIA Data Science specification and is awaiting formal NVIDIA certification. “We’ve been doing scientific computers for 20 years, just called different things,” notes CEO and founder Randall Copeland.

The biggest change with the NVIDIA Data Science specification is not as much about the GPU as the software stack, Copeland says. “A computer designed to run better for Revit or 3ds Max is not the same as the computer that runs CUDA best.”

Copeland says NVIDIA has done a “great job developing a market for artificial intelligence and deep learning.” They recognized their CUDA architecture was well-suited for massive data sets, and “they help people who are good in something else who have to become AI experts.”

A Personal Data Sandbox

The phrase “data scientist” was first coined by a Google executive in 2010, says NVIDIA’s Geoffrey Levene, director of global business development for data science workstations. “Now [the phrase] has caught up to us; data doubles every 18 months in every vertical.” From one industry to the next the process is similar, Levene says. Data must be “wrangled,” formally known as extract transform load (ETL). “Then you write the code to see what the data can do,” he explains.

From the exploration, the data scientist builds a model of the data use case. “This is the training part, used for inference and prediction,” Levene says. “Training is time-consuming; inference is fast. ETL is a lengthy process.” The workflow usually involved tabular data and “GPUs can accelerate tabular data,” he says.

Having a “personal sandbox” for data work is a boon for data scientists, Levene says. “Some are finding they do a week’s work in one day with GPU-accelerated workflows.” Levene also observes that “98% of AI for product development is machine learning.”

Artificial intelligence has been around for a generation, Levene notes, but there wasn’t enough data in many industries. Now data is abundant, thanks to both the internet and mobile devices. “Go back a year ago—you either bought time on a cloud GPU or spent up to $500,000 on a system” that took weeks for IT to install, says Levene. “Now you can order a couple of workstations and go to work.”

More Dell Coverage

Artificial Intelligence for Design and Engineering Workflows

In this white paper, learn how artificial intelligence and machine learning can improve design and simulation.

More Lenovo Coverage

Making the Case for Engineering Workstation Upgrades

In this Making the Case whitepaper, Lenovo outlines how upgrading to the latest generation of professional workstations can provide a return on investment through increased engineering efficiency and greater flexibility.

More NVIDIA Coverage

More Velocity Micro Coverage