USD as the Language of 3D Virtual Worlds

Three years ago at GTC 2019, NVIDIA CEO Jensen Huang publicly launched NVIDIA Omniverse as a collaboration platform. Over time, NVIDIA’s vision for the platform expanded significantly, aiming to make it a visually rich GPU-accelerated platform with simulation features, ideal for connecting and creating 3D virtual worlds for design collaboration and large scale digital twin simulation. At GTC21 Fall (November), as he announced the commercial availability of Omniverse Enterprise, Huang said, “Omniverse is where we will design, train, and continuously monitor the factories, buildings, warehouses, and cars of the future.”

The file framework for describing these 3D virtual structures and digital twins is Universal Scene Description (USD), a powerful open sourced 3D scene description. An important technology that enables the collaborative architecture of NVIDIA Omniverse. While USD is in its early development phase - much like HTML 1.0 in the beginning of the internet - companies like NVIDIA and Pixar are actively contributing to the schemas.

Pixar’s Gift to the 3D Community

USD originated at animation studio Pixar, as an extensible, powerful framework for collaboratively sharing 3D graphics elements. . The file contains mesh, light, shader, camera angle, and data required to realistically depict 3D objects and scenes. The animation studio published USD as an open-source format in 2016, allowing the 3D modeling community to collectively enhance it. As such, it found support in 3D Studio, Autodesk 3ds Max, Autodesk Maya, and other 3D content creation programs commonly used in the media and entertainment industry. It’s also supported by the two major game engines, Unreal and Unity.

Dick Van Gelder, who previously worked as the Senior Engineering Lead at Pixar, is now Director of Software Development for NVIDIA Omniverse. In his talk at GTC21 Spring (“Introduction to USD,” April 12, 2021), he said, “USD is the core representation of the world inside Omniverse.” On the decision to select USD, Gelder said, “Pixar makes these complex films, with hundreds of artists working together in [Pixar’s headquarter in] Emeryville, California … Pixar has iterated for decades to come up with a set of tools.”

With character animators, virtual set builders, and effect artists using different modeling tools, Pixar had to develop a 3D framework that allows assets from different sources to coexist in the same scene. That was the genesis of USD.

The problem Pixar was able to solve was the same hurdle to collaborative design collaboration in other industries that leverage 3D software tools, from architecture to engineering. Therefore, when NVIDIA began developing the Omniverse platform, it was attracted to USD as the solution to overcome these issues.

USD Beyond Entertainment

In the LinkedIn blog post “Plumbing the Metaverse with USD” (March 2021), Michael Kass, Sr. Distinguished Engineer at NVIDIA, noted, “At the core of any true 3D web or the open Metaverse, we need an open, powerful, flexible and efficient way to describe the shared virtual world. This will not be an extension to HTML or a JavaScript rendering library … We believe the right description of shared virtual worlds for the open Metaverse already exists. It is Pixar’s open-source USD.”

With access to mathematically accurate 3D models of real-world objects, from standard mechanical parts and car models to body implants, CAD users are well-suited to be content providers to Omniverse applications, reasoned Brian Harrison, Director of Product Management, NVIDIA.

However, it’s not just a one-way street. “The real potential is to allow these [CAD and FEA] applications to participate, so that applications you build in Omniverse can reach out to those original [design and simulation] applications,” said Harrison.

In the associated GTC panel discussion hosted by VentureBeat tech writer Dean Takahashi, Kass further explained, “At Pixar, most of that collaboration [based on USD] was not real-time, but [at NVIDIA Omniverse] we decided to push USD to use it to represent live, dynamic virtual worlds. On top of USD, we added the ability to synchronize files across multiple users.”

Since USD is open source, CAD, PLM (Product Lifecycle Management), 3D visualization, CAE and FEA vendors may develop plug-ins and add-ons to enable import/export features in their packages. USD’s unique layering structure also opens the possibility to add other data, such as manufacturing requirements and part sourcing info, to 3D assets in Omniverse.

NVIDIA Omniverse Connectors

NVIDIA has also begun working with its software partners to develop Omniverse Connectors to facilitate bi-directional data exchange from other applications. For Enterprise customers, there are currently connectors for Unreal Engine 4, Autodesk Maya, Autodesk 3ds Max, Autodesk Revit, Trimble SketchUp, and McNeel & Associates Rhino, with more currently in beta.

Harrison said, “In the future, I see ISVs [independent software vendors] picking up on this [connector development] and accelerating it as they see a greater need. They're the best ones to write these connectors. They know their products. And they know the ins and outs, they know what kind of data that they can bring forward, what kind of data they can consume. The other avenue is to develop converters.”

But negotiating the desired level of details for each user’s need will likely be an ongoing challenge in the collaborative setup.

Assets imported from Omniverse don’t necessarily need to be confined to meshes, Harrison explained. “USD supports trim NURBS, so these assets are very accurate,” he said.

Connected Virtual Worlds

For developers working on Omniverse-centered applications, NVIDIA offers prebuilt USD content libraries, free viewers, and Python API tutorials. It’s also possible to establish live links between Omniverse and other 3D authoring tools using USD as the exchange format. In a video tutorial released in December 2020, Gelder showed how changes in Autodesk Revit (for architecture) can trigger instant updates of the same model rendered in Omniverse. A similar approach may be used by other automotive and aerospace design teams.

Bentley, an infrastructure software company, recently expanded its iTwin platform by adding Omniverse Enterprise integration. “This integration allows engineering-grade, millimeter-accurate digital content to be visualized with photorealistic lighting and environmental effects on multiple devices including web browsers, workstations, tablets, and virtual reality and augmented reality headsets from anywhere in the world,” says Bentley in its announcement. A similar approach may be adopted by other automotive, aerospace, and infrastructure design solution providers.

At CES 2022 in January, NVIDIA announced a free version of Omniverse for individual creators and artists. For engineering organizations and businesses, NVIDIA offers Omniverse Enterprise. Servers and workstations configured to support and run Omniverse Enterprise are available from Dell Technologies

More Dell Coverage

Artificial Intelligence for Design and Engineering Workflows

In this white paper, learn how artificial intelligence and machine learning can improve design and simulation.

More NVIDIA Coverage