NVIDIA Advances Humanoid Robotics Development

NVIDIA announced at SIGGRAPH that it is providing robot manufacturers, AI model developers and software makers a suite of services, models and computing platforms to develop, train and build the next generation of humanoid robotics.

Part of the suite are new NVIDIA NIM microservices and frameworks for robot simulation and learning, the NVIDIA OSMO orchestration service for running multi-stage robotics workloads, and an AI- and simulation-enabled teleoperation workflow that allows developers to train robots using small amounts of human demonstration data.

“The next wave of AI is robotics and one of the most exciting developments is humanoid robots,” says Jensen Huang, founder and CEO of NVIDIA. “We’re advancing the entire NVIDIA robotics stack, opening access for worldwide humanoid developers and companies to use the platforms, acceleration libraries and AI models best suited for their needs.”

NVIDIA NIM and OSMO 

NIM microservices provide pre-built containers, powered by NVIDIA inference software. Two new AI microservices will allow enhancement of simulation workflows for generative physical artificial intelligence in NVIDIA Isaac Sim, a reference application for robotics simulation built on the NVIDIA Omniverse platform.

The MimicGen NIM microservice generates synthetic motion data based on recorded teleoperated data from spatial computing devices. The Robocasa NIM microservice generates robot tasks and simulation-ready environments in OpenUSD, a universal framework for developing and collaborating within 3D worlds.

NVIDIA OSMO, available now, is a cloud-native managed service that allows users to orchestrate and scale complex ‌robotics development workflows across distributed computing resources.

OSMO is designed to simplify robot training and simulation workflows. Users can visualize and manage tasks such as generating synthetic data, training models, conducting reinforcement learning and implementing software-in-the-loop testing at scale for humanoids, autonomous mobile robots and industrial manipulators.

Data Capture Workflows  

An NVIDIA AI- and Omniverse-enabled teleoperation reference workflow, demonstrated at the SIGGRAPH computer graphics conference, allows ability to generate synthetic motion and perception data from minimal remotely captured human demonstrations.

First, developers use Apple Vision Pro to capture a small number of teleoperated demonstrations. Then, they simulate the recordings in NVIDIA Isaac Sim and use the MimicGen NIM microservice to generate synthetic datasets from the recordings.

The developers train the Project GR00T humanoid foundation model with real and synthetic data. They then use the Robocasa NIM microservice in Isaac Lab, a framework for robot learning, to generate experiences to retrain the robot model. Throughout the workflow, NVIDIA OSMO assigns computing jobs to different resources.

Expanding Access to Technologies 

NVIDIA provides three computing platforms: NVIDIA AI supercomputers to train the models; NVIDIA Isaac Sim built on Omniverse, where robots can learn and refine their skills in simulated worlds; and NVIDIA Jetson Thor humanoid robot computers to run the models. Developers can access and use any part of the platforms.

Through a new NVIDIA Humanoid Robot Developer Program, developers can gain early access to the new offerings as well as the latest releases of NVIDIA Isaac Sim, NVIDIA Isaac Lab, Jetson Thor and Project GR00T general-purpose humanoid foundation models.

1x, Boston Dynamics, ByteDance Research, Field AI, Figure, Fourier, Galbot, LimX Dynamics, Mentee, Neura Robotics, RobotEra and Skild AI are among the first to join the early-access program.

Availability 

Developers can join the NVIDIA Humanoid Robot Developer Program to access NVIDIA OSMO and Isaac Lab, and will soon gain access to NVIDIA NIM microservices.

Sources: Press materials received from the company and additional information gleaned from the company’s website.

More NVIDIA Coverage