Artificial Intelligence + Virtual Reality = Holodeck (Almost)

The holodeck as depicted in the TV series “Star Trek” is a hologram-powered simulation environment. Its use is primarily recreational. But the show also proposes pragmatic applications of the holodeck’s technology. “The Star Trek: Voyager” series features a holographic medical doctor, the Emergency Medical Hologram Mark I. In some episodes, the holodeck is pressed into service as a platform for battle training and forensic analysis.

The concept of reality-mimicking environments constructed with solid holograms remains theoretical. However, similar environments built with convincing pixel-objects are well within reach. From the Google Cardboard ($15) to the more advanced HTC Vive ($799) and Oculus Rift ($599), the headgear required to deploy and display VR content is becoming more affordable.

From CAD to VR

There’s no shortage of accurate, detailed 3D content—the cumulative legacy of CAD use in engineering. VR content is usually put together in game engines, such as Unity, Unreal or CryENGINE. Converting CAD data from the engineering world into game engine-compatible VR is not a straightforward task.

“CAD software makers might want to think about the hand-off process to the engine,” said Michael Kaplan, NVIDIA’s global lead for Media and Entertainment. “Or, would they prefer to develop their own CAD-to-VR conversion software? After all, they’re software companies.” For some, that might be biting off more than they could chew. Kaplan pointed out: “Most of them have never written real-time VR software.”

AI, once considered the stuff of science fiction, is picking up momentum. AI programs like autonomous navigation and image recognition in self-driving cars are expected to come from deep learning, a process that uses massively parallel computation to teach programs to see and react to their physical surroundings.

At NVIDIA’s GPU Technology Conference, NVIDIA CEO Jen-Hsun Huang unveiled the GTX-1, described as “the world’s first deep-learning supercomputer.” He also introduced the TESLA P100, “for deep learning [at] 16-bit performance,” according to Huang. The new products are augmented by the company’s VRWorks software suite for VR application developers.

An attendee gets ready to experience VR at this year’s NVIDIA GPU Technology Conference. Image courtesy of NVIDIA.

CPU maker Intel pursues the same market with its Intel Deep Learning Framework. IBM Watson, a supercomputer that defeated its human opponents in the TV gameshow “Jeopardy!” in 2011, also wants a piece of the action in deep learning. As of February, Watson has learned to detect signs of human emotion.

Benefits of Convergence

Currently, AI and VR are separate industries, each pursuing its own objectives, each wrestling with its own challenges. In my view (which, I must admit, is influenced by an overactive imagination), the convergence of the two could propel digital simulation to new heights. A VR environment running on autonomous agents and deep learning algorithms could give us unprecedented insights into products and processes.

In a holodeck-like simulation environment, a design study is more than a quality assessment. A virtual product trial could be a stirring, searing, unforgettable experience that induces laughter, tears or fear. If powered by AI and deep learning algorithms, such an environment could suggest design changes and configurations that engineers might have never conceived solely by intuition and expertise alone.

If IBM Watson’s emotional intelligence is a harbinger of things to come, future simulation programs might be able to detect how we feel about suggested design changes. If next-gen simulation environments are to look more like the holodeck and less like the standard finite element analysis software interface, then the challenge for developers is to come up with a new way of posing questions—not through dropdown menus and input fields, but through a more natural interaction. Voice command, for example, would be consistent with the holodeck.

VR-powered simulation is still too far away to worry about, you say? By my calculation, 2024 is only eight years away.