Autonomous Vehicles Take VR Test Drive

At CES 2020 in Las Vegas (one of the last big in-person tech shows before the COVID-19 shutdowns began), BMW Group tickled the automotive enthusiasts’ fancy with a demo of the BMW i Interaction EASE, a concept car for the future. Part of the “wow” factor was its use of the panoramic windshield as an augmented reality (AR) display. 

“It can show additional information on the windscreen that is tailored to the situation at hand and the vehicle’s surroundings. Thanks to 5G connectivity the vehicle knows exactly where it is and can offer the user information on the surrounding buildings, businesses and other objects as and when required,” according to BMW’s CES announcement. 

Public records of U.S. patent application 20180089899 from Apple, published in March 2018, describes “an AR system that leverages a pre-generated 3D model of the world to improve rendering of 3D graphics content for AR views of a scene, for example an AR view of the world in front of a moving vehicle.” It further explains the system may “augment local data (e.g., a point cloud of data collected by vehicle sensors) to determine much more information about a scene, including information about occluded or distant regions of the scene, than is available from the local data.”

“AR is not only useful for developing autonomous cars; it will likely be part of the vehicle itself,” notes Sandeep Sovani, director of Global Automotive Industry, Ansys.

For autonomous car developers, the current shutdown is a quandary. With fewer cars on the road, it seems to offer an ideal situation to test their car’s self-navigation system. But the current low traffic flow doesn’t reflect how the car is expected to perform, making the test results less reliable. In this article, we explore how some developers are continuing to make progress, aided by AR/VR technologies.

Simulation is Unavoidable

Conducting road testing to satisfy the safety requirements of autonomous vehicles (AV) is a daunting task, in terms of the sheer number of hours and miles involved. Research and analysis firm Rand Corporation decided to find out how many miles of driving it would take to demonstrate autonomous vehicle reliability. 

The conclusions vary depending on the desired degree of reliability, but the most cited statistics from the Rand report reveals that “to demonstrate with 95% confidence, [it would take] 8.8 billion miles (400 years).” 

For this reason, Ansys and other leading simulation software makers are betting that AV developers will inevitably be relying on simulation for the lion’s share of the testing.

“In the development process, engineers might use Ansys SPEOS,” says Sovani. Ansys SPEOS “predicts the illumination and optical performance of systems,” according to the product homepage.

“You need to project bright light onto the head-up display through various optics. The image shouldn’t be distorted, and perfectly tuned for the cabin’s ambient lighting,” says Sovani. “Ansys SPEOS can do the optical calculation required.”

For designing the display itself, Sovani suggests the use of Ansys SCADE Display, part of the Ansys SCADE collection. The software lets you prototype head-up displays commonly used in the aerospace, rail transportation and automotive industries, among others.

The entire cabin environment may be prototyped and simulated in Ansys VRXPERIENCE, Sovani suggests. “You could use VRXPERIENCE to simulate thermal camera images or radar feed, for example,” says Sovani. (For more on VRXPERIENCE’s role in acoustic simulation, read the article on page 26.)

Both Ansys SPEOS and VRXPERIENCE are the upshots of Ansys’s 2018 acquisition of OPTIS, an optical simulation software firm. Ansys SCADE was the outcome of the company’s 2012 acquisition of Esterel. 

Sovani also envisions engineers using immersive VR caves to simulate the vehicle’s autonomous reaction to different weather and traffic conditions (nighttime, rainy weather, encounter with an animal, heavy traffic, for examples). “You can run the software on your desktop, but the more effective use is to put on VR googles and view the results,” says Sovani. 

“AR/VR technologies may also be helpful to give reluctant consumers a way to experience autonomy, to see what it’s like to ride in an autonomous car in virtual reality,” notes Sovani.

Meeting in Pixels During the Shutdown

To test-drive cars not yet physically built or mocked up, Volvo has been using Varjo’s XR-1 hardware. The driver drives a real car wearing the XR-1 headset to experience the inside of a car that only exists in pixels. XR, which stands for extended reality, is AR hardware that can combine digital and physical realities. (For more, read “Are you ready for extended reality?” August 2019). 

“The driver is really relying on the real-time video signals from the headset to drive the car,” notes Urho Konttori, chief product officer and founder of Varjo. “The high-quality graphics and 5G connection in Varjo headsets make this feasible.”

Some of Varjo’s automotive customers are now using VRED from Autodesk to digitally prototype, review and validate their work. The software offers several application programming interfaces to connect to VR hardware and software. Multiuser collaboration support was added in VRED 2019.2 release. This allows, for example, the chief designer to review and improve on weekly design iterations during the shutdown via VR. 

“The XR (AR and VR) workflows in products like Alias and VRED help our customers to better create forms and shapes, understand proportions in the context of their design ideas, and empower collaborations and digital decision-making,” says Thomas Heermann, associate VP of Design and Creation Products, Autodesk. 

The use of AR/VR in automotive is about more than visualization. It’s also seeping into design creation and editing tasks. “Autodesk Alias Create VR allows conceptual designers to use VR to create mobility products in 3D in VR, while VRED empowers designers, engineers and decision makers to visualize design ideas, and put them in a real-world context,” adds Heermann.

The ease of use with VRED, as Konttori sees it, comes from its handling of imported STEP files and material assignment. “You just need to assign the materials once,” he says. “After that, whenever you open the model again, the same material properties show up.”

The collaborative real-time use of VRED requires powerful workstations as the rendering takes place on local machines, not in the VR headset. In such usages, the GPU plays a critical role to constantly pump out and refresh the graphics as users move around and their perspectives shift. 

“But this is not new. The software is basically sending the participants’ XYZ positions and head orientations in the virtual world, just like you do in multiplayer games in the virtual world,” points out Konttori. “With NVIDIA RTX GPUs, the application really benefits a lot from the real-time raytracing possible.”

Collaborative use of VR is not confined to Varjo’s customers themselves. “We keep a minimum number of people in the office. These are people who must put together new PCBs, for example. Out of the 130 R&D engineers we have, about 100 are working from home now,” says Konttori. 

Part of Konttori’s weekly routine is to review new designs of Varjo headsets. He does so by viewing the design in his VR-1 headset from home, using the Keyshot rendering program’s KeyVR function for visualization. “COVID has made remote work a common practice,” notes Konttori. 

Varjo currently offers XR-1 Developer Edition (mixed reality or augmented reality), along with its VR-2 and VR-2 Pro headsets. The hardware offerings are augmented with its software with support for Unity and Unreal game engine data. 

During the shutdown, VR products seem to be in greater demand than AR, Konttori notices. To use mixed reality properly, you would need to overlay a variety of digital data on top of a life-size product—a car, in the case of automotive design. “Let’s face it. People’s living rooms are not meant to house a car,” Konttori quips. 

But Konttori says he had seen significant increase in the customers’ desire to explore VR-based collaboration during the shutdown. As a result, the company recently entered into a partnership with MeetinVR in May to offer their products as a bundle. 

“No doubt companies are sensitive to expenditure during tough times, but allowing engineers to work remotely with peak performance is also important,” observes Konttori.

More Ansys Coverage

More Autodesk Coverage