Is AR/VR Ready to Go Beyond Visualization?

Offering new ways to facilitate immersive experiences for individuals or groups, virtual reality (VR), augmented reality (AR) and mixed reality (MR) devices have now become a dominant medium for storytelling in manufacturing. 

Volvo uses Varjo’s VR headsets to let its test drivers virtually operate new car models that are still in development. Toyota uses AR in its showrooms to let car buyers virtually see what’s under the hood. The Volkswagen Group uses a VR training app developed on Unity game engine. BAE Systems publishes digital work instructions with PTC’s software, for employees and technicians to view on Microsoft HoloLens and other compatible devices. 

These cases represent how prevalent the use of AR, VR and MR has become in storytelling—not to depict an epic struggle between good and evil—but to expose potential design flaws, verify if a new concept is worth pursuing or to explain complex procedures. 

Use of the same tools for content creation—to design engine blocks and airplane seat holders inside virtual space, for example—seems like the natural progression. We spoke to leading CAD software makers and AR/VR pioneers to understand if the technology can become a legitimate design tool, home of the next-gen CAD applications. The possibility exists, but there are more than a few technical and logical barriers to overcome.

A New Kind of Workstation? 

Will AR, VR and MR headsets replace the current desktop and mobile computing devices? Several leading workstation makers have already released their own headsets. You might view it as their preemptive strike, in recognition of the threat posed by the emerging devices. 

HP’s VR headset HP Reverb is widely available in regular and professional editions, with prices ranging from $450 used to $640 new. Lenovo recently unveiled its ThinkReality A6 HMD (head-mounted display) at Augmented World Expo in May. Lenovo hasn’t revealed its price or release date. 

These devices, some affordably priced to be an impulse buy, are bound to whet the enterprise buyers’ appetite, setting them up to explore high-ticket items like Microsoft HoloLens 2 or Varjo’s VR. 

“Buyers tend to use VR for staff training beforehand, then AR for in-field work,” observes John Ludwig, HP’s product manager of Consumer and Commercial VR. “For example, in automotive, they might use VR in the design phase, then switch to AR in the production line.” 

In the design phase, a physical copy of the new product is seldom available for one simple reason: It has not yet been built. VR becomes the perfect medium to visualize and experience the concept in digital form. For example, an engineering team may import the detailed life-size CAD model of a new vehicle into VR for closeup examination and design review. This can be followed by the use of AR in the assembly and maintenance processes. 

“You could technically open your Windows desktop in VR,” says Ludwig. In other words, wearing a Reverb headset, you could be examining a digital assembly and looking at its bill of materials in Excel at the same time. “But that’s still not a widespread practice. Creation is still happening primarily on desktop computers,” says Ludwig. 

Higher end devices such as HoloLens 2 ($3,500) includes hand tracking, allowing you to interact with digital objects with natural gestures (for example, playing a virtual piano using your real fingertips). But for CAD modeling, the precision requirement is much higher. “CAD input is still very much driven by numbers,” notes Ludwig. “Today’s touch controls and hand tracking technologies are just not precise enough to do CAD.” 

Emerging VR CAD

New software like Lightshape, a modeling application for AR headset users, points to the future of CAD, says Tim McDonough, VP and GM at Unity. “The idea is, you sculpt out the high-level organic shape in AR in Lightshape, then bring it back into CAD for proper detail works,” he notes. 

Lightshape uses Unity’s technology to ingest CAD content and display it in VR. Software programs such as Lightshape are “the solutions that work together with CAD rather than replace them,” notes McDonough. “In other words, you work in CAD, quickly jump into VR to view the design, then back to CAD for refinement.”

Carmaker Audi uses a turnkey version of Lightshape for collaborative engineering. With it, a group of people can view and evaluate design concepts together before the actual vehicle is built or mocked up. 

“With this [system], a whole team of users in one physical room can communicate in VR and examine virtual models. Furthermore, more VR headsets and Holodecks [VR system] can connect from all over the world. All users are located within the same virtual space and discuss their respective problems on the 3D model in question at hand. This way, complicated situations can be expressed and evaluated, completely independent from physical restrictions,” according to Lightshape’s online product homepage. 

In May, a startup called flyingshapes also released its first commercial product, a VR CAD program. Extending beyond viewing and measurement functions, flyingshapes includes 2D profile sketching, extrusion, surfacing and cross-sectioning tools. Although the VR-based toolset is limited compared to a typical desktop CAD program’s range of functions, the program shows VR’s potential to become a geometry modeling environment. 

“Pre-CAD geometry authoring inside AR or VR is a very nascent field,” observes McDonough. “Right now, the more prevalent workflow is to extend the CAD workflow with rapid review cycles in AR/VR.”

Game engines like Unity play a crucial role in the CAD-to-VR and CAD-to-AR pipeline, by giving users a way to transfer mathematics-driven CAD geometry into triangulated surface geometry more suitable for immersive displays. Unity and its rival Unreal offer plug-ins for this purpose. (For more, read “From Solid Geometry to Responsive AR/VR,” DE, August 2018; digitalengineering247.com/r/19062).

“There are tons of authoring applications, like CATIA [from Dassault Systèmes], Revit [from Autodesk] and SketchUp [from Trimble]. Users have created lots of 3D content in these tools. Our job is to build connectors, data pipelines from these apps into Unity,” says McDonough.

Enter Autodesk

With a collection of 2D and 3D CAD programs, Autodesk is well-positioned to propose its own solution for modeling in VR. In October, Autodesk launched Alias Create VR, a conceptual design program delivered as part of the Alias portfolio.

The toolset lets you create and edit curves, and also explore surfaces, forms and shapes while fully immersed in VR. Sketches and models can be easily exported to Alias or other CAD packages for further detailed work.

“Overall, we’re seeing the use of AR-VR to review, collaborate and tell stories,” notes Thomas Heermann, senior director of Automotive and Conceptual Design, Autodesk. “What’s great is that designers can now use AR/VR to express, capture, and communicate their ideas faster than ever before,” he says. “The increase in productivity is enabled by the sense of scale and 3D position and enabling its users to step inside the machine and interact with 3D data as if it were real.”

But for AR/VR CAD programs, the established modeling habits from the desktop regime remain a challenge. “For example, translating mouse-optimized commands directly into AR/VR can be difficult, as using a mouse is different than working with controllers. We’ve also learned that the simplicity of the AR/VR medium toolset opens new possibilities like sketching in 3D space, and intuitive, natural modeling workflows that are closer to traditional sculpting than technical modeling, and all with the ease and comfort of traditional sketching,” says Heermann.

Rethinking Geometry Modeling

Generally, AR adds a layer of digital data on top of the real-world view. By contrast, entering a VR world means losing contact with the real world. 

“It becomes clear that software is now more than just 2D chrome on a monitor. We are now designing a [virtual] place in which designers will live while they are working. This is a massive shift in how we think of traditional software and one that will transform the industry. There are certainly ergonomic challenges and limitations with hardware that prevent the traditional ways of CAD modeling and their accessibility in AR-VR,” says Heermann. 

With its acquisition of Vuforia in 2015, PTC gained a significant foothold in the AR space. PTC’s Vuforia-based AR apps are used by its clients for design review, sales and marketing, training, and maintenance. 

“AR/VR promises to retain the efficiency and low cost of CAD, while bringing us back to the natural process of working with our hands to create products,” says Luke Westbrook, CAD Product Management Specialist, PTC. 

For AR/VR modeling app developers, the key is to strike a balance between the natural modeling methods (think of a child sculpting shapes out of a block of Play-Doh) and the need for precision. 

“Most importantly, natural but sustainable input methods must be worked out […] no one is going to be able to wave their arms around for more than a few minutes to create a design. Even if they could, using our hands is an imprecise input device, and CAD models need precision,” notes Westbrook.

Theoretically, voice commands, virtual dials and pen input with pointed tips can help solve these issues, but not without penalty. “These reduce the natural feel of the process, but increase precision and longevity,” Westbrook points out.

CAD geometries are precise mathematical descriptions of shapes, thus suitable for manufacturing. By contrast, many AR/VR programs translate data into triangulated geometry; if you examine shapes too closely in AR-VR apps, it becomes apparent. Ideally, CAD users prefer to work with native CAD files when editing, to preserve the geometry’s integrity. 

“It’s possible to work with CAD files inside of AR/VR. However, it may mean that parts of the model are editable while others are not. For example, the number of triangles will be limited to ensure performance, which may mean that areas that are not essential in a scene are not loaded. These types of workflows have already been explored in the CAD industry when moving to newer platforms such as mobile and web browsers,” says Heermann.

“It’s not only possible to work with native CAD files in AR/VR; it’s possible to stream the CAD session itself into AR/VR, eliminating the need for exporting out to the desktop CAD tool,” says Westbrook. “While this is technically possible, it’s still not quite ready due to limited compute power on most AR hardware devices. However, developments in remote streaming and 5G are promising for such use cases. VR, on the other hand, is generally more readily accommodating to working with graphically intensive models since their compute power is coming from the base machine itself.”

A Peek at the Crystal Ball

Many HMDs capable of high-resolution imagery require a high-power workstation to run, because the headsets do not have adequate computing power to continuously pump out dynamic, responsive high-resolution visuals. Varjo’s VR, for example, requires the device to be attached to a workstation-class machine to sustain high-resolution images at a good frame rate. This requirement alone rules out many AR, VR and MR headsets from becoming total replacements for workstations. 

Newer HMDs like the Microsoft HoloLens 2 give you cord-free movement, because the headset has a Qualcomm Snapdragon chip for computing, with Wi-Fi connectivity. But that’s an exception to the market. 

Nreal, the firm that makes AR glasses that look and feel like ordinary sunglasses, managed to cram all the AR components into the eyebrow bar of its eyewear. It weighs a mere 88 grams (about twice as heavy as a pair of Ray-Ban sunglasses). Of all the HMDs, this is perhaps the most sensible one to wear and work for 8 hours a day, but this is an outlier. The norm is much closer to a device like the Oculus Rift, which weighs 8.2 lbs. 

People have grown accustomed to sitting and working before desktop computers all day long, but very few people will be willing to wear a 6- to 8-lb. headgear and work with it for the entire workday. Thus, the most likely scenario for the foreseeable future is for people to use the HMD for a few minutes to an hour at a time as needed (to verify aesthetics, proportions and ergonomics), then do the rest of the modeling tasks on a desktop system. 

“Overall, we do foresee CAD modeling being done in AR/VR. However, simply porting the existing CAD user interface paradigms and tools directly into AR/VR will likely not be the best approach. The ultimate challenge with bringing CAD modeling to AR/VR will be in the experience. AR/VR is an extremely powerful new platform, much different than desktop, web and mobile,” says Heermann.

More Hewlett Packard Coverage

More Lenovo Coverage

Making the Case for Engineering Workstation Upgrades

In this Making the Case whitepaper, Lenovo outlines how upgrading to the latest generation of professional workstations can provide a return on investment through increased engineering efficiency and greater flexibility.

More PTC Coverage