Mixed Experiences Ring in New Era of Design Collaboration

Augmented reality (AR) catapulted into mainstream consciousness a few years ago with the release of Pokémon Go, a widely hyped consumer game. The technology is carving out a role transforming plant floor operations and in-field product maintenance and support by creating a platform to deliver more intuitive and informative forms of work instructions and training.

More recently, AR, and its close counterpart virtual reality (VR), are taking root in design workflows. The dynamic duo are ushering in mixed-reality experiences that are helping far-flung engineering teams to collaborate more effectively on early concept ideation and during extensive design reviews. 

The technologies are also providing a more efficient, effective way for engineers to share ideas and orchestrate design changes with customers and non-technical stakeholders. The models deliver a level of visualization and context not possible with traditional 3D modeling tools like CAD and simulation.

“The best use of AR in the design process is for collaboration,” contends Brian Thompson, senior vice president, Creo product management, at PTC. “What AR helps customers do is change their point of view of what the product is and how the product presents itself by reducing the cognitive load that you have to endure when you see a design sitting on a flat screen in 3D. You get a much better feel for the reality of a product when you see it at scale in front of you before you see on a table top or on the factory floor.”

Bringing Realism to Design

PTC, a leading provider of CAD and PLM that has made major investments in AR, is seeing a significant uptick in interest among its base of Creo CAD and Windchill PLM users wanting to leverage the technology as part of their design workflows, Thompson says. Creo and Windchill have long had robust collaboration, digital mockup and visualization capabilities via Creo View, software that lets design teams easily access a range of engineering data, including 3D CAD models, 2D drawings and electrical schematics.

AR Design Share, now integrated into Creo View, takes design collaboration and visualization to the next level, Thompson says. Designers can explore work-in-progress concepts by bringing them to life in full-scale and in the context of real-world environments. 

For example, consider customers that are already using the Creo Manikin extension to explore the ergonomics of plane cockpits or vehicle consoles using the software’s range of anthropometrically-correct manikins. By putting on a Microsoft HoloLens or another AR headset, teams can go beyond Creo Manikin’s 3D-based human-centric simulation and fully explore a design at scale, which provides far more realism and context for making design choices. 

“The ability to walk into a cockpit or around a piece of equipment is critical,” Thompson says. “[With AR], you can see how big an opening is for a worker’s hands or what it’s like for someone to walk through an opening [in a cockpit or building].”

AR’s potential to establish context and real-world scale when evaluating designs is important not just for engineering collaborators, but also for customers who must evaluate work in progress, especially when components or surfaces are customized to their specific needs. In the typical workflow, the engineering team sends around a 2D PDF drawing and spends time walking the customer through the changes or any other areas of the work product they want to highlight, says Thompson. 

The visual rendering of the product is often shared in a 2D PDF file instead of a fully featured 3D CAD file because most non-technical stakeholders don’t know or have access to CAD, in addition to concerns about intellectual property. 

Sharing design intent in this manner opens the door to miscommunication. “Even if you’ve engineered what the customer needs or wanted, you run the risk of the customer not being able to interpret that properly,” Thompson says. 

An immersive AR experience minimizes the need to interpret 2D drawings and circle or mark areas you want to change, Thompson adds. AR Design Share, which can attach an AR experience to a specific modification or component as opposed to the whole product design, serves as a more natural and streamlined approach to managing design changes with customers, he says.

AR/VR and mixed reality experiences can also be instrumental for letting the designer view the product through the customer’s perspective, according to Prashanth Mysore, director for global strategy business development at Dassault Systèmes. 

Although traditional CAD and PLM allow for a digital design mockup, including form and function, a mixed reality experience allows the designer to literally get into the driver’s seat of a car and explore the accessibility of the dashboard controls, for example, or how the clutch feels when shifting. 

“The innovation process is increasingly about adopting the customer experience as part of the design,” Mysore explains. “AR for designers really helps the way they innovate in terms of having the end customer in mind at the early design stage.”

AR can also help engineers communicate complex systems such as buildings or aircraft. One of the hurdles to systems-level design is visualization, given that complex systems have layers of engineering data that is spread across multiple systems and global design centers, Mysore says. 

“When you have so many layers of engineering data, you require a good amount of visualization for design reviews,” Mysore says, explaining that extended reality experiences are increasingly able to fill that role and help communicate design intent for cars, aircraft, even industrial plant floor equipment and operations.

To make mixed reality experiences more accessible to its customers, Dassault Systèmes has included extended-reality capabilities in SolidWorks 2019. With the new capability, which publishes CAD scene data into AR/VR tools or web browsers, product development teams can capture more accurate digital representations of a design and partake in interactive experiences to improve design reviews and sales engagement, as well as a vehicle for training product users.

AR/VR’s ability to boost visual collaboration in a design context also has an impact when design responsibilities are spread across a broad chain of partners. As engineers go through the iterative process of optimizing design goals (specifications) and constraints (cost or manufacturability), they typically move through the process sequentially, which makes it harder to reconcile and optimize conflicting goals and constraints across the different disciplines of engineering, manufacturing and supply chain, notes Joe Barkai, an independent consultant, speaker and author specializing in the engineering and manufacturing space. 

That’s where tools like AR/VR and mixed reality experiences come in. “Complex design decisions can be biased whether intended or not,” he explains. “Visualization and simulation level the playing field and allow each to see it from their ‘discipline’s’ point of view and show the outcome of their decisions using terms of the other participants. Visual information provides more of a common language.”

Virtual Reality Gets Real

Although AR is beginning to make waves, VR, historically limited by cost and complexity, is also starting to surface in engineering and design workflows. ANSYS, for example, offers VRX OPTIS, an immersive simulation experience that helps engineers explore the driving environment and lighting systems exactly how a driver would experience the car.

Startup flyingshapes recently announced its new application, which introduces a VR CAD workflow for 3D spatial modeling in a fully immersive virtual environment complete with haptic controls and interactive tools for sketching, surfacing and collaborating with teams on the cloud.

Aimed at the early concept stage of the design workflow, the tool’s VR capabilities help design professionals by improving communications, reducing discrepancies between the prototype and final model, and shortening iteration cycles, according to Dr. Johannes Mattmann, co-founder.

Using a VR headset, engineers sketch and shape CAD models and can use hand gestures to position the objects with exact lines, curves and surfaces in a 360° workspace. The software, powered by Unity real-time rendering capabilities and leveraging high-end NVIDIA or AMD GPUs, functions as the digital equivalent of a clay model with the added benefit of easy access to the full stack of CAD data. 

In a typical workflow, engineers start with sketches, go to CAD and then build clay models, but they have to go back and forth to CAD data to get measurements to be precise about changes, says Jonas Kunze, co-founder, flyingshapes. 

“At some point, you need the data, and with flyingshapes, you never lose the data,” he says.

The Road Ahead

Although AR and VR are certainly making inroads into design processes, the technology is still in the early stages, and a variety of challenges remain. For one, much of the hardware is either still very expensive and out of reach for mainstream customers or not nearly precise enough for professional design and development use. Field of vision is one of the key hurdles, especially for glasses-based systems that are essential for enabling hands-free design work, notes David Weinstein, director of professional virtual reality for NVIDIA. 

To address some of the issues, NVIDIA has a pair of projects underway: one is prescription AR glasses, which will have a user’s prescription built in and make for a more comfortable viewing experience. There are also plans for a “foveated AR” headset, which combines gaze tracking and deep learning technologies to adjust image resolution to match where a user is looking. “You have to be extremely precise with AR, which presents opportunity and problems,” Weinstein says.

Beyond the technology challenges, there’s also the cultural hurdle of getting people to work differently and for product engineers to embrace different design workflows. 

Emphasizing the value that new kinds of visual collaboration can bring to the design process is one way to get engineers on board. “Move the conversation away from AR or VR and focus on visual collaboration,” Barkai says. “Identify where poor collaboration has led to mistakes, delays or rework. That’s where AR and VR can be really helpful.”

More Ansys Coverage

More Dassault Systemes Coverage

Unified Modeling And Simulation (MODSIM) For Sustainable Product Development

Companies in all sectors must address the need for greater sustainability to meet customer demands. The development of new products necessitates rigorous testing and evaluation, while efforts must be made to decrease emissions and operate in a more economical fashion.

More NVIDIA Coverage

More PTC Coverage