The Art of Balance in Mixed Reality

Remember the saying about getting something fast, good and cheap? You get two out of the three, but not all three. Want something fast and good? It won’t be cheap. Fast and cheap? Sure, but it won’t be good. Good and cheap? Well, it won’t be fast. You might have seen some variations of this warning posted in dry cleaners and diners to remind the customers to keep their expectations realistic.

The balancing act for AR-VR (augmented reality, virtual reality) devices is a similar proposition. If you want high-resolution visuals, you need a lot of processing power, which adds weight and cost to the device. If you choose to get the processing power from an external source such as a graphics processing unit (GPU) accelerated workstation, the connector cord limits the user’s movements. If you engineer a cord-free device with a lot of built-in processors, the device weighs more and generates heat, affecting the user’s comfort.

Similarly, if you want to support gesture recognition—the kind that doesn’t require installing hefty trackers inside a room—you have to embed sensors into the device, making the hardware heavier and pricier.

This article looks at some players in the AR-VR space to understand how these decisions are made and why they feel their choices are in the best interest of the targeted users, industries and markets.

Microsoft Launches HoloLens 2

This February, at the Mobile World Congress in Barcelona, Spain, Microsoft released HoloLens 2, an upgraded version of its mixed reality (MR) headset. Priced at $3,500, the device is significantly above other AR devices such as the Epson Moverio BT-300 ($699), the Toshiba dynaEdge AR glasses ($1,899) and the Google Glass Enterprise ($1,800).

One distinguishing feature in HoloLens 2 is its finger- and hand-gesture recognition. The sensors embedded in the headset create a virtual vision box or a frame that the device can “see,” so to speak. This allows you to use natural hand gestures to interact with virtual objects, for example, using your hands to play a virtual piano or push virtual buttons. For some, this sophisticated hand tracking may be well worth the extra cost of the headset.

PTC is among the early adopters, writing applications driven by its Vuforia platform to deploy on the HoloLens 2. “One of the key value drivers for AR/MR in the industrial space is improving worker efficiency with ‘hands-free’ procedural guidance and instruction,” says Jim Heppelmann, president and CEO, PTC. “Vuforia Studio’s efficient AR/MR authoring environment and HoloLens 2 usability and comfort enhancements are a winning combination for industrial manufacturers.”

The ability to interact with virtual machines—or the digital twins of real-world equipment—using natural gestures, such as turning wrenches and adjusting dials, is critical in virtual training applications. With joysticks and controllers, the trainees cannot build accurate muscle memory to replicate the actual movement required in the field.

HP Launches Reverb

In April, HP launched Reverb, a high-res Windows VR headset. The Pro edition is priced at $649. By comparison, HTC Vive currently sells for $499, and Oculus Rift for $399. Reverb is marketed as high-res because the device offers “double the resolution with 9.3 megapixels combined panel resolutions, full RGB subpixel stripe and an extended 114-degree field,” according to HP’s description online.

Part of the secret to achieving high-res is the choice to go with an LCD display instead of OLED. “With LCD, we could get a much higher resolution panel in a much smaller size. This let us reach our resolution goals, while at the same time reducing the size and weight of the headset,” explains John Ludwig, HP’s global lead for VR headsets.

The first-generation Reverb is powered by a workstation, thus attached to a cord. “There is a massive performance difference between what you can fit on someone’s head and a VR Ready Windows PC,” said Ludwig. (See “What is a VR-ready machine?” to the right.) To overcome the awkwardness of the cord, HP offers a backpack-style workstation that you can wear. Dubbed the HP Z VR Backpack G1, it’s priced beginning at $3,799.

Human-Eye Resolution VR

In February, Helsinki-based Varjo launched what it calls “the first human-eye resolution VR headset for industrial use,” priced at $5,995. “The resolution of other VR devices is a fraction of what the average human eye can see. Varjo’s Bionic Display delivers an unprecedented human-eye resolution of 60 pixels per degree. It revolutionizes professional VR by bringing every detail, texture, contour and color into 20/20 focus—even for users who wear glasses or contact lenses,” the company writes online.

“The human eye region [in Varjo’s headset VR-1] is made from a micro OLED screen, typically seen only in broadcast quality camera viewfinders of highest end SLRs,” explains Urho Konttori, founder and CEO of Varjo. “VR-1 has an optical fiber cable to cater for the massive data transfer needed to achieve human eye resolution. It’s thin and light, 10 meters long. We designed it to be as little in your way as possible, but (it) still enables you to make use of reliable connection and be able to do compute on a powerful machine or a rack-full.”

Interaction in VR-1 is enabled via SteamVR controllers. “The soon-to-be released SteamVR Knuckles controllers will have fantastic hand and finger tracking along with the discrete actions of button presses and the haptics. We feel it’s the best of both worlds,” says Konttori.

Gaming Gets Serious About VR

In March, a Sony blog post revealed it had sold more than 4.2 million PlayStation VR systems. The VR system was released in 2016 for $399. Sony says it is compatible with the yet-to-be released PlayStation 5 game console, which means gamers who want to invest in Sony’s VR before the PS5 arrives will have some degree of future proofing.

In April, at Facebook’s F8 developer conference, the company announced that its Oculus Rift S and Oculus Quest VR headsets would begin shipping on May 21, each of which sells for $399. The Oculus Quest is interesting in that it cuts the cord, offering a VR experience without a separate PC, console or smartphone, while retaining sensors that allow six degrees of freedom. It is powered by a Qualcomm Snapdragon 835 chipset and has OLED display panels capable of displaying 1440×1600 resolution per eye with a refresh rate of 72Hz. It comes with two hand controllers. Time will tell if it finally hits the sweet spot of cost, quality and user experience that could pull VR into the mainstream.

The Oculus Rift S is more powerful, but must be plugged into a computer. It has a fast-switch LCD panel with a resolution of 2560×1440 and an 80Hz refresh rate, according to the company.

Resolution’s Place in the Virtual World

The two primary game engines—Unity and Unreal—both offer pathways to bring 3D CAD data into AR-VR-MR environments. With a rich game history, the engines can support high-res imagery with a high level of details. But to take advantage of the feature, the AR-VR hardware itself also needs to be capable of producing and sustaining the same type of visuals.

“We’ve developed ways to bypass hardware limitations with Unreal Engine features—such as pixel streaming and multi-user collaborative editing—in order to bring the visual fidelity of high-end PC environments to social experiences,” says Marc Petit, GM for Unreal Engine, Epic Games. “This allows end users to share interactive content utilizing a range of form factors, from headsets to LED walls and CAVEs, all while saving time and money.”

“In VR, some enterprise applications have a hard requirement for higher resolutions so users can interact with complex 3D models and scenes. These applications could involve examining the fine details of a full-size car inside and out, or reading instrument panels in a cockpit during a flight simulation,” says Timoni West, Unity’s director of XR. “AR is all about context: Where the user is in the environment, who they’re with and what they’re looking at. The world is augmented around them, and as they get closer to a piece of digital content it will be rendered at the highest fidelity possible. The eventual goal is that 3D content and reality are almost indistinguishable.” (For AR-VR differences, see sidebar above right.)

Some devices use what is called foveated displays, which is a method that mimics how real human eyes perceive images. With the mechanics of the human eye, the focus zone appears in sharp detail, but the peripheral areas are blurred. Mimicking this approach could let AR and VR applications reduce the rendering burden.

To accomplish this, some form of eye-tracking technology is needed, as the device needs to know where the user’s eyes are focusing to determine where to render in high res and where the resolution can be reduced. Therefore, foveated display’s rendering advantages also come at a cost—the added weight of eye-tracking sensors. Varjo’s VR-1 is one of the devices that employs foveated displays.

A Cautionary Tale in Meta Vision

Founded by the tech entrepreneur Meron Gribetz, Meta Vision came online in 2013. With impressive working prototypes, Meta was able to convince big names in engineering and design, including SolidWorks, to pledge support for its hologram-driven AR system. Workstation maker Dell also signed on to become a Meta 2 developer kit reseller. (For more, read “Modeling for the augmented reality age,” DE, August 2017.)

But this January, reports emerged to reveal the company was in financial trouble. (“AR headset company Meta shutting down after assets sold to unknown company,” The Verge, January 2019.)

Meta’s technology, demonstrated at several trade shows and seen by DE editors on various occasions, proved solid. Priced at $949, the Meta 2 developer kit seemed to straddle both the enthusiasts and the professionals. The Verge article attributed Meta’s downfall to a derailed round of funding with overseas investors and an ongoing patent lawsuit.

The rise and fall of Meta demonstrates that even with the right technology and a groundswell of support, a startup in this space faces unpredictable challenges. In this respect, established larger players such as Microsoft, HP and Facebook’s Oculus represent a safer bet for developers with limited resources and software makers that need to decide which devices to support.

More BOXX Technologies Coverage

More Dell Coverage

Artificial Intelligence for Design and Engineering Workflows

In this white paper, learn how artificial intelligence and machine learning can improve design and simulation.

More Lenovo Coverage

Making the Case for Engineering Workstation Upgrades

In this Making the Case whitepaper, Lenovo outlines how upgrading to the latest generation of professional workstations can provide a return on investment through increased engineering efficiency and greater flexibility.

More NVIDIA Coverage

More PTC Coverage