xRS Week: AR-VR Training Apps Get More Accurate with Physical Props, Eye Tracking, and Hand Tracking

Hosted by the analyst firm Greenlight Insights, xRS Week kicked off in San Francisco this month (Oct 16-18). A strategy conference on extended reality, the event brought together pioneers, heavy weights, and startups in the augmented reality and virtual reality (AR-VR) market.

At the conference, Greenlight Insights released a report titled “Optics in Focus: The vital role of the optical interface in unlocking the true potential of AR.”

“By not engaging our natural visual cues, we are limiting users’ ability to accurately gauge distance and interact with any degree of precision with nearby computer-generated assets or interfaces, focal rivalry, and vergence-accommodation conflict (VAC) represent a critical threshold for quality [in AR head-mounted displays],” the paper points out.

Eye tracking brings new insight

In the presentation titled “Rethinking Transformation in the Age of Immersive,” Vinay Narayan, VP of Platform Strategy, HTC, discussed the role of eye tracking in firefighting simulation.

“With eye tracking, we can finally understand what the firefighters are looking at. You combine that with heart rate sensors, and you can correlate what they're reacting to, what they overlook. Tunnel vision is actually one of the biggest causes of injury among firefighters,” he noted.

The firefighting simulation apps from Flaim Systems, based in Victoria, Australia, incorporates Vive and other VR headsets. The company offers two products: Flaim Trainer, which simulates professional firefighter scenarios; and Flaim Extinguisher, which simulates domestic and daily fire hazard scenarios.

Varjo, another firm at the conference, recently released Varjo VR-2, the second version of its VR hardware. The system includes eye tracking, allowing users to use eyesight itself as for navigation and selection (like a mouse or a pointer) to activate context-sensitive menus and commands.

Replica equipment adds realism

The Greenlight Insights paper also states, “enabling close interaction and naturalistic control via hand tracking is an important milestone for any AR system. These developments show the importance of bringing content into arms reach and point to the need to develop an optical interface that can effectively engage our visual system to give accurate depth cues.”

In conversation with Greenlight Insights Analyst Ben Delaney, Joel Breton, President of Sixense Studios, Sixense, observed, “Training is the killer app. You can train people faster, cheaper, and they retain what they learned much longer.”

Sixense focuses on healthcare, training, and entertainment with its VR offerings. The company's technology is incorporated into the welding and vehicle spray painting applications deployed by Lincoln Electrics and Honda, among others.

In Flaim Systems' firefighting applications as well as Sixense's training apps, the use of replica equipment plays an important role. In Flaim Trainer, users use a haptic-enabled water nozzle that can simulate water pressure. In Sixense's spray simulation application, users use an industry-standard paint gun.

The incorporation of eye tracking, hand tracking, and replica equipment that help build muscle memory brings AR-VR-powered training to a new level, setting it apart from entertainment applications that focus primarily on immersive visuals.