• by Tanya Gazdik , May 17, 2023

BMW and Meta are continuing to work on mixed reality usage in an automotive context.

Meta Founder Mark Zuckerberg posted an update on Instagram showing the gains that have been made.  

The deal between the two companies was originally announced in 2021. The BMW Group says it is open to including additional partners to create an industry standard for connecting XR consumer electronics devices to vehicles.

The BMW Group Technology Office USA in Mountain View, California presented a research update with Meta’s Reality Labs Research demonstrating the ability to accurately display stable virtual reality (VR) and mixed reality (MR) content to passengers in a fast-moving car, even when making turns, going over speed bumps, and accelerating. 

By integrating Meta Quest’s tracking system with BMW’s sensor data, the researchers enabled a series of stable “car-locked” gaming, entertainment, productivity, and meditation experiences.

BMW envisions a number of potential use cases for extended reality (XR) devices in vehicles, says Claus Dorrer, head of BMW Group Technology Office USA. Those include assisting a driver in locating their car in a crowded parking lot, as well as  alerting them to hazards on the road and surfacing important information about the vehicle’s condition.

“It is too early to tell exactly how or when this technology will make it into customers’ hands,” Dorrer says in a release, but the implications of future AR glasses and VR devices — for passengers as well as drivers — are promising. 

"The research partnership with Meta will allow us to discover what immersive, in-vehicle XR experiences could look like in the future and spearhead the seamless integration of such devices into cars,” Dorrer says. 

The proof-of-concept prototype has overcome some key technical challenges. VR headsets use cameras and motion sensors to understand their exact position, so when the user moves their head or looks around, the virtual content remains stable. 

“Our research prototype shows that we can enable entertaining and comfortable passenger experiences that are anchored to the car itself, including VR and MR gaming, entertainment, productivity, and even meditation capabilities,” says Richard Newcombe, vice president of research science, Reality Labs Research, in a release. 

The technology has the potential to transform how we can safely interact with our environment while traveling, and as we progress into reliable world-locked content on the road to AR glasses, he says.

"We’re hopeful it will be possible for passengers to see things like markers for landmarks, restaurants, places of interest and more,” Newcombe says.