Audi Design Used Magic Leap 2 to Develop a New AR-Based Driver Interface

This January Audi presented the the Audi activesphere—a concept car featuring a new AR-based driver UI designed and created with Magic Leap 2. The Audi Design Studio, which conceived and designed the fourth of their sphere concept cars, focused on advancing Audi’s vision of premium mobility forward. Audi dimensions, the new UI concept, uses augmented reality to fundamentally transform the driver experience.

“Removing physical displays and touchscreens creates a new experience space for the passengers while using the power of mixed reality opens up new possibilities to intuitively integrate them into the mobility ecosystem and it goes even beyond the car,” said Jan Pflueger, Creative Technologist at Audi Design.

To achieve this, Audi used Magic Leap 2 to develop a dynamic, MR-based interface, designed to transform the driving experience. As opposed to the limitations of conventional dashboards, the AR interface enabled by the expansive, up to 70 degree Field of View of Magic Leap 2, is an immersive, reconfigurable layout—seamlessly integrating digital content within the driver’s real-world view of the road and cockpit where and when it is relevant.

The interface is designed to be individually configurable for both driver and passengers to access features like selected drive mode, location, and destination, pulling content and arranging itself for the driver in the most useful, usable configuration. Audi leveraged the superior optics of Magic Leap 2—designed for precision and low latency—and took advantage of its open platform to give users the ability to interact precisely with these MR-based information zones.

Audi is using both hand and eye tracking to display information virtually while also recreating a sensory touch control. Virtual content initially is only viewable, but by focusing their eyes on specific information, the cluster of information expands to convey more detail and enables interaction through hand gestures.

The car’s functions can be controlled simply by intuitively following the user’s gaze, while the MR-based user interface reacts to changes in real time like a conventional instrument. A particularly user-friendly feature, the virtual control moves towards the user so that they can interact comfortably with the user interface from any position.

“The centerpiece of the new system is innovative mixed reality headsets – available individually for each driver and passenger. Users also have access to a comprehensive digital ecosystem while they’re in the Audi activesphere,” Audi explained in a press release announcing the project. “The Audi activesphere concept is the first to use a pioneering generation of this technology, which in turn adds the dimension of interaction to the dimension of superimposed real and digital worlds. With unprecedented optical precision, highest resolution, and excellent contrast, the system brings control surfaces and displays, invisible to the unaided eye, into the user’s field of vision while behind the steering wheel.”

The beautifully minimalist interior design that Audi is known for is fully realized through the use of Magic Leap’s AR technology. Only when users need an element does it appear, and it can be operated just as intuitively as it would in the real world. The diverse functions of the car are now not organized in a typical car with screens and physical function, but instead contextually integrated within view of the elements they are related to. Since the Magic Leap 2 sensor array maps the interior with millimeter precision, content can be displayed according to personal requirements and even used for individual interaction.

According to Audi, “The possibilities of this technology are by no means exhausted. In off-road mode, for example, high-resolution 3D topography graphics can be projected onto the real landscape and information on navigation and the destination can be displayed. Traffic safety information, i.e., alerts for traffic jams or slippery roads, can also be used here. The connection between the headset users and the car, along with its ecosystem, offers countless possibilities, even when outside the car.”

Audi is a participant in Magic Leap's Early Access Program.

Read Audi's press release here.