A cinematic experience

Pushing the limits for live AR on stage

When the car becomes an actor, the show becomes an adventure


For Volkswagen's brand event in Beijing we were pushing technical boundaries once again. Three moving cameras, a moving target, four tracking systems and 4k video output.

A live play in a virtual world

For the release of yet another 'Advanced Mid-size SUV' we created a virtual environment for the car to move around in and tell a story. In contrast to prerendered video playback, when moving through the virtual space in real time, the car takes an active role on the stage. It becomes an hero in its own adventure, merging the two worlds in front of the audience's eyes.

We sent Volkswagen's new model on a journey from snow-covered mountains across steep abysses into a futuristic panorama of a modern chinese city – It's a versatile SUV that's mostly driven in cities, but sells the offroad feeling.

Illustrating technological features

The presentation of a new car model often comprises facts and features that are not immediately visible to the audience. With AugmentedReality we can make these hidden features apparent and make power trains, batteries, radar waves apparent and clearly illustrate safety features like night vision and traffic jam assistance systems.

When the real car on stage gets augmented, it is not necessary to produce and render a full film set to convey the message. Everything happens in the here and now, so only parts that are really relevant to the story need to be rendered. Like in a theater play the spectators see an immediate connection between the real object and its invisible properties.

Technology

5 screens, 4 tracking systems, 3 cameras, 1 moving target » one show

The technological part of the show presented a new challenge for us. In addition to the 7.5m mechanically tracked camera crane that was supplied by Egripment, we used a second, optically tracked camera rig that opened up a whole new set of possibilities in terms of dynamics and camera angles.

To increase the dynamics of the show even further, we also tracked the moving car on stage, making it literally drive through our virtual worlds.

On the other hand the sheer amount of pixels that needed to be filled with content was equally challenging for a realtime generated show. The stagebigger than a 4K screen plus an 8k LED surface on the ceiling.

Software solutions

To match this technical complexity we set up a rendering pipeline that was able to cater both the needed flexibility and rendering quality at the same time.

From within our weapon-of-choice 'vvvv' we employed Unity3D as a rendering engine for high-end shaded parts of the scene. Our newly developed process control runs Unity3D in absolute lockstep with vvvv, allowing both softwares to render into the same scene in full 3d. Abstract graphcis and state control from v4, and asset heavy stuff and animations with glossy shading from Unity.

This combined approach opened up a new world of possibilities and set new standards for our work in the future.

Credits

Client: Volkswagen AG
Contractor: BLACKSPACE GmbH
Partner: Acht Frankfurt GmbH
Team: Sebastiano Barbieri, Dennis Boleslawski, Miriam Guter, Alex Henker, Eno Henze, Matthias Husinsky, Woeishi Lean, Christian Tielmann, Valérie Vogt
AR Camera Crane: Egripment Deutschland GmbH
Handheld camera tracking: TrackMen Ltd.
Object Tracking: bright!
© NSYNK Gesellschaft für Kunst und Technik mbH 2018