What's New For Photorealistic Virtual Reality – Tell Me It's Not Real

What would you think if you were able to create, in real-time, photorealistic scenes adding green screen shots of talents and augmented reality graphics, then mix them all together so seamlessly that the audience could not tell which are real and which are not?

You will probably think that people are already used to visual effects, and that achieving such results is a common task in today's film making. However, for live, on-air broadcast operation, photorealism is one of the toughest challenges, if we are thinking on real-time virtual set production, because what really makes the difference for the viewers is to be unable to tell whether the images they are watching are real videos or digital renders.

Creating such realism requires the combination of a number of technologies, hardware and software. Camera tracking is almost mandatory to perfectly match the camera movements of the scene and the characters, while allows shooting the talents as required. But also, the background scenes must have an extremely high render quality, which can only be achieved using hyper realistic rendering technology and carefully designed sets to create photorealistic images. Thanks to its Combined Render Engine, InfinitySet can combine the Epic Games' Unreal engine with Brainstorm's own eStudio render engine, resulting in a perfect solution to create state-of-the-art hyper realistic scenes, which can also integrate essential elements for broadcast operation, such as 3D data-driven motion graphics, charts and many other, providing results which are indistinguishable to real footage, no matter if we are working in HD, 4K or even higher resolutions.

Amazingly enough, InfinitySet not only matches the tracking data of any number of cameras in the scene with the real-time rendered Unreal background, but also controls the parameters of the real lights or the Chroma Keyer as the set conditions require such changes. For instance, if the talent is walking around the green screen and the light conditions change in the virtual set, InfinitySet can tell the real lights on the green studio to change accordingly, and also remotely alter the Chroma keyer settings to maintain the best quality, even when using external hardware. Because of this, InfinitySet can simulate different conditions of the day, different lights, different ambiences, using a single scenario, in real-time and with full control, and even adding volumetric illumination.

Another interesting feature is InfinitySet's ability to cast virtual shadows over the real chroma keyed shadows. These virtual shadows depend on the virtual scene illumination and can affect the virtual elements. On top of that, InfinitySet is also capable of creating accurate selective defocus and bokeh, matching the depth of field of the lenses on a given scene. All of the above adds up to the advanced features which make InfinitySet the most advanced virtual set and Augmented Reality solution: TrackFree™, TeleTransporter, 3D Presenter or VideoGate, to name just a few.

Once the scene is solved, it is also important to understand that Broadcast graphics workflows have specific requirements that are far beyond from what a game engine can do, often requiring external database connections, statistics, tickers, social media or lower-thirds, such a variety of graphics that are alien to the game engine framework but essential for broadcast operation. With the Combined Render Engine, InfinitySet can effortlessly integrate Aston graphics as Augmented Reality objects within the scene, with a render quality that matches that of the Unreal Engine by using Brainstorm's PBR (Physically Based Rendering) rendering capabilities. This allows for adding advanced audio-visual aids such as in-context typography, motion graphics, statistics, charts and many more, all perfectly integrated in the composition both in animation, tracking, perspective matching and of course image quality.

With all the above, InfinitySet's advanced virtual technology represents a real cost-saving solution, as it provides new possibilities for production houses and broadcasters for creating different scenes and environments, no matter if we are talking about sitcoms, news, sports or entertainment shows. Also, this approach avoids one of the common complaints broadcasters have when using virtual technology, that is the quality of the virtual sets, so it makes this technology a real alternative to remote live outdoor shooting or physical set construction.

Author: Miguel Churruca, Marketing and Communications Director, Brainstorm

This article is also available in the May edition of Broadcast Film & Video.

www.brainstorm3d.com