Broadcast News

Bookmark and Share
10/06/2014

4K Imaging In Augmented Reality

News Image
The buzz surrounding 4K, or Ultra High Definition, has played a leading role at every key industry trade show during the last twelve months, writes Andrew Butler, applications manager, Vitec Videocom.

There has been a raft of new products released to support the 4K trend at every stage of the broadcast workflow. At point of capture, the latest professional Ultra HD camera ranges from the likes of Sony, Red and Blackmagic, have further fuelled the interest.
The 4K spotlight has also been shining on the need for the highest quality lenses capable of delivering enough resolution to project a crisp and error-free 4K image on to the camera sensor. There is also the matter of needing adequate bandwidth in cabling and the latest wireless technology to transfer these higher resolution images.
While an increasing number of broadcasters and telcos currently have the infrastructure in place to make 4K a mass market proposition, advances in encoding such as HEVC and possibly even Google's VP9 are set to be key enablers for making UHD more practical. HEVC's compression rate will probably see UHD needing only between 10-20 per cent more bandwidth than full HD, making it entirely feasible for the mass market. In addition, when it comes to infrastructure the wider roll-out of fibre is expected to iron out these issues in the medium term.
As this occurs, rights holders and content distributors will be looking for additional revenue streams to offset the costs of upgrading productions and infrastructures to 4K standard. One method open to them is the use of augmented reality advertising during live events, notably sport where it has become a common feature.
High precision movement control and the performance of the lenses is critical to success for any 4K application, however there is an additional consideration to factor in when adding augmented reality to the mix. This is to provide sufficient resolution of pan and tilt positional data for the UHD frame when using shots at the extreme tight end of the zoom range.
The aim of any augmented reality system, such as those used for inserting a logo for a sponsor onto the pitch during an international rugby match, is to seamlessly composite an image of the real background with a computer generated view of a 3D model to the extent that it appears physically 'real'. This requires highly accurate pan and tilt positional information for the camera, which is typically provided by an encoded head such as the Vinten i-Series encoded heads.
Encoders in the support fluid head measure the direction of the camera and the encoders in the lens measure the position of the lens barrels or motors. Any minor delay or backlash between optics and encoders results in an error in the data sent to the rendering engine. The erroneous data will define the viewing vector and field of the virtual camera viewing the 3D model, which in turn will damage the illusion of both images existing on the same plane.
In developing the Vinten i-series products, the standard resolution required was calculated to meet the requirements of a 1080p high definition production using one of the longer lenses, such as a Fujinon XA101x8.9 BESM, that a typical sports production might deploy.
When the lens is zoomed all the way in, its horizontal angle of view might be just 36 minutes, or 1/600th of the 360 degrees the camera head could pan. On a 1080 HD frame, the horizontal pixel count would be 1920. By multiplying the horizontal pixel count of 1920 by 600 that informs how many data points would be needed in the head's 360 degree rotation in order to resolve 1920 pixels in the 36 minute of horizontal view, the highest telephoto point of the zoom lens.
The overall result is it would take 1.15 million data points through 360 degrees of rotation in order to adequately describe the pan angle of that camera, zoomed all the way in, at pixel level. To put that in perspective, Vinten's i-series of encoded heads provide a minimum of 1.48 million data point per 360 degrees of panning rotation, which surpasses the level of performance necessary to resolve each individual pixel.
When the positional data resolution in an augmented reality system is below one data point per pixel, if an operator pans the graphics that are composited with the video it could potentially oscillate between pixel one and pixel two, and the rendering computer will not be able to resolve the precise angle. The worst case scenario could involve the rendering computer becoming so confused as to where the graphic should be composited that it would show either a fuzzy graphic image when the camera pans or tilts, or a the graphic floating slightly over the composited video.
For 4K augmented reality applications, double the number of data points would be required, equating to 2.35 million through 360 degrees of rotation of the pan and tilt axis. This would appear to be a higher resolution than the Vinten heads appear to output, however the encoder technology in the i-series can deliver up to 12m counts per 360 degrees. As the heads and the VRI interface box architecture are designed to enable this upgrade with no modification, it can be exchanged by an engineer in a matter of minutes.
This ultra-high precision measurement of camera viewing vector is vital in augmented reality 4K productions to ensure the graphics are perfectly located, move smoothly and therefore look part of the real scene to viewers.
In terms of the zoom and focus data, it is unlikely that lens manufacturers will have to change the encoding performance when designing new 4K lenses. An HD lens typically has around 55,000 encoder counts between telephoto and full wide angle, the two stops on the zoom. Shooting in 4K with a 4K lens, sensor size being equal, it is still the same focal length difference between the largest and shortest focal length of that lens. Therefore the 55,000 encoder counts should be adequate enough for the rendering engine to use a mathematic model to determine what the field of view is, and the magnitude of any distortion in the image to a degree that the compositing looks real.
As 4K requires higher bandwidth video transport systems, it would be reasonable to think that the camera tracking information for 4K would follow suit requiring a four-fold increase in data flow between the VRI interface box and rendering computer, potentially overloading the bandwidth of the serial connection between them. Not so with the Vinten system, which was architected so that all of the processing and scaling can be completed in the VRI box and the data packet transmitted has the same format and size, regardless of the resolution of the image and therefore the encoders.
An increase in the frame rates (which could be beneficial to motion quality in 4K) would result in a higher volume of data transmitted between the VRI and rendering engine. However, in the VRI system there is sufficient processing power and bandwidth to handle the data required at up to double current frame rates.
The excitement and testing activities in recent months have indicated that, augmented and virtual applications in 4K are becoming a reality for increasing numbers of broadcasters and independent production/OB companies and that any future 4K productions will rely not only on the latest camera equipment, lenses and transmission, but the highest performing, precise capabilities of the camera support equipment.

The article is also available in BFV online.
(IT/JP)
Solidmate Ltd Memory Card Hire London

Top Related Stories
Click here for the latest broadcast news stories.

16/10/2023
Wooden Camera Now Taking Pre-Orders For Its Elite Accesory System
Wooden Camera is now taking pre-orders for their Elite Accessory System for RED® KOMODO-X™. This collection offers an Elite System for users who want
12/10/2023
Wooden Camera Launch Elite Accessory System
Wooden Camera has announced the launch of pre-orders for its new Elite Accessory System for RED® KOMODO-X™. This collection offers an Elite System for
09/06/2014
4K OK? Understanding UltraHD... (Pt 2)
"The latest workstations and professional laptops are now up to the task of 4K content creation and NLE developers are all beginning to support native
09/06/2014
4K OK? Understanding UltraHD... (Pt 1)
In a bid to get his head around the intricacies of 4K format ahead of this summer's landmark broadcasting event, editor Iain Todd spoke to some indust
04/06/2014
BroadcastAsia2014: 4K To Transform Onscreen Viewing
HD (high definition) viewing will soon be a technology of the past, as a result of the proliferation of 4K coming Asia's way. 4K, also commonly known
04/04/2014
Sony And FIFA To Collaborate On A Range Of 4K Initiatives
Sony and FIFA (Fédération Internationale de Football Association) have announced they will collaborate on a range of 4K initiatives at the 2014 FIFA W
30/08/2018
Gearhouse Broadcast Invests In Canon And Fujinon Lenses
Gearhouse Broadcast has invested in the latest 4K studio/field and ENG lenses from Canon and Fujinon. The new delivery continues Gearhouse's commitmen
20/03/2024
ARRI To Unveil Technology Portfolio Of Camera Equipment At SportsInnovation
From March 20-21, 2024, the biggest and brightest names in international sports and media technology will gather for SportsInnovation at the Merkur Sp
14/10/2014
JVC Unveils 4K Technology Prototypes
The electronics giant unveiled new additions to the 4K family at this year's IBC show, including the new GY-LS300 Super 35mm sensor 4K camera... GY-LS
29/04/2014
Asia Gears Up For 4K
4K is "coming Asia's way," according to the organisers of BroadcastAsia, the digital multimedia and technology conference taking place this year in Si
12/04/2018
Few Lens Optics For Decades, Then Came The Large Digital Sensors
As software takes control of virtually every production and delivery function, cameras and lenses have reached the stage of looking magnificent as the
12/08/2015
Pyser-SGI Announce Two New Fujinon Ultra HD Lenses
The market need for 4K video content is surging amidst a 7-fold, year-on-year increase in the shipment of 4K-compatible television sets to approx. 11.
06/11/2013
Panasonic: Preparing For The 4K Explosion
Panasonic remains one of the world's leading exponents, designers and developers of cutting-edge broadcast and communications equipment. After a year
02/10/2014
JVC Unveils 4K Technology Prototypes At IBC
The electronics giant unveiled new additions to the 4K family at this year's show, including the new GY-LS300 Super 35mm sensor 4K camera... GY-LS300
24/11/2017
ES Broadcast Hire Increases Fujinon 4K Broadcast Lens Offering
ES Broadcast Hire has announced the further expand its considerable range of UHD live production lenses with the addition of a fleet of Fujinon's late