What Challenges Does File Based Ancillary Processing Bring?

Screen’s CTO Simon Hailes discusses the issues around file based processing of media.
Since launching MediaMate four years ago as Screen’s platform for performing file based non-real time processing, Screen has encountered many challenges.
MediaMate was a response to the anticipated move towards increasing use of offline processing of files in the broadcast domain. Its structure is designed to be general purpose, but since our main focus was Subtitling, the first applications were obviously subtitling based.
Re-purposing Screen’s core DVB IP into a non-real-time environment proved very effective – consumer grade compressed streams were quite easy to handle from an IT perspective, and early implementations of DVB Bitmap and DVB Teletext subtitling were efficient in the customer’s environment, even with moderate hardware performance.
As we extended the number of modules in the product, we began to repurpose more IP relating to broadcast features, for example Teletext subtitles in the form they take in video server files. The application of the technologies was not difficult, but the challenges started to multiply.
One of the major challenges is that despite standards being in place for the association of ancillary data with the video, the implementation of these standards is varied.
For example, SMPTE-436 (a standard for the storage of VBI/ANC in MXF) generally works if you record it on one server and play it back on the same type of server, but if you take the file and try to play it on a different manufacturer’s server it may pose some challenges.
We had to create a product which was capable of generating 436 which will work on the server of the customer’s choice.
Other differences place specific limitations on the use of SMPTE-436, resulting in the configuration having to be tuned to the customer’s site, often with little information about what the requirements and limitations are.
The adoption of SMPTE-436 is not universal, and numerous other ancillary formats are in use. IMX (‘Tall video’ e.g. where 608 lines are encoded for PAL such that the VBI is included in the coded video) still makes a lot of sense for SD – you can see the data when you play it back on any PC player. Hence we needed to consider and include a module which can insert directly into I-Frame video.
Other types of data association include data in user data of the video coding, standards based or not, proprietary coding mechanisms including side files (e.g. Omneon VBI files), separate streams (legacy Pinnacle).
Many people we encounter are looking to adopt a standards based approach, and recognise that reading of legacy formats is useful, but insertion of these formats is less attractive. There are exceptions however; Screen for instance has an on-going enquiry for insertion into GXF, not because the customer wants to continue coding to GXF, but because they have thousands of legacy files and insufficient capacity to re-encode.
Additional challenges relate to the size of files and the IT infrastructure.
As HD becomes the standard for video assets, the file size has increased. On GigaBit Ethernet, it’s not unusual to be able to get 350 Mbits transfer rate in a CIFS copy to/from a fast SAN. When a file is being read and processed, ~250Mbits can be achieved.
For HD, this rate is not much faster than real time. In a recent installation, the lack of bandwidth was compounded by the operators using the same machine to move additional files, so reducing the bandwidth available. With two processes to do on each file (due to a requirement for legacy encoding in parallel with SMPTE-436), the overall throughput reduced to quarter real time (4 hours for one hour of media). Using 8 GigaBit Lan and dedicated processing servers, it is anticipated that the throughput could be increased to twice real time (half hour per hour of media).
So, should you consider coding ancillary data in the video server file?
It certainly is achievable. Data in the asset seems attractive at first, and has the anticipated advantages of the ability to QC the package ahead of broadcast. But it also has significant disadvantages; late ancillary assets cannot be coded, the ever present aspect of the essence data not being editable, if for example you switch away, a subtitle may remain on air, and the additional complexity it brings operationally.
If you do choose this route, ensure that the IT requirements are not under estimated, including what other processes may be accessing the servers concurrently and look at the penalties of pre-processing as opposed to real-time insertion at play-out.
www.screensystems.tv
viewer.zmags.com/publication/381c52e3#/381c52e3/1