Dynamically Optimising The Storage Costs Of News Archives

News archives are inherently frustrating from a business perspective, says Mike Palmer, Vice President, Business Strategy, Masstech. While some content will ultimately prove to be very valuable and important, much of the archived material will likely never be used or monetized.

The ROI in a news archive resides in a relatively small number of stories and video assets, with that value only realized at a future date. The rest of the archived content is simply a drain on resources. The obvious problem is that it is difficult, if not impossible, to accurately predict which stories will have future editorial value, and which ones will not.

However, the editorial behavior of most news archives does generally follow a consistent pattern: the longer a story or asset remains in storage without being touched, the less likely it is that it will be used in the future. There are exceptions to this model, of course, but many of these are known – for example, major world events that are likely to be commemorated annually or decennially – and that content can be flagged at the time it is saved based on the user's editorial judgement. Understanding this pattern presents the opportunity for many news organizations to reduce their overall storage costs by dynamically managing the characteristics of the video files they are storing, without deleting content.

The cost of archive storage is a function of the number of stories in the archive, the length of time stories are saved (i.e. their age), and the size of the video files associated with each story. This is generally true whether your storage medium is solid state memory, spinning disk, digital tape or the cloud. As such, reducing the size of unused – or highly unlikely to be used – video files directly results in lower storage costs.

Historically, once video has been written to the archive, it was assumed that the file will never change. For some organizations this may be the correct business decision, but for others, the ability to reduce video file sizes later during the lifecycle of some content may represent an opportunity to reduce costs and/or enable more content to be saved in the storage they already own.

Making video file sizes smaller typically involves lowering their bit rate, which can have an impact on quality – although not necessarily to a degree that will be noticeable to viewers in the resulting productions. News video is most often archived at production bit rates, which produce large files. These high bit rates allow the video to be returned to production workflows at a later date with no loss of quality. However, keep in mind that all production files are eventually knocked down to much lower distribution bit rates for delivery to consumers. If we can accept storing unused files at bit rates normally used for distribution, we can achieve a 50% or greater reduction in long-term archive storage costs.

Consider an automated process that, after a specified amount of time has passed since a story was written to the archive or last retrieved, transforms the corresponding video file from production to distribution bit rates. In effect, this is a conscious trade-off in predicted story value versus storage cost. The fact that we never actually delete the older assets hedges our bet while still allowing significant storage savings.

How big an impact might changing the video bit rate have? As an example, let's look at 120 TB of storage, equivalent to a 50-slot digital tape library with LTO6 drives. When archiving at 50Mbps, a newsroom that archives about 20 hours of content per week can store roughly five years of content. If we instead archived all stories at 10 Mbps, that same storage could contain 25 years of content. And if that same tape library is upgraded with LTO7 drives, the increased storage density would provide more than 280GB of capacity in the same physical footprint, into which a whopping 60 years of content could be archived.

Of course, we don't want to reduce absolutely everything to the lower bit rate; we want to keep the content most likely to have future value at production quality. The inclusion of a mechanism for journalists to flag the "good stuff" when they first store the content enables the automated process to leave those clips unmodified, thus optimizing the bit rates of archived video dynamically without requiring any additional user actions.

For more detail on this strategy for controlling your news storage costs, please see our white paper at http://info.masstech.com/news-storage.

(JP)