A GUIDE TO LIBRARY MIGRATION: PART 1 – COMMERCIAL CONSIDERATIONS

Whether for preservation or monetisation purposes, looking after the health of your content library has never been more important. In part one of our guide to library migration, LMH Managing Director Gary Edwards takes an in-depth look at the key commercial considerations around the question of asset library migration.


Download this guide as a PDF
Contact us about your library migration project


LIBRARY MIGRATION: MAKING THE MOST OF YOUR ARCHIVE CONTENT

Whether you are a content owner/producer, a national archive or a rights holder, the concept of vast archives of historical content will be anything but alien. What may be more surprising is the huge quantity of this content which – despite the move to a digital era where file-based formats are used almost exclusively – is still stored on legacy-format media.

This archive content may have remained untouched for years, either for want of accurate cataloguing or because the commercial justification was lacking to think about doing anything with it.

In the case of preservation archives, the argument for library migration is two-fold: longevity of your assets and a reduction in physical storage, be it for financial or practical reasons.

The same applies for commercial libraries, with the added – and significant – incentive of the prospect of monetising your content, so that the argument for bringing your archive into the 21st century suddenly becomes a self-justifying one.

With an explosion in the demand for content driven by SVOD/TVOD (subscription/transactional video on demand) providers such as Netflix, Amazon’s Prime Video, Hulu and Disney, as well as OTT video delivered via social media and other internet platforms, there now exists a huge market ripe for leveraging your video assets.

This guide explains some of the key steps and technical considerations you should take when looking to migrate your library and outlines the services we at LMH can provide for you.

THE BUDGET – OTHERWISE KNOWN AS THE “GO” BUTTON

We hear a lot about the “Go” button. Invariably this means getting permission to go ahead and spend some money servicing your archive. It is the most important chapter in this guide.

For an organisation to press the Go Button there are a number of questions and situations that need to be addressed. The most common are:

“WE DON’T KNOW WHAT WE HAVE”

By the very nature of the challenge, knowing what you actually have in your archive can be an impossibility. Organisations no longer have VTRs to check a tape. Information about duplicates, dubs, euro variants, rushes or missing parts is only available in the records and even that can be sketchy. What usually happens at this point is that it’s deemed just too complicated to do anything and the status quo remains.

Prior to digitisation researching the metadata you do have is an option, albeit one reliant on laborious and time-consuming human intervention.

“THE LIBRARY IS IN A MESS”

Library inventories will often be in physical form and/or as an electronic data base. Using physical records such as spine labels and record reports whilst marking up the XLS inventory can yield quick results and a slimmer library for digitisation, but is unlikely to prove an effective way to maintain your archive long-term. The reality is digitisation and subsequent hosting in a cloud environment for viewing, curation and monetisation is the only sensible option.

“DIGITISATION COSTS TOO MUCH”

There is often a misconception regarding the cost of digitising your legacy content for library migration. Using the architecture, automation and agile project monitoring and even the cost-effectiveness of being located out-of-town, LMH can deliver at very affordable price points. Often a return on investment can be achieved in just 12 months when set against the cost of physical storage alone – to say nothing of any monetisation revenue.

“IT COSTS TO GET OUR TAPES OUT OF STORAGE”

We can often assist in this regard, especially on larger-volume projects. Also, once the assets are with us – and we can hold over 100,000 tapes – we will store them free of charge for the length of the project or sometimes even longer. Once the project is finished, we can help dispose of the tapes in an environmentally friendly way, helping you meet your green obligations.

“LETS JUST CARRY ON PAYING FOR STORAGE”

Why? You are throwing good money after bad – the fact is the costs associated with supplying ingest services can only ever increase. It is a matter of supply and demand.

The Time to press that Go button has arrived, folks. Your precious content is degrading every day you wait.

Mass library migration is a peculiar mix of old and new. Take the youngest, most modern tape machine – an HDCAM SR. This video tape machine is now 10 years old! These machines are rare – sometimes very rare – commodities. They have to be hoarded and stored for years, with non-working models being harvested for spares to service migration projects, and the number of machines left in working order dropping all the time. The skill sets to fix the machines are just as rare: I am 54 and I was one of the last VTR engineers in the field.

PRESERVATION AND MONETISATION – THE DRIVERS

DRIVER FOR MONETISATION

Due to improved connectivity and reduced costs, today’s cloud-enabled environments have allowed the Artificial Intelligence process to flourish. There are myriad algorithms and associated workflows readily available to weave their magic over your precious archive.

From scene detection for product placement and text analysis to extract metadata from content via keywords using natural language understanding, to machine learning, there are now many products to serve your content.

A common blocker we come across is the cry “we don’t know what we have, we may have duplicates” – again, using cloud-based analytics like Own Zones Connect or Discover can clean up the format clutter archive and also detect duplicates.

Companies such as Mirriad have award-winning algorithms for searching your content and finding the perfect spot and then placing that billboard just in the right place, potentially a place that would not be immediately obvious to a human. The results are great and hard to believe the product placed there did not exist when the movie was originally shot.

These processes open up many possibilities to breathe life – and as importantly value – into the library. This in turn releases budget, and budget enables action.

There is one problem you may encounter: your library might still be in its original form.

PRESERVATION TO ENABLE MONETISATION

There are many considerations when preserving a library, many of those physical.

What can get overlooked these days is that many formats now suffer from some form of physical degradation. For those of us that have worked in the media and entertainment industry it is easy to think that digital formats such as Digital Betacam, HDCAM and DVCAM and Pro etc are immune to trouble: this is not the case.

Correct storage conditions, temperature and humidity control are key but often the damage was done many years ago. Therefore, remedial techniques have been developed to counter this.

As is more widely known, baking tapes at the very consistent temperatures using laboratory ovens does allow a window of opportunity to replay the tape successfully and without damage to your machine or asset. The tape will revert to its former state after a week or so but at least you now have the electronic file asset.

Other issues that can affect transfer are mould, this is caused by excessive moisture when storing the tapes and requires a full risk assessment process. Face protection for handling and cleaning is required for staff. Some mould outbreaks are so severe that they have required experimental treatments to be tested, such as the use of formaldehyde.

Moving beyond physical restoration considerations, an appropriate codec choice is critical. This is the main separator in terms of rationale, i.e. Preservation or Monetisation. File size and thus storage costs are crucial to all asset libraries, but especially important to commercial archives being processed for monetisation.

The chosen codec needs to be “good enough” to resolve the level of quality required to serve the monetisation requirements but not “over done” to the extent that it raises storage costs and increases the complexity of property applications needed to resolve the content.

LIBRARY MIGRATION FOR MONETISATION: USER CASES

Internal Production Library: This might be for example a reservoir of legacy content that can supplement programme output at a fraction of the cost of making new content. These types of library can feed the recycling of content, especially within post/distribution companies in the mid-level content market. Once digitised, the content can be analysed by scene detection algorithms such as Gray Meta Curio to create a Media Asset Management index using the file metadata. This enables quick access to the relevant files for production use and is an effective way to recycle content and save money.

SVOD/TVOD: A more distribution-focused rationale, here the codec and wrapper choice or choices are targeted at perhaps a mezzanine – master – proxy approach. The mezzanine file would be the file you “put in the bottom drawer”, i.e. the top-quality file to keep. The master is the work-horse file; this is the asset that you distribute to satisfy your international sales deal. Lastly the proxy: this is how you achieve the sale in the first place, the play-anywhere-on-any-device file. This tiered approach may cost more at the outset but provides a full set of materials to service the packaging requirement of VOD channels.

DIGITAL LIBRARY ACCESS

The location of your archive is a key consideration. Commercial cloud, hybrid cloud or on-premises are all valid options, and the final decision is usually closely linked to the reason you are considering library migration in the first place.

User access to the library is very important. Preservation organisations tend to prefer on-premises data storage due to increased access control, as well as skill-set considerations. Also, their user case tends to need a much wider access architecture, often extending to public access. For the commercial library owner access is equally important but they will tend to focus more on content security and digital rights management. Typically, their user base will be much more restricted.

CODECS – TECHNICAL CONSIDERATIONS FOR OPTIMAL MONETISATION

Which codec and which wrapper to ingest to are complicated choices – and crucial because they have a massive effect on the commercial success of your project.

If your choice is too proprietary your downstream infrastructure becomes costly and unwieldy; too high a data rate and your storage becomes too expensive. Conversely the opposite can have a detrimental effect on quality and thereby on the markets you can sell into, again all impacting on the cash generation of the project one way or the other.

As an example, at LMH we have in the past preserved collections of 8,000 tapes, most of which were under 30 minutes. This was to an uncompressed file format; this generated a storage pool of over 750TB. That is a serious amount of data!

FFV1: THE NOT-SO-NEW KID ON THE BLOCK

FFV1 is a lossless intra-frame codec that allows significant reduction in storage size whilst retaining quality equal to an uncompressed file. For that reason, it has gained popularity amongst video archivists.

On a recent project we were obtaining an average compression ratio of 2.5:1, with a maximum of 5.5:1 and a minimum of 1.5:1. To put this in perspective, at the average rate, this would reduce an archive of 100 TiB to roughly 40 TiB.

Total reassurance is provided by using a process known as framemd5 which checks every single frame from the source against the output to show each frame is identical.

Combined with the flexibility of the Matroska (.mkv) container, it opens up possibilities to archive associated metadata embedded within a single video file.

Given the world of cloud storage, and monthly costs, a data reduction of this size will provide significant economies over time.

One downside is commercial video digitization hardware does not natively support the FFV1 codec, specialist skill sets are required to drive any FFV1 system typically via FFMEG command line control.

The good news is that at LMH we have a lot of experience and practical knowledge in this area.

LIBRARY MIGRATION CASE STUDIES

ATP MEDIA

ATP Media undertook a project to host its non-commercial archive in a new cloud location, managed via a new MAM.

LMH was chosen for the library migration project. This involved feeding content to the archive storage location Object Matrix, with footage from over 6,000 tennis matches filmed between 1990 and 2009.

Consisting of HD and SD tape material, the archiving project required LMH to devise an extremely detailed workflow. This included a full clean of the tape in an Indelt TCMatic tape cleaner, the preservation of all tape metadata, asset labelling, collation of record reports into one automated xml report, full monitoring of the Sony ISR tape channel condition data and multiple Sony Flexicart ingests via MediaFlex Mediacart control.

The ingest was carried out using our SERVOLOCK mode, which enables the file recording to be automatically started and stopped without the need for any human pre-qualifying intervention.

Key Focuses:

– High attention to quality and QC
– Volume
– Preservation
– Upload to private cloud location batch

 

BFI – HERITAGE 2022 PROJECT

Launched in 2016, the aim of this project was to digitise 100,000 items of video by March 2022.

The programme comprises selections from 15 national collections, including:

– Imperial War Museum
– Wellcome Trust
– Regionals such as SASE and NWFA

LMH is very proud to have been selected as one of the vendors tasked with achieving this enormously significant and ambitious preservation project.

KEY FOCUSES

– Capture the entire content of the tape
– Large volumes – over 20,000 tapes
– Point-to-point electronic delivery to the BFI
– Project progress updates

PRESERVATION AND MONETISATION: THE SAME THING?

No not exactly, but what is true is there is a symbiotic relationship. If you first preserve, you can then monetise.

Without sounding like a doom merchant if this year’s events have taught us nothing else it is the status quo will always break. The supply of your content from the normal channels can suddenly stop. We only have to think back to 2011 to the tsunami that destroyed Sony’s tape factories in Fukushima which sparked the beginning of the end for tape in our industry.

This is why putting off addressing your most precious items – your content legacy – is a very dangerous tactic and in fact not a cost saving at all.