A GUIDE TO LIBRARY MIGRATION PART 2: TECHNICAL CONSIDERATIONS

Whether for preservation or monetisation purposes, looking after the health of your content library has never been more important. In part two of our guide to library migration, LMH Managing Director Gary Edwards outlines some of the technical challenges of a library migration project – and reveals how LMH is perfectly placed to help you overcome those hurdles.


Download this guide as a PDF
Contact us about your library migration project


In theory, making the decision to migrate your content library to a digital platform seems very straightforward. The threat of damage and deterioration of your physical tapes, the gradual obsolescence of formats as fewer and fewer tape machines survive to play them and the fact that industry delivery requirements are now almost exclusively digital mean that there can be little doubt that now is the time to take action.

Just as in Part 1 of our guide to Library Migration we saw that commercial considerations will have an impact on how best to proceed with your migration project, so too do the technical challenges of moving your assets to a digital format. With thousands of tapes requiring processing, finding the best method requires expert help.

At LMH we have established systems that incorporate automated steps wherever possible, minimising the need for manual intervention, which can both slow the process down and risk introducing an element of human error – as well as driving the cost up of course. This guide outlines how this approach delivers both economic and qualitative benefits to your migration project.

THE METHODS – @ LMH

ROBOTICS

Assuming that your archive is just that – an exhaustive collection of all content output over a number of years – it makes little sense to contemplate using manual processes to digitise your assets if you can possibly avoid them. Loading hundreds of tapes, many of which may not be accurately catalogued, one at a time by hand into VTR decks and manually recording details of their content would be a daunting task and is not the way to maximise productivity or cost-efficiency. We think exactly the same here at LMH – it’s inefficient and not the way for us to offer attractive costs to our customers.

Using robotic tape machines to take as much of the strain as possible is the solution. The most common is the Sony Flexicart, which holds 40 small or 20 large tapes with a two-VTR configuration. There have been others: for example, SAMMA, once sold by Oracle, used a four-sided rotating tape caddy and offered some pre- and post-ingest analysis on video levels and tape RF.

There was even a free-roaming robot derived from the automotive industry which was sold by a Swedish company. The name escapes me, but it probably had more personality than your average tape operator!

These processes require an orchestration system to control everything in the chain including the robot, the VTR machine in the robot and the digital encoder that receives the base band video signal and ultimately makes the resultant file. This is something we will examine a little later. The point to press home is that up to 20 hours out of 24 can be utilised for ingest using robotics, which delivers economies of time and budget.

Of course, there will be plenty of libraries that partially or entirely consist of tapes that can only be migrated manually. Indeed, I would estimate that as much as 50% of the libraries we come across require some form of manual processing.

Open-reel formats such as 2” quadruplex and 1” A/B/C/IVC , Panasonic M11 and pioneering cassette formats such as Betamax , Philips V2000, Sony ½ Helical EIAJ are all handled manually. This even extends into the “modern era” of tape design, with D1/2/3/5/9 formats having to be transferred by hand.

Even with these formats, all is not lost. Here at LMH we have built dedicated manual migration systems to handle precisely this challenge. We use multiple decks to deliver the scale required, dipping into our sizeable reserves of decks to keep big projects rolling. We also rely heavily on monitoring, using traditional waveform monitors and vectorscopes to tease out the best possible image from a tape. The care taken at the ingest stage means that the resultant digital file can then benefit from the same automation techniques as formats we can process automatically, such as assisted QC and workflow tracking and monitoring. This will result in a better-quality end result, as well as helping keep costs under control.

We could even offer robotics for U-Matic High and Low band as well as VHS if the volumes were significant enough.

REPAIR AND RECONDITIONING

Physical repair of cassette shells is often needed, which can sometimes only be done if a store of blank cassettes of the same format exists to donate parts or even allow a complete re-shell of the V0 cassette.

As is more widely known, baking tape for three, four or five days at the very consistent temperatures that can be achieved using laboratory ovens does often resolve the stickiness that forms with tape substrates, allowing a window of opportunity to replay the tape successfully and without damage to your machine or asset. The tape will revert to its former state after a week or so but by that point you will have the electronic file asset.

Other issues that can affect transfer are leader repair, and mould. The former is quite straightforward: the tape becomes detached from the cassette spool and needs to be re-spliced. The latter however is much more of an issue. Severe mould, which is caused by excessive moisture when storing the tapes, requires full health and safety processes to be adhered to, such as face protection for handling and cleaning. Some mould outbreaks are so severe that they have required experimental treatments to be tested, such as the use of .

Even for assets that haven’t suffered such drastic damage, cleaning the tape prior to ingest is generally a good idea. Our Indelt TC-Matic Betacam tape cleaner is one of our key tools for carrying out this process. Not only does the TC-Matic rigorously clean and re-tension the tape, it also utilises reflective and pass-through sensors to detect holes or damage to the tape, recording this data to an xml file. This process allows us to remove as many flaws as possible prior to starting the ingest process, which of course in turn means we can produce the best possible digitised file from your asset.

ORCHESTRATION: CONTROLLING DEVICES AND BUILDING DYNAMIC WORKFLOWS

At LMH we use MediaFlex by Transmedia Dynamics (TMD) to control many of our ingest devices, and to help us build dynamic, customer-specific workflows. This provides a robust platform for modelling and executing media-specific processes.

The system controls all of our tape ingest estate, the Flexicart, the VTRs and the encoders. Mediaflex will also perform indexing of all file types used in a typical packaging workflow such as video, audio, subtitles, closed captions, graphics, and documents.

As well as our own in-house systems designed around Google Data Studio, Mediaflex delivers business analytics by enabling our production management team to identify trends, issues, and operational metrics on all aspects of our business workstreams.

We can build workflows using these and other orchestration toolsets. These can provide mass media sifting, processing and delivery workflows using API control of devices such as Vidchecker, Telestream Vantage and Aspera or Signiant.

Using the shallow dive capability of assisted QC tools you build a sifting system to identify the technical aspects of an undefined set of incoming assets and thereby traffic-control that data into onward process such as transcoding and delivery to different platforms at the specific codec required.

ORCHESTRATION: SERVOLOCK

SERVOLOCK is one of a number of additional upgrades introduced at LMH as part of our collaboration with TMD.

It addresses a laborious and potentially costly task that needs addressing when digitising a library: namely deciding when to start and stop recording. This may sound basic but researching each tape out of potentially thousands to find the correct ‘In’ and ‘Out’ points (often referred to as SOM and EOM – start of media and end of media) can result in a lot of human labour.

The orchestration system can be enhanced to detect dedicated control signals that emanate from the videotape machine. These signals are unique to the presence of coherent content. Once these are detected for a fixed period the associated system can be told to start recording, and the opposite can happen at the end of the recording when these signals disappear. This means the entire content can be captured without any human intervention, which is a cost saving up front.

This fail-safe, automated method is popular with archivists as the preservation of everything on the assets is often a must.

There is a need to balance data storage costs as the resultant file produced by this automated process can be larger than one captured by human intervention. Manually entering the SOM/EOM therefore remains a valid option but is more costly up front.

ISR: INTERACTIVE STATUS REPORTING

ISR is a system present on Sony VTRs to track and categorise channel condition errors. From a human perspective it represents a traffic light system of Red, Amber, Green.

These errors are the result of the DCT (discrete cosine transform) codec breaking down due to a loss of RF – the signal read from the magnetic tape material. Sony used this SNMP protocol to wrap the reporting into a machine-readable signal. Via our orchestration system LMH has harnessed this connection on each VTR in each of our Flexicart robots to provide automated data on the quality and condition of each tape.

MONITORING

One of the biggest headaches in a large library migration exercise – by which we mean anything from a few hundred tapes up to and in excess of 50,000! – is keeping track. We found developing bespoke monitoring systems was the only way to tackle this problem, as there was no off-the-shelf solution that made the grade for the complexity of our workflows.

PUSHED NOTIFICATIONS

Pushed notifications enable us to keep our clients informed about how their migration project is progressing. These can be weekly or monthly scheduled email reports that are pushed automatically to an email address of your choice. These are fully customisable but will typically contain format breakdown and project status summary throughput information.

Library migration: Weekly status notifications

WORKFLOW STEP TRACKING

Here we can track each step of the workflow from the point the tape is turned into a digital file all the way through to the upload to the client and beyond into Exception.

Exception is a term we use to describe an asset that has not met the basic level of quality within a particular project. We tend to work with the client to define where this quality threshold sits before we start. We will then digitise every asset in turn whilst judging them against this threshold. Once we have selected each tape for ingest once, we will revert back to the Exceptions and attempt remedial action.

Library Migration: Workflow Step Tracking

MEDIA METADATA INTELLIGENCE

Here we can assess the average run times of each collection, as well as the overall run time. This data can be useful for billing and feedback to the library owner so they can update their metadata database.

Library Migration: Average Runtime Report

We can also monitor the make-up of a collection in terms of its video tape format.

Library Migration: Format Report