Audiovisual archives & digital preservation

Bertram Lyons is Senior Consultant for AVP

I want to tell a story to demonstrate the inherent relationship between audiovisual preservation and digital preservation.

1. What is it about audiovisual preservation today that requires us to engage in digital preservation?

My story begins with a collection of folk music analog audio recordings in the Alan Lomax Archive.

The year is 2002. The recordings -- I’ll focus on the 2,000 quarter-inch magnetic reel to reel audio tapes -- date from 1948 to 1997. They vary in composition from paper, to acetate, to polyester. They display a variety of states of physical condition, from excellent, to disheveled, to disintegrating, to sticky.

Audiovisual preservation practice of the time included (aside from physical conservation and preventive care) a practice of creating “preservation copies” on larger reels at faster tape speeds (e.g., if my original tape was 7.5 IPS, I might create a long-term preservation copy at 15 IPS or 30 IPS).

This did create a “high-resolution” master (similar to how we create 96khz/24bit BWF audio files today when creating new masters), but left us in the same physical predicament that we were in when we started: decomposing magnetic audio tape.

In 2002 we were not yet facing the drastic reduction in support for the open-reel magnetic audio tape format. We were just beginning to see it. But we were definitely aware of technological obsolescence and it was apparent that at some point in the future, magnetic audio tape (even if the tape survived) would not be supported with modern technology.

By 2002, the popularity of digital audio as distributed on CDs made it clear that digital audio recording was a new possibility for storage of audio signals.

The Alan Lomax Archive, at that time, coached by our colleagues at the Library of Congress, made the decision to begin creating digital preservation masters of analog audio tape (still holding on to the legacy tape-to-tape practices for the initial period).

Some early state-of-the-art audio digitization in 2002 consisted of creating 24bit-48khz digital signals (based on limitations of some analog-to-digital hardware), but by 2004 the community (and the technology) agreed on a 24bit-96khz standard for sampling originally-analog audio signals and encoding them in Pulse-Code Modulation (PCM) values.

So here we were, tiptoeing into digital preservation of audiovisual archives — taking our original analog tapes, reproducing them with Ampex tape decks, passing them through Prism Analog-to-Digital Converters (ADCs), re-encoding the analog signal into a PCM numeric signal, taking 24-bit sized samples of the analog signal ninety-six thousand times every second.

We were no longer creating preservation copies onto the same recording medium that was becoming obsolete (that was a positive outcome), but we found that we had moved into an entirely new domain. The digital domain. We now had to learn about the significant characteristics of digital information:

• How is audio information stored as numbers?
• What are WAV files and how do they package the new audio information?
• What is required to replay the new digital audio information today, and tomorrow?
• What are the vulnerabilities of digital information?
• What do we need to do to counteract these vulnerabilities?
• If we can no longer write descriptive and technical information on the box of the tape, where and how do we create and store this information for reuse?
• What systems are needed to provide access to this new digital information?

In 2002, at the Alan Lomax Archive, due to the extremely high cost of external hard drive storage at the time, affordable digital storage for multi-TB collections for smaller organizations was limited to MAM-A Gold CDs where we were able to store about 700MBs of 24bit-96khz digital audio on a single optical disc. This created its own new issues. Recordings longer than 40 minutes usually required two optical discs and had to be split into two separate audio files. Also, a new concern about disc failure required always the creation of 2 copies of each CD. Digital preservation, for the Alan Lomax Archive, started as a hybrid practice of storing digital information on optical discs and managing multiple copies on the same shelves where the analog tapes were being stored. We were doing “digital preservation” work but still in the same physical storage environment with the analog originals.

By 2004, external digital storage finally became a commodity technology that was affordable at scale and “digital preservation” became a new skill set required for our audiovisual archive. We finally moved from managing many individual physical things (i.e., optical discs, which themselves were becoming obsolete already!) to managing a few physical things (i.e., hard drives) with many individual digital files stored within. Digital information not only changed the target of audiovisual preservation, it now was changing the entire management process. With the combination of a change in signal storage and the separation of the “signal” from the “storage medium”, digital information significantly altered the practice of audiovisual preservation. Sustained audiovisual preservation now required digital preservation for its ultimate success.

Since 2004, which also noted the release of the first guideline for the digitization and digital preservation of sound recordings (IASA-TC 04), over the past 15 years, we as an international audiovisual archives community have learned a few core concepts with regard to ongoing audiovisual preservation in a digital era (a more thorough review of these concepts can be found in IASA-TC 03).

In this early era, we learned:

• The lifespan of any audiovisual carrier is going to be affected ultimately by two key features:
  • Physical composition and decomposition timeline
  • Technological dependence for replay and timeline of engineering support 

• Audiovisual information inherently requires technology for recording and replay
  • Technology is largely driven by the consumer market, which ties audiovisual technologies to a changing landscape driven by the consumer market
  • Disruptive audiovisual technologies will affect our ability to preserve the previous technologies they disrupt 

• Current audiovisual information is born-digital 

• To bring legacy audiovisual information with us into the future, we need first to bring it into the digital domain 

• There will be technologies in the future that disrupt digital technologies at which point our audiovisual paradigm will change, and we must be ready for that. 

This brings us to part two of my story.

2. What is it about digital preservation today that is critical for audiovisual archives?

As the Alan Lomax Archive story continues, we now found ourselves as both audiovisual archivists and digital archivists simultaneously. We needed strategies from the domain of digital preservation in order to succeed with audiovisual preservation.

What are those strategies?

Two large areas of digital preservation became immediately useful:

• Bit preservation
• Representation information

With bit preservation, we needed to learn practices:

• to create inventories from computer file systems;

• to create checksums against digital files to help identify whether they have undergone any change during storage and/or movement;

• to ensure recurring backup copies are created daily/weekly/monthly to provide roll-back opportunities when problems occur;

• to ensure redundant copies are created and stored in different geographic locations;

• to refresh storage media over time (a process we were already familiar with in the analog domain - to combat physical degradation and technical obsolescence); 

• to provide security for software and hardware that are our liaisons to the bits themselves;

• to document disaster plans to respond to problems and reduce risk;

• and to document clear policies and responsibilities for funding, managing, and monitoring the above practices.

These skills above gave us the foundational skills as audiovisual archives to ensure the new audio and video bits we were creating would survive into the future. It is the equivalent of our previous skills caring for the physical audiovisual carriers on shelves in storage rooms and ensuring proper climate, proper identification, proper housings, locks on doors, fire prevention and response, non-UV lighting, and other strategies that ensure the long-term survival of physical objects. As audiovisual archivists moved into the digital domain, we needed these same types of strategies for our new digital content.

With representation information, we needed to learn practices:

• to understand how numbers are encoded to represent audio and video signals;

• to understand how file formats are defined to store and package numeric audio and video signals;

• to document the formats and encodings that we use so that future generations can replay the audio and video signals;

• to ensure the existence of software and hardware necessary to decode and replay the audio and video signals;

• to recognize when risk to decodability is present for encodings and/or formats;

• to plan transformation of the encodings to new encodings when risk has been identified;

• and to document our decisions so that future generations will understand what we have done.

The skills above gave us the foundational skills as audiovisual archivists to ensure the new audio and video bits we were creating would remain usable and understandable today and in the future. It was the equivalent of our previous knowledge that allowed us to understand how analog signals encoded on physical carriers were played-back appropriately and which machines were required to do so. As well as, how to recognize when it was time to move to a new carrier format, and how to ensure clear documentation of the requirements for each format’s playback specifications.

These requirements have led to a variety of trends over the past 15 years:

• Audiovisual archivists are developing stronger tech skills
  • File management
  • Byte-level analysis of formats (forensics) and file systems
  • Checksums
  • Technical characteristics of all format types
  • Common Computer OS and command line comfort
  • Coding and logic
    • OS scripts to increase productivity in file and data management (bat, sh)
    • Coding languages for more complex programs - Java, Python, Ruby, Perl, etc.
  • Open-source and community managed and shared software projects
    • Islandora, Archivematica, Fixity, MDQC, Exactly, Bagger, etc.
  • Micro-services and system integration
    • No longer a search for the magic bullet - for the one-system-to-rule-them-all
    • The work of digital preservation and collection management is broken into manageable parts and modular tools/services developed for each part and connected to other tools/services -- plug and play
  • Continued focus on reducing vendor lock-in and reducing complexity
• Digital storage is an ever pressing concern for audiovisual archivists
  • Onsite and cloud storage - organizations are leveraging a balance of onsite storage and offsite internet storage options to handle constant growth and scale
• Holistic digital preservation planning is a driving trend for developing sustainable practice at an audiovisual archive
  • Data Seal of Approval
  • Nestor
  • ISO 16363 - Trustworthy Digital Repositories
  • These all focus on an approach that looks at organization structure and planning, digital object management, and technology and security infrastructure.

These trends have been underway within the international audiovisual community for more than 15 years now and there has been much advancement.

The International Association of Sound and Audiovisual Archives (IASA) has published a suite of freely available guidelines to establish a foundation for audiovisual archivists:

• IASA-TC 03 (2017) The Safeguarding of the Audiovisual Heritage: Ethics, Principles and Preservation Strategy
• IASA-TC 04 (2009) Guidelines on the Production and Preservation of Digital Audio Objects
• IASA-TC 05 (2014) Handling and Storage of Audio and Video Carriers
• IASA-TC 06 (2018) Guidelines for the Preservation of Video Recordings

The sister organizations that constitute the Coordinating Council of Audiovisual Archives Associations (CCAAA) all work singularly and together to share knowledge and good practice for all areas associated with digital audiovisual preservation.

Outside of the audiovisual community, the digital preservation community provides a network for resources related to the strategies of digital preservation. The International Conference on Digital Preservation (iPres) occurs every year at a new location around globe. 15 years old, this conference is the international home for digital preservation conferences.

There is also the Preservation and Archiving Special Interest Group (PASIG) which happens to be gathering in Mexico City for their spring 2019 meeting in February.

In the US, we also have a yearly Digital Preservation conference put on by the National Digital Stewardship Alliance, among a few others.

Today it is clear that audiovisual archives require a combined knowledge of the domains of audiovisual content and digital preservation. Professional organizations and communities have worked together for the past 15 years (or more) to build networks and share good practices. As legacy audiovisual carriers become obsolete and the machines needed to access their content disappear, the pressure is mounting to ensure that all archives are migrating physical audiovisual carriers to digital file-based formats. The challenge for us all is time and money. Working together, in our audiovisual and digital preservation communities, we can do our best to bring the audiovisual legacy of the past into the digital present, and to ensure all digital audiovisual content we have today is carried with us and understandable to our future selves.