Blog

Ross Harvey is Professor at Monash University in Melbourne, Australia.

Preparing a new edition of Preserving Digital Materials with Jaye Weatherburn (Rowman & Littlefield, 2018) has provided the opportunity to reflect on a large body of material about digital preservation. What I have observed suggests strongly that in the future, 2017 will be considered as a watershed year for digital preservation.

Sometimes the progress of digital preservation seems to resemble a tortoise inching along. Solid foundations are in place, and the need for action is better understood, but these only brush the surface of the many, varied, and evolving challenges. Four challenges seem particularly resistant to change: managing digital preservation, especially its lack of integration into mainstream practice; funding digital preservation – there’s never enough; peopling digital preservation, there being a lack of skilled people; and making digital preservation fit—lack of scalability of digital preservation activities.

Richard Wright runs Preservation Guide, a consultancy on audiovisual preservation. Before retiring, Richard worked as a Technology Manager at BBC Information & Archives from 1994. 

Not everyone knows that there even are television archives. Europe is fortunate in having a tradition of public service broadcasters. They are publicly supported in various ways (licence fee, limited advertising, direct government funding) but all have a remit to provide high-quality information and entertainment. Broadcasting can be seen as ephemera, but yesterday's ephemera becomes today's heritage. Of particular interest in a time of fake and false news is the role of public service broadcasters in providing quality factual material: news and current affairs.

Public service broadcasters, particularly in Europe, have also led the way in maintaining archives of their productions. While drama and entertainment programmes are kept for repeats and for sale to other countries, factual content is heavily recycled to add depth and interest to current programmes. In the BBC, about 30 to 40 percent of 'the news' is actually archive material. Other uses include biographies; retrospectives on people, places and political situations; cultural history; and a wide range of factual content that needs archive footage for context and historical memory. Up to 2010, about 20% of the BBC television archive was accessed each year, and 95% of that use was internal: back into the BBC for adding depth to new programmes. The other 5% was commercial use. Broadcast archives had little or no public access. In the UK, public access to BBC Archives was via copies of tapes sent from the BBC to the British Film Institute.

Michelle Lindlar is Digital Preservation Team Leader at Technische Informationsbibliothek (TIB) in Germany.

Dear past self,

You left your IT job a couple of months ago to work in digital preservation. It seemed like a really exciting and good fit given your background and interests. I think you’re still trying to work out the culture shock of working in a public service environment and trying to figure out what digipres is ‘zactly. I thought a few pointers and words of encouragement from future-you-who-has-been-in-the-job-for-a-while would be good.

National and State Libraries Australasia (NSLA) is a leading library sector collaboration of the National, State and Territory libraries of Australia and New Zealand. 

How Australasian libraries are working together on digital preservation

How often do you leave a conference, or a meeting, with the best intentions of “collaborating more”, only to see those intentions evaporate quicker than rain on bitumen on a hot summer’s day, as soon as you return to your desk and look at your email inbox? We don’t make these statements flippantly; most of us share a sincere commitment to work together, recognising that for us to move forward and progress our work, we need help from others. Sadly though, what happens next is that reality takes over. Business-as-usual kills the best intentions.

Despite this, the national libraries of Australia and New Zealand and state and territory libraries of Australia have found a model of collaboration that has proven to be extremely rewarding. National and State Libraries Australasia (NSLA) brings these ten libraries together under a shared vision of connecting library professionals to information, and to each other. The libraries have many differences – they range significantly in size; they have differing mandates, and differing priorities – and yet, NSLA is a success story.

One of the significant NSLA success stories is the Digital Preservation working group. Formed in 2012, the group brings together representatives from each of the ten member libraries to identify best practice for preserving digital content – practices that are best served by a collaborative approach.  

Panos Constantopoulos is Professor in the Department of Informatics at Athens University of Economics and Business & Digital Curation Unit in Greece

An answer to the question “what should be preserved?” would typically include digital content of various forms and related metadata.  But what stands for content and where is the limit between content and metadata? The changing notion of ‘document’ and the evolution and spread of computationally supported business and research processes, entail a greater need for keeping process and contextual knowledge.

Digital content traditionally includes material that is the product or record of some activity and comes in several forms - text, graphics, images, audio, video, data sets. Recent years have witnessed a shift in content granularity and structure. This is related with the ability to manage both the identification of entities that possess independent information value and the associations of those entities. Reference to self-contained parts of documents is an old practice (chapters, sections, tables, images, columns, etc.). Yet it is in the universe of the Web and linked data that this practice becomes fully operational and on a large scale: here the parts are independently identified through their URIs following some naming scheme and the conceptual bond that makes them parts of a whole is explicitly represented in the form of appropriate relations between them. The parts can then be reused in different ways. In this perspective, it is useful to maintain the knowledge of the successive uses as well as of the contexts of use.

Glasgow, 30^th November 2017

Dear colleagues and friends around the world,

Welcome one, welcome all! The first International Digital Preservation Day (#IDPD17) is finally here!

This day is for everyone who works in digital preservation. It’s about their work.  It’s about the opportunities created by the digital materials they safeguard and make accessible. It’s about the hard work and ingenuity, often unrecognised, that makes a secure digital legacy possible. And it’s about fostering links across this growing but highly dispersed community. Supported by digital preservation networks around the world – old friends and new - IDPD17 is open to participation from anyone and everyone interested in securing our digital legacy.

"We build our computer (systems) the way we build our cities: over time, without a plan, on top of ruins." - Ellen Ullman

John Geddy’s map of St Andrews, c.1580. National Library of Scotland (MS.20996) http://maps.nls.uk/view/00001427

Neil Jefferies is Head of Innovation at Bodleian Libraries, University of Oxford

I would like to query the rationale for actually doing periodic fixity checking in isolation. This has bugged me for a bit so I am going to unload.

As far as I can see, the main reasons would be undetected corruption on storage and tampering that doesn’t hijack the chain of custody.

All storage media now have built-in error detection and correction using Reed-Solomon, Hamming or something similar which is generally capable of dealing with small multi-bit errors. In modern environments, this gives unrecoverable read error rates of at worst around 1 in 10^14 and generally several orders of magnitude better – which is around one in 12TB total read. Write errors are less frequent – they do occur but can be detected by device firmware and retried elsewhere on the medium. These are absolute worst case figures and result in *detectable* failure long before we even get to computing fixity. The chance of bit flips occurring in such a pattern as to defeat error correction coding is several orders of magnitude less – it is similar to bit flips resulting in an unchanged MD5 hash. Interestingly, in most cases the mere act of reading data allows devices to detect and correct future errors as the storage medium becomes marginal so there is value in doing that.

Read Jaye Weatherburn's account of sessions at iPRES 2017. Jaye, who works at the University of Melbourne, attended iPRES with support from the DPC's Leadership Programme which is generously funded by our Commericial Supporters.

At iPres 2017 (in September | Kyoto, Japan) I attended the Digital Preservation Storage Workshop: Exploring Preservation Storage Criteria and Distributed Digital Preservation. 

I was particularly keen to attend this workshop, as in my role at the University of Melbourne I am actively working with the research support community to develop better understanding of what digital preservation storage means and its requirements, as part of our Digital Preservation Project. We have used the most recent version (version 2) of the preservation storage criteria to run a workshop with our archivists, records managers, and IT staff, and have found this list of 58 criteria extremely useful both for increasing knowledge and understanding about preservation storage needs, and for generating discussion about what is required for preservation storage for different digital collections at the university.

The preservation storage criteria were originally developed by Kate Zwaard, Gail Truman, Sibyl Schaefer, Jane Mandelbaum, Nancy McGovern, Steve Knight, and Andrea Goethals, and have been further developed through workshops and presentations at various conferences and meetings during 2016-2017 (iPres conferences, PASIG meetings, the Library of Congress Designing Storage Architectures for Digital Collections 2016 meeting).

Part 4: Costs and tools

The National Library of Scotland, Edinburgh Parallel Computing Centre, National Galleries of Scotland and the Digital Preservation Coalition are working together on a project called Cloudy Culture to explore the potential of cloud services to help preserve digital culture. This is one of a number of pilots under the larger EUDAT project, funded through Horizon2020.

We’ve already published an introduction to Cloudy Culture and reports on uploading and file fixity and downloading. This final report covers the use of preservation tools in the cloud using MediaInfo as an example, and the costs of using the cloud. The costs are based on a cloud service hosted by the Edinburgh Parallel Computing Centre (EPCC) using iRODS data management software (https://irods.org). The research questions we ask are:

1. Can we use arbitrary preservation tools e.g. MediaInfo, in the cloud, even when the cloud uses one operating system and the tools run on another?
2. How quickly did the tool run?
3. How do EPCC cloud service costs compare to local storage and Amazon Cloud costs?