More and more diverse digital material is pouring into the archives of heritage institutions, often also in large amounts. This makes automatic processing necessary. Which preservation tools are used by the digital heritage institutions? Ania Molenda, project leader of the Preservation Tools project at the Dutch Digital Heritage Network, asks this and other questions to the heritage field. She incorporated the results of her research in a report The Use of Preservation Tools among Dutch Heritage Organizations.
In 2018, Joost van der Nat examined whether Dutch heritage institutions had a digital archive or an e-depot available at their organization. Ania Molenda approached the twenty-seven parties who answered yes to this question for a follow-up study in 2019. Twenty-two institutions, ranging from a library and archive to a museum, completed a questionnaire about the use of preservation tools. The survey focused on the tools that the institutions use when processing digital files before they are included in an archive or a digital repository. This part of the digital preservation process is known as pre-ingest and ingest.
(Pre-)ingest
Ideally, a heritage institution can make agreements with an archive creator to supply digital material according to their guidelines, says Ania Molenda. "University repositories, for example, ask researchers to supply their research data according to certain requirements. The advantage of this is that the material can immediately enter the so-called ingest phase. In other words: the material is ready for inclusion in the archive. Some control of course still takes place, but only where it’s absolutely necessary. Preservation tools help to check, for example, whether the supplied material contains damaged files."
The practice, however, is unruly. Two-thirds of the heritage institutions that participated in the survey indicated that they are unable to make such agreements for the supply of archival material. "Private archive creators, for example, do not always have the possibility to prepare the material according to the guidelines." In that case, a heritage institution must prepare the material before it can be included in the archive in the so-called pre-ingest phase. "With preservation tools heritage organizations can check, for example, whether the supplied material contains hidden, protected or difficult to identify files. The tools also help with the organization and description of the material so that later the files can be easily found in the archive."
Digital sustainability
Why is the preparation of the archival material so important? Ania: "It has to do with the sustainability of long-term preservation. In order to ensure that, collecting essential information about the files is key. Take file formats for example. Some file formats are more sustainable than others. As a heritage institution, you want to accept those file formats that you know you can keep for a long time."
111 preservation tools
Currently the researched institutions use 111 different preservation tools. On average, an institution uses between one and ten tools. "They are used for both pre-ingest and ingest," says Ania. It is striking, however, that during acquisition conversations between archive institutions and creators, the preservation tools are hardly used to assess whether the material meets the requirements. Especially at this stage - together with the archive creators - institutions could already analyse the archival material with preservation tools.
Furthermore, the survey respondents appear to rarely use preservation tools to deduplicate, identify dependencies between files and to identify encryption or password protected files. Ania: "Many institutions would like to use encryption detection tools, because no one wants to include encrypted files in their archives. However, it is not yet clear which tools are best suited for this."
New file formats
Ania also asked the institutions what challenges they face. "The collections are changing rapidly because institutions are increasingly collecting new types of digital objects such as social media, websites and e-mail, for example. These archival objects also bring new file formats to the collections. Social media, for instance, require new tools to both to collect and process them for long-term preservation. In short, the number of preservation tools used is expected to increase in the coming years."
Learn from each other
Ania's research shows that institutions often use different terms to describe similar things. "While one institution speaks of an "archive" the other calls it a "collection". One organization calls a process "pre-ingest", the other "ingest". That is because their processes are organized differently and because they have other legal obligations, for example."
Although many institutions now face very different challenges in setting up pre-ingest and ingest for their digital collections, they can often use the same tools in their preservation processes, according to Ania. "If the institutions are more aware of these differences, they can start discussing and learn from each other's experiences. Institutions that are starting to set up their pre-ingest and ingest processes can learn a lot from fellow institutions. They don't have to reinvent the wheel."
Follow-up
The Preservation Tools project continues. "Now that we know which preservation tools are used by Dutch heritage organizations and which are missing, we can see where the Dutch Digital Heritage Network can provide support to the heritage organizations. For example, by helping with further development of open source preservation tools that are interesting for institutions but lack funding. Or through knowledge development and sharing, such as describing best practices. In this way, we will soon be able to better support institutions that start with the automatic processing of digital collections."
More information
Read the research report "The Use of Preservation Tools among Dutch Heritage Organizations". The details of the survey can be found in the Appendices. You can also contact project leader Ania Molenda via email mail@aniamolenda.com.