Handbook

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

Introduction

There are compelling reasons and, in some cases, political pressure, to engage in greater collaboration within and between organisations in order effectively to confront and overcome the challenges of digital preservation. The range of skills required to do this demands flexibility within organisational structures to facilitate working in multi-disciplinary teams. There is a significant overlap in the digital preservation issues being faced by all organisations and across all sectors so it makes sense to pool expertise and experience. Communication with key stakeholders, using terms and language understood by them (see Advocacy) will play a major part in building and maintaining collaborations.

Internal collaboration

The usual assumption is that collaboration is external. However, most libraries and archives will be managing a combination of paper-based and digital resources for the foreseeable future and will need to structure their organisation to manage the disparate needs of the two. The blurring of boundaries and the lifecycle changes which digital technology produces means that sections and departments which are structurally distinct, will now need to co-operate in order to integrate the preservation and management of digital materials with other materials.

Such co-operation and joined-up working may well prove impossible unless there are mechanisms put in place to facilitate it and there is clear executive buy-in and sponsorship to promote action. At the strategic level, a cross disciplinary committee or project team charged with developing and overseeing objectives is one means of ensuring that all relevant sections can be brought together. At the operational level, consideration will need to be given to defining what specific tasks are required and where those responsibilities logically lie. Setting up of working groups to investigate specific issues is one means of blending the range of skills required. Good communication and advocacy to other stakeholders will also be important (see Advocacy).

Whilst the need for a policy or strategy relating to digital preservation may be well established within the repository team, the driver for internal collaboration is typically in response to a specific challenge faced by an organisation.

Advantages

• Makes good use of available skills and expertise and makes the case that digital preservation is an institutional issue and not one owned exclusively by the repository.
• Promotes cross-team working by improving understanding of shared objectives and who needs to contribute.
• The sooner digital preservation becomes part of the daily work of an organisation and its employees, the better it is for their transition to and readiness for a more digital world.
• Recognises the diversity of skills required for the digital environment in general and digital preservation in particular.
• Is more likely to be focused and aligned with institutional objectives and priorities.
• Maintains a high profile for the work.

Disadvantages

• May be frustrating and time consuming in the short term.
• Communication may be difficult initially - for example there are some issues surrounding terminology with the term 'archives' meaning different things to archivists and IT colleagues
• Senior management may be unwilling to risk perceived lack of control.
• Staff may feel uncomfortable with new ways of working.
• Organisational structures may not be sufficiently flexible to facilitate effective collaboration between different sections.

External collaboration

There may be a number of drivers for external collaboration. There is the simple desire for lone specialists to work with other professional colleagues and seek external validation of their ideas or direction of travel. At the other end of the scale is the response to external funding opportunities, with funders now placing greater emphasis on collaboration. Some examples of types of external collaboration in the digital preservation sector are included below:

• Collaboration around a specific problem to make progress easier and more affordable. The Digital Preservation Coalition itself is an example of this in the UK. Members are encouraged to engage and collaborate on a number of different digital preservation related issues both at a high level and on specific topics. Another example is the Section for Archives and Technology of the Archives and Records Association, which brings together members of the professional body to look at specific aspects of the work and to share current practice.
• Collaboration around a standard. An example of this would be the call in 2015 to work together around the revision of the OAIS reference model. In an initiative coordinated by the DPC, practitioners working in the field were encouraged to engage and feed into a shared response. (See http://wiki.dpconline.org/index.php?title=OAIS_Community)
• Collaboration around a specific piece of software or system. An example of this would be the user groups that evolve around digital preservation software solutions, both commercial and open source. When exploring a piece of software for the first time there is huge value in being able to share experiences and learn from others.
• Collaboration within a specific geographical area. There are many examples of organisations collaborating based on their geographical proximity and the ease of working together that this offers. One example of this is the Digital Preservation Group within Archives & Records Council Wales (ARCW) (see Case studies).

Advantages

• Organisational commitment and authority.
• Formal agreements offer a clear allocation of responsibilities between partners.
• Enhanced understanding of complex issues.
• Greater practical benefit from pooled resources and expertise.
• Enhanced reputation through successful delivery of a project or being able to manage digital preservation.
• Improved prospects for future mutually beneficial collaboration.

Disadvantages

• Difficulty of establishing unambiguous agreements able to be accepted by all parties.
• Time taken to establish teams or a collaborative framework.
• Difficulties of communicating across different professional and organisational frameworks.
• Potential bureaucratic barriers.

External collaboration can work on an informal or a formal basis and colleagues across the sector have always shared experiences, with informal collaboration often forming part of an individual's continuing professional development. Larger more complex collaborations are more likely to have a formal partnership agreement that can be useful to define the scope and boundaries of the working relationship and attribute specific responsibilities.

Resources

Benefits from Research Data Management in Universities for Industry and Not-for-Profit Research Partners

http://opus.bath.ac.uk/32509/

Applies a stakeholder mapping using the KRDS Benefits Framework to examine the data management benefits associated with Faculty-Industry and Faculty-Not-for-Profit research collaborations with the University of Bath. It presents a summary list of benefits to different stakeholders that can arise from research data management and data preservation in these collaborations.

Aligning National Approaches to Digital Preservation Conference Proceedings 2012

http://educopia.org/publications/anadp

This publication contains a collection of peer-reviewed essays that were developed by conference panels and attendees. It aims to establish a set of starting points for building a greater alignment across digital preservation initiatives and highlights the need for strategic international collaborations to support the preservation of our collective cultural memory (342 pages).

North West Region Digital Preservation Group

https://nwrdpg.wordpress.com/

The North West Region Digital Preservation Group is an example of an informal geographical collaboration involving local authority, academic and specialist archivists. Outcomes include guidelines for depositors, a workbook for archivists and pilot studies on web archiving and email archives.

Case studies

DPC case note: Welsh journals online: effective leadership for a common goal

http://www.dpconline.org/component/docman/doc_download/450-casenotewelshjournals.pdf

This Jisc-funded case study examines a complex digitisation project at the National Library of Wales, an example of an organisation where there are many stakeholders and many different skills are required. Nominating a single senior member of staff as the lead officer for digital preservation and allowing them to work across different sections of the institution mitigated the risk of uncertainty around responsibility for preservation actions. June 2010 (3 pages).

DPC case note: Freeze Frame preservation partnerships

http://www.dpconline.org/component/docman/doc_download/434-casenotefreezeframe.pdf

This case note examines the relationship between the relatively short lived Freeze Frame project at the Scott Polar Research Institute and the institutional repository which offered to provide long term preservation services to ensure ongoing access at the end of the project. The study shows that small organisations don't necessarily need to establish a sophisticated preservation infrastructure when they embark on digitisation. Partnership can bring unexpected benefits to both parties, but needs to be thoughtfully managed and documented. April 2010 (4 pages).

Community Action via UK LOCKSS Alliance

http://www.slideshare.net/edinadocumentationofficer/ukla-dpc-final

Presentation given by Adam Rusbridge at the Digital Preservation Coalition on Getting Started in Digital Preservation, 28 February 2011. It discusses the role of the UK LOCKSS Alliance and collaboration in e-journal preservation.

Archives & Records Council Wales Digital Preservation Working Group

http://www.nationalarchives.gov.uk/documents/Cloud-Storage-casestudy_Wales_2015.pdf

This National Archives case study discusses the experience of a cross-sectoral working group of Welsh archives cooperating to test a range of systems and service deployments in a proof of concept for cloud archiving. It explains the organisational context, the varied nature of their digital preservation requirements and approaches, and their experience with selecting, deploying and testing digital preservation in the cloud. January 2015 (10 pages).

A collaborative infrastructure for permanent access to digital heritage in The Netherlands

http://www.ncdd.nl/wp-content/uploads/2016/03/Network_Digital_Heritage_Netherlands.pdf

In 2014 the Network Digital Heritage (NDE) was set up in 2014 by a group of national organizations in the Netherlands. The network presented a strategy for the development of a national, cross-domain infrastructure of digital heritage facilities. One of the programmes focusses on digital preservation (Sustainable digital heritage). The aim of this programme is to work on the cross-sector sharing, utilisation, and scaling up of facilities for sustainable preservation and access, while devoting attention to cost management and the division of duties. This programme is carried out by the NCDD, the National Coalition for Digital Preservation. (3 pages).

The SPRUCE project

http://wiki.opf-labs.org/display/SPR/Home

The Sustainable PReservation Using Community Engagement (SPRUCE) project (2011-2013) sought to inspire, guide, support and enable HE, FE and cultural institutions to address digital preservation gaps and to use the knowledge gathered from this activity to articulate a compelling business case for digital preservation. This multi-institutional collaboration brought archivists and technology experts together through mashup events and a hackathon. Two key outputs from the project were the Business Case Toolkit (http://wiki.dpconline.org/index.php?title=Digital_Preservation_Business_Case_Toolkit) and COPTR (Community Owned digital Preservation Tool Registry) (see http://coptr.digipres.org/Main_Page).

Filling the Digital Preservation Gap Case Study

http://digital-archiving.blogspot.co.uk/2015/12/research-data-spring-case-study-for.html

A collaboration between the Universities of Hull and York. The aim of the project was to address a perceived gap in existing research data management infrastructures around the active preservation of the data. Both Hull and York had existing digital repositories and sufficient storage provision but were lacking systems and workflows for addressing the active preservation of data.

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

Introduction

The aim of this section is to help institutions understand, develop and implement digital preservation policies and strategies. These will help an organisation to set digital preservation goals, priorities and mechanisms that will also support the acquisition, life cycle management and dissemination of digital materials.

Policy and strategy are terms that are often used interchangeably or in different hierarchical sequence in different institutions. For consistency, the Handbook defines 'policy' as the highest level document and 'strategy' as the documents and procedures that support the implementation of the policy. In principle the development of policy precedes the development of strategy. In turn strategy may be developed or revised/reviewed on a regular basis, whereas policy may have a longer review cycle. Thus a policy serves the organisational need whilst individual strategies may serve different business units or divisions.

Policy and strategy documents provide a foundation upon which all activities around management of digital materials can be based. Policy and strategy documents that are well formed and consultative provide for high levels of both consensus and compliance in the day-to-day activities of managing digital materials. In turn, this provides for certainty that digital materials are being managed appropriately and to best effect. Policy documents also form the basis for cost planning and for funding applications. Strategy can be used as a flexible means to both adapt to changing situations and to demonstrate that learning that has been applied.

Within any institution there will be a range of stakeholders who have a stake in the life cycle management of digital materials. They may contribute to the management of those materials, they may create or manage metadata associated with those materials, or they may have management responsibility for collections. The end-users are also key stakeholders as their needs determine what is important for preservation. The views of stakeholders and their roles in relation to the management of digital materials must be considered at both the policy and strategy level.

You may find it useful to apply the iterative four-step management method of Plan–Do–Check–Adjust as a model for continuous improvement and effective development, implementation and revision of policies and strategies.

Digital preservation policy as part of the wider organisational context

If you are embarking upon, or thinking of embarking upon the creation of a digital preservation policy for your organisation, then it is necessary to start by investigating the context in which the policy will exist.

It is likely that a broad range of policy documents will already exist across your organisation covering a wide variety of issues such as staffing, information technology, risk assessment, and finance. There mayalso be a number of policies relating to more specific issues of records and collections management that will be relevant to digital preservation activities. It is essential to consider both the content and established style and structure of all relevant policies within the organisation as well as the how digital preservation policy will fit within the wider landscape. No single policy or strategy document can stand alone so to achieve support for and successful implementation of a digital preservation policy it is essential to embed it in the broader policy context.

Important considerations in developing policy and strategy

An important aspect of policy development is consideration of the specific needs of your organisation and its key drivers. Alignment with organisational business drivers ensures that strategy and its implementation are also aligned with business need. Policy and strategy documents should make explicit links with and between relevant and existing policies and strategies and build on existing practice. Collaboration, sharing and consultation with stakeholders are essential processes in the development of policy and strategy.

Digital preservation policies are ideally technology neutral, i.e. not dependent upon any one technology platform or system. However in reality this may be unachievable. In such cases they should be focused on principles, aims and objectives that the requisite technology can support.

In order to develop clear, coherent and robust documentation and processes it is essential to adhere to a set methodology and to establish a plan for review so that the policy and strategy remains relevant and current.

1. Establish purpose. The first step is to establish the main purpose of the digital preservation policy, its scope and key aims. These will keep the process of policy development focused and its content coherent. Thought should be given at this stage to how the document will used, both as a tool for advocacy and to help guide the creation and implementation of strategy.
2. Research. As expressed above, it is essential to understand the organisational context in which the policy will exist. Time should be spent investigating existing policy, understanding the organisation's business drivers and the needs of key stakeholder groups. This phase will also incorporate research into best practice for digital preservation policy and strategies, examining the tools and resources available as well as reading policies and strategies from other organisations. Many resources are available with suggestions of what to include in your digital preservation policy and strategy (see Resources).
3. Identify elements and develop structure. Based on the research carried out in the previous phase, the main topics and issues to be addressed in the policy and strategy should be selected. Developing a clear structure for the documents is essential to ensure the documents are useful in practice and to facilitate easy updates and review. The structure should reflect any standards or existing best practice for policy and strategy documents within the organisation.
4. Develop content. Policy content should be high-level and set broad aims and objectives. It should avoid identifying specifics such as details of particular technology solutions, although it may contain reference to commitments established on an organisational level. Information on practical application of the policy will be defined by relevant strategy documents. Content may also be aspirational in relation to aims and objectives but care must be taken not to set unobtainable goals. Recommendations on how to address specific issues within your policy and strategy are available from numerous sources (see Resources).
5. Stakeholder review. It is essential to gain buy-in from various stakeholder groups to ensure your policy and strategy are both fit for purpose and will have support from across the organisation. Presentation of the draft documents to key stakeholder groups is an important part of the drafting process and any feedback provided should be considered carefully. This can also be a key step in advocacy for digital preservation within your organisation, allowing stakeholders to feel engaged with the process and to understand how digital preservation activities relate to their own work. (see Advocacy)
6. Gain approval. Most organisations will require that new policy documents are officially ratified by your management board. Make sure to be aware of the process the organisation and any requirements that will need to be fulfilled. Once ratified the policy will carry more weight and as a result will be easier to implement as part of ongoing strategy.
7. Regular reviews. Policy and strategy documents should not be static and must be responsive to changes in stakeholder needs, the wider organisational context and updates to best practice. A regular review cycle should be established but may also be triggered by significant changes in any of the areas mentioned above.
8. Implementation. Establish an implementation plan to make policy and strategy a reality in terms of day-to-day operations. Remember they are a mean to an end, not an end in itself.

Resources

Digital Preservation Policies Study

http://www.webarchive.org.uk/wayback/archive/20140615022334/http://www.jisc.ac.uk/media/documents/programmes/preservation/jiscpolicy_p1finalreport.pdf

This JISC funded study published in 2008 created a model framework for a digital preservation policy and accompanying implementation clauses based on examination of existing digital preservation policies. Although focussing on the UK Higher and Further Education sectors, the study draws widely on policy and implementations from other sectors and countries.

An additional output was a series of mappings of digital preservation links to other key institutional strategies in UK universities and colleges with the aim of helping institutions and their staff to develop appropriate digital preservation policies and clauses set in the context of broader institutional strategies. (60 pages).

Digital Preservation Policies: Guidance for archives

http://www.nationalarchives.gov.uk/documents/information-management/digital-preservation-policies-guidance-draft-v4.2.pdf

This guide published by The National Archives in 2011 explains the key characteristics of a digital preservation policy. It discusses why there is a need for a policy and how it supports digital preservation. The primary audience for the guidance is publicly funded archives. (16 pages).

Analysis of Current Digital Preservation Policies: Archives, Libraries and Museums

http://www.digitalpreservation.gov/documents/Analysis%20of%20Current%20Digital%20Preservation%20Policies.pdf?loclr=blogsig

This report published in 2013 by Madeline Sheldon, a Junior Fellow with NDIIPP at the Library of Congress, discusses the current state of digital preservation policy planning within cultural heritage organizations. The collection of new or recently revised digital preservation policies or strategies, published during 2008 and 2013, resulted in a high-level analysis of the contents within those documents. A summary overview of the findings was also made available as a post on The Signal blog. (23 pages).

APARSEN D35.1 Exemplar good governance structures and data policies

http://www.alliancepermanentaccess.org/wp-content/uploads/sites/7/downloads/2014/06/APARSEN-REP-D35_1-01-1_0_incURN.pdf

This report summarises the level of preparedness for interoperable governance and data policies. It concludes with selected recommendations that should be taken into account when drawing up data policies concerning digital preservation. (2014, 43 pages).

SCAPE Catalogue of Preservation Policy Elements

http://wiki.opf-labs.org/display/SP/Catalogue+of+Preservation+Policy+Elements

The European Project SCAPE (2011 - 2014) was tasked with looking at policy and producing a catalogue of policy elements to assist those writing policy. This wiki gives some information on the background to the policy work and then has pages for each element which the SCAPE project suggested that organisations should consider when writing policy, with a focus of planning and watch activities. There is also the final report of this work made publicly available in February 2014 on the SCAPE website.

Published Preservation Policies

http://wiki.opf-labs.org/display/SP/Published+Preservation+Policies

An extensive web directory prepared by the SCAPE project in 2015 listing digital preservation policies that are publicly available online for libraries, archives, data centers, and miscellaneous institutions.

Practical Policy Recommendations

https://www.rd-alliance.org/group/practical-policy-wg/outcomes/practical-policy

Actionable policy elements from the Research Data Alliance including both human and machine readable versions of each example.

Case studies

A Digital Preservation Policy for Parliament

http://www.parliament.uk/documents/upload/digitalpreservationpolicy1.0.pdf

The purpose of this Policy published in 2009 is to state and communicate the principles that guide the UK Parliament's activities to secure the preservation of its digital information resources. Further policy documents, procedures, standards, and guidance will be developed in future to address specific aspects of the Strategy. (17 pages).

Hampshire Archives and Local Studies (HALS) Digital Preservation Policy

http://documents.hants.gov.uk/archives/HRODigitalPreservationPolicy.pdf

To address the risk of losing digital materials, HALS has developed a digital preservation policy and strategy. The policy outlines the Record Office's approach to digital preservation, whilst the aim of the strategy is to describe this approach in more detail, including technical specifications where appropriate.

DPC case note: Cabinet papers - policy as a measure of commitment

http://www.dpconline.org/component/docman/doc_download/449-casenotecabinetpapers.pdf

This case note from The National Archives examines the relationship between policy and practice in digital preservation. Grant giving organisations should request copies of applicant's digital preservation policies when funding data creation, as these are an indication of the organisation's commitment to long-term access. The National Archives has digitised a significant volume of the UK's Cabinet Papers, and have a carefully considered policy framework for the long term management of digital resources. May 2010 (3 pages).

DPC case note: Welsh journals online: effective leadership for a common goal

http://www.dpconline.org/component/docman/doc_download/450-casenotewelshjournals.pdf

This Jisc-funded case study examines a complex digitisation project at the National Library of Wales, an example of an organisation where there are many stakeholders and many different skills are required. Nominating a single senior member of staff as the lead officer for digital preservation and allowing them to work across different sections of the institution mitigated the risk of uncertainty around responsibility for preservation actions. June 2010 (3 pages).

British Library Digital preservation strategy 2013-2016

http://www.bl.uk/aboutus/stratpolprog/collectioncare/digitalpreservation/strategy/dpstrategy.html

The British Library's Strategy includes four priorities. Each priority is accompanied by a series of actions. These priorities are aligned with the Library's overall approach to Collection Care and its five principles of sustainable stewardship: to predict, protect, prioritise, preserve, and enable.

Strategic Priority 1: Ensure our digital repository can store and preserve our collections for the long term
Strategic Priority 2: Manage the risks and challenges associated with digital preservation throughout the digital collection content lifecycle
Strategic Priority 3: Embed digital sustainability as an organisational principle for digital library planning and development
Strategic Priority 4: Benefit from collaboration with other national and international institutions on digital preservation initiatives

Further details of each can be found in the full version of the strategy in a linked pdf document (16 pages).

Wellcome Library's Preservation Policy

http://wellcomelibrary.org/what-we-do/library-strategy-and-policy/preservation-policy/

The purpose of the Wellcome Library's Preservation Policy is to provide a comprehensive statement on the preservation and conservation of the Library's collections. It is intended to cover all material in all formats. The policy contains three parts that cover general statements, the management of physical materials and the management of digital materials.(25 pages).

UK Data Archive Preservation Policy

http://data-archive.ac.uk/media/54776/ukda062-dps-preservationpolicy.pdf

This policy published in 2014 outlines the principles which underpin the main activities of the UK Data Archive (the Archive) the active preservation of digital resources for use and re-use within its core user community. From a preservation point of view this policy generally conforms to the OAIS Reference Model, with additions and alterations which are specific to the materials held within the Archive. The Archive has a series of strict requirements for its digital preservation activities. These requirements are laid down in this policy, and the manner in which these requirements can best be achieved in relation to regulatory requirements, archival best practice, information security and available funds is also detailed below. Consequently, the Archive's preservation policy is based upon open and available file formats, data migration and media refreshment. (16 pages)

Digital Preservation Strategies for a Small Private College

http://files.archivists.org/pubs/CampusCaseStudies/CASE-16-MegMiner-Final.pdf

The POWRR Project (2011 – 2014) investigated, evaluated, and recommended scalable digital preservation solutions for libraries with smaller amounts of data and/or fewer resources. Well established "best practices" in digital preservation (DP) do little to address day-to-day realities in repositories that cannot dedicate funds or staff to DP workflows. Meg Miner, Illinois Weslyan University, discusses what can be done to ensure good stewardship for born digital and digitized institutional records before a complete preservation system is in place. 2015 (13 pages).

University of Edinburgh - Developing a Digital Preservation Policy

http://www.dpconline.org/component/docman/doc_download/1321-making-progress-hsbc-nov-2014-lee

A great presentation and case study by Kirsty Lee at the University of Edinburgh to the DPC Making Progress in Digital Preservation workshop in October 2014 explaining the methodology that she is using to build a digital preservation policy at Edinburgh. (14 pages)

Image: digitalbevaring.dk CC BY 2.5 Denmark

Who is it for?

Both senior administrators (DigCurV Executive Lens) and operational managers (DigCurV Manager Lens) within institutions. Also existing or potential third-party service providers.

Assumed level of knowledge

Intermediate (basic understanding of the issues, some practical experience).

Purpose

• To form the basis for further development of policies and strategies appropriate to individual institutions.
• To provide existing examples of good practice which might serve as models.
• This section outlines a number of strategies which have been used successfully by institutions in developing approaches to digital preservation. Each sub-section discusses the approach, its potential advantages and disadvantages, and then provides exemplars of the approach together with further reading on the topic. Strategies such as these will form a core component of corporate policy development to address digital preservation. Sound policy development combined with effective working practices and procedures (see Organisational activities) has been essential to effective digital preservation programmes.

    Download a PDF of this section.

Save

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

  Download a PDF of this section

Introduction

This section is for you if you have yet to start digital preservation or have just begun to do so. It provides a rapid introduction to a number of approaches that will support you in getting started, moving towards using other sections of the Handbook, and in building confidence and skills.

The section developed out of the "Getting Started in Digital Preservation" workshops run by the Digital Preservation Coalition. It supports ‘learning by doing’ and assumes a minimal level of prior knowledge. As you progress you will benefit from dipping into the resources and case studies, other topics and sections in the Handbook, and the Glossary for any unfamiliar terms.

Digital preservation can seem like a daunting prospect. It can help to map out the skills and resources you have and the materials you want to preserve. That way you start with what you know rather than what you don't. The first step in digital preservation is almost always to undertake a rapid assessment. This will have two or three components:

• knowing the practical capacity of your organisation;
• understanding the organisation's goals and missions; and
• knowing a little about the digital materials in question.

Get to Know Your Organisation and Your Data

Creating a Digital Asset Register

As part of a rapid assessment: it is vital to understand the nature and extent of your digital collections. A digital asset register will be incredibly useful for assessing the extent and significance of the collection, identifying priorities and planning digital preservation actions. A high-level assessment of the collection will help with more detailed mapping later: a comprehensive and detailed audit could be time consuming. So the advice in early stages is to keep the asset register simple. Ask the following questions:

• What is the subject of the collection?
• Where does it come from and what is its function?
• Where is it stored and what kinds of media are used?
• Why is it being retained?
• Who is responsible for it; who are the users; who are the subjects of the data?
• How is the data accessed?
• How is the data likely to change and grow in the near future?

Assessing Your Organisation’s Readiness

Organisational maturity is another factor to consider. The National Digital Stewardship Alliance (NDSA, 2013) in the USA has recommended a simple 4 level model to help organisations understand and improve their technical capacity in digital preservation. The four levels are:

• Level 1 - protect your data
• Level 2 - know your data
• Level 3 - monitor your data
• Level 4 - repair your data

These ‘Levels of Preservation' are intended to be progressive, and are used to measure maturity against five components: storage, file fixity, information security, metadata, and file formats. An organisation's capacity to undertake digital preservation is indicated by its maturity level across these five components. More comprehensive maturity models are available, such as the Digital Preservation Capability Maturity Model (Dollar and Ashley, 2014), if a more well-rounded exploration of organisational maturity is required.

First Steps to Securing Your Data

This section provides an overview of initial actions to secure your data once you have assessed your organisation’s readiness and compiled basic information about your data. The following steps are essential to ensuring a minimum level of preservation when a new collection of digital material is received. This is typically referred to as bit preservation. Quite literally, preserving the streams of binary digits, or bits, that make up your digital files (without preserving the means to decode the bits into meaningful information).

Prompt check in on receipt

When a new collection of digital material is received from a supplier it is essential to ensure that what has been received is what is expected. Depending on the source of the material, it may be possible to request new copies of missing or poor quality files. These checks are made and any replacement requests submitted, the greater the likelihood of successful resolution.

Key tasks include:

• Scan for viruses and malware to make sure there are no unwanted surprises in the collection. Perhaps keep the collection 'in quarantine' until you have checked it.
• Check all expected files are present. If the material is accompanied by a manifest, check the files against it
• Open a random selection of files to verify their integrity and/or expected quality levels
• Promptly request replacements for any damaged or missing files, where possible

Create a Verifiable File List

In order to check over time that your digital files are being preserved, it is first necessary to record exactly what files are in your possession. It is, therefore, important to create a verifiable list of files in each collection. These lists should likely contain information such as file names, locations and sizes, format types and checksums. A checksum is a short alphanumeric string that represents the contents of a file which acts as a ‘digital fingerprint’ allowing comparison over time. Once the list has been created, it is a simple process to verify that all files are present and undamaged, at any point in the future (see fixity checking below).

Various software tools can be used to automatically generate this data; these are commonly referred to as characterisation tools. For example, you may wish to use The National Archives PRONOM (a register of file formats and their behaviours) and DROID (a tool that uses PRONOM to analyse the files on a system). Having a list of the file formats, versions and quantities in your collections will help you make a case to senior management for the support and resources that will be needed to do the job properly and sustainably. This information can also be used to update and enrich your digital asset register. The range of formats in use should be consolidated to minimize duplication and eliminate problem formats. This process is known as normalization.

Key tasks:

• Generate a verifiable file list
• Update digital asset register

Stabilise your files: make copies

No matter how good your digital storage, your digital material will always be at risk of damage, decay or accidental deletion. Making more than one copy of your digital materials and utilising more than one type of storage solution mitigates a variety of digital preservation risks.

Key tasks include:

• Keep (at least) one copy easily accessible on non-removable disk. You will need to regularly revisit your material to ensure its fixity, so keeping it accessible will make this easier
• Make (at least) one additional copy, if necessary on a less accessible, but cheaper storage medium such as tape
• Keep one copy in a different geographical location to the others to mitigate against disaster

Revisit and inspect: Fixity checking

By revisiting your digital materials on a regular basis (e.g. every 6 months) you can ensure that no damage or accidental loss has occurred. If it has, you can recover problematic files from the copies or backups you have made previously. Future fixity checks will generate new digital fingerprints (or checksums) for the files in your collections. If they do not match the ones originally created, bit loss or damage has occurred.

Key tasks include:

• Revisit your collection on a frequent basis, recalculate the checksums, identify files that have become damaged
• Retrieve copies of damaged files and repair as necessary
• Perform test recoveries of data backed up by third party services, to ensure backups are being performed as agreed

Document your processes

From the outset of creating a digital collection, it is important to document as much as possible about a collection's assets, the tools and workflows. This documentation is an important component of technical and descriptive Metadata. It is necessary to retain this information for the purposes of longevity. As, with any project, staff retention can be an issue. If staff leave they often take essential knowledge and skillsets.

Where next?

Having taken the first steps in digital preservation, where do you go next? This will obviously depend on your own requirements and priorities, but this table provides a number of suggestions and other sections of the Handbook will help you move forward with them:

Resources

A Preservation Primer

http://knconsultants.org/a-preservation-primer/

This clear practical short primer on preservation for beginners was written by staff at Portico. It summarizes the issues and outlines various short and long-term preservation options that an organization might take to begin planning for long-term digital preservation of its content, beginning with near-term protection and concluding with full preservation and long-term protection. (83 pages).

Putting Parsimonious Preservation into Practice

http://www.nationalarchives.gov.uk/documents/information-management/parsimonious-preservation-in-practice.pdf

The principle of Parsimonious Preservation was originally developed in 2009 at The National Archives in the UK as an approach for small or medium sized institutions to permit them to begin work on digital preservation but is also practical for large scale institutions. It now underpins advice and guidance given to the UK archive sector on digital preservation. (11 pages).

Community Owned digital Preservation Tool Registry COPTR

http://coptr.digipres.org/Main_Page

COPTR describes tools useful for long term digital preservation and acts primarily as a finding and evaluation tool to help practitioners find the tools they need to preserve digital data. COPTR aims to collate the knowledge of the digital preservation community on preservation tools in one place. It was initially populated with data from registries run by the COPTR partner organisations, including those maintained by the Digital Curation Centre, the Digital Curation Exchange, National Digital Stewardship Alliance, the Open Preservation Foundation, and Preserving digital Objects With Restricted Resources project (POWRR). COPTR captures basic, factual details about a tool, what it does, how to find more information (relevant URLs) and references to user experiences with the tool. The scope is a broad interpretation of the term "digital preservation". In other words, if a tool is useful in performing a digital preservation function such as those described in the OAIS model or the DCC lifecycle model, then it's within scope of this registry

DPC Getting Started in Digital Preservation Workshops

https://www.dpconline.org/knowledge-base/training/training-resources

The DPC Getting Started in Digital Preservation workshops are events designed to raise awareness of digital preservation issues, increase involvement with digital preservation activities and sign-post the support and resources available to help you on your way. They provide an introduction to digital preservation, build an understanding of the risks to digital materials, include practical sessions to help you apply digital preservation planning and tools, and feature speakers sharing their own experience of putting digital preservation into practice. You can find details of forthcoming workshops and the programmes and speaker presentations at previous workshops on the DPC events page.

Digital Preservation Management: Implementing Short-Term Strategies for Long-Term Solutions

dpworkshop.org/

An excellent free online tutorial that introduces you to the basic tenets of digital preservation. It is particularly geared toward librarians, archivists, curators, managers, and technical specialists. It includes definitions, key concepts, practical advice, exercises, and up-to-date references. The tutorial is available in English, French, and Italian.

Canadian Heritage Information Network (CHIN) Digital Preservation Toolkit

https://www.canada.ca/en/heritage-information-network/services/digital-preservation/toolkit.html

CHIN has released a suite of documents to identify digital material found in museums, the potential risk and impact of lost material, and how to get started in the development of Preservation Policies, Plans and Procedures. The toolkit includes a Digital Preservation Inventory Template, Digital Preservation Policy Framework Development Guideline, Decision Trees, and a Digital Preservation Plan Framework.

Digital Preservation 101, or, How to Keep Bits for Centuries

https://tpverso.files.wordpress.com/2016/03/swierczek.pptx

This presentation by Julie Swierczek, former Digital Asset Manager and Digital Archivist at Harvard University Art Museums is a good advocacy for and explanation of, digital preservation to othernon-specialist institutional colleagues including "why archivists cry themselves to sleep at night when the general public conflates archives with backup copies of data" (82 slides but many are images with good slide notes that make this easily understandable).

The National Archives Digital Continuity Guidance

http://www.nationalarchives.gov.uk/information-management/manage-information/policy-process/digital-continuity/step-by-step-guidance/step-2/

This guidance takes you through the process of creating an information asset register, and includes a template in Excel spreadsheet format. The register can be useful for Records Managers/Information Managers as a model which they can demonstrate aligns with business risk management.

Risk Management for Digital Preservation

https://vimeo.com/171082277

From a series of video covering topics from the 'Getting Started in Digital Preservation' roadshows, this video provides a brief introduction to the use of risk management for Digital Preservation.

Case studies

Bishopsgate library case study

http://wiki.dpconline.org/index.php?title=Bishopsgate_library_case_study

A collections audit and business case focused on taking the first steps of digital preservation at the Bishopsgate Institute Library. (28 pages).

Starting Small: Practical First Steps in Digital Preservation

http://www.slideshare.net/hakbailey/starting-small-practical-first-steps-in-digital-preservation-13385434

One example of how digital preservation principles can be added to the collections management activities of a small institution (Dartmouth College USA from 2010–2012), without needing a lot of additional resources. (26 slides).

DPC case note: West Yorkshire Archive Service accepts a digital collection

http://www.dpconline.org/component/docman/doc_download/511-casenotemlawyas.pdf

In this Jisc-funded case study staff from West Yorkshire Archives Service report on their experience in taking their first large digital archive. This made them confront new problems and new ways of working, they conclude that "If we try we may fail; if we don't try we will certainly fail". October 2010 (4 pages).

DPC case note: Glasgow Museum takes first steps in turning an oral history headache into an opportunity

http://www.dpconline.org/component/docman/doc_download/502-casenoteglasgowmuseums.pdf

This Jisc-funded case study examines how Glasgow Museums' took some simple steps in addressing digital preservation and created short and long term opportunities. Activities such as creating an inventory, assessing significance and promoting access provide the basis for building confidence to manage the wider challenges, and can bring early rewards if properly embedded within the mission of an organization. September 2010 (4 pages).

Digital Preservation Planning Case Study

http://www.dpconline.org/component/docman/doc_download/863-2013-may-getting-started-london-planning-case-study-ed-fay

A set of DPC Getting Started in Digital Preservation workshop presentation slides by Ed Fay from May 2013. An excellent concise overview of planning for digital preservation and how to approach it . (20 slides).

References

Dollar, C.M. and Ashley, L.J., 2014. Assessing Digital Preservation Capability Using a Maturity Model Process Improvement Approach. Available: http://static1.squarespace.com/static/52ebbb45e4b06f07f8bb62bd/t/53559340e4b058b6b2212d98/1398117184845/DPCMM+White+Paper_Revised+April+2014.pdf

NDSA , 2013. The NDSA Levels of Digital Preservation: An Explanation and Uses, version 1 2013. National Digital Stewardship Alliance. Available: http://www.digitalpreservation.gov/ndsa/working_groups/documents/NDSA_Levels_Archiving_2013.pdf

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

Introduction

This section together with Why digital preservation matters, is designed as a briefing for those new to digital preservation. It is structured into three inter-linked sub-sections covering Threats to digital materials, Organisational issues, and Resourcing issues. It links to more detailed treatment in other sections of the Handbook as appropriate, but has a particularly close relationship to the Getting started section, which is also particularly designed with those new to digital preservation in mind.

Digital preservation can often seem daunting at first. It is important to realise than those with existing skills in either information management or information technology within organisations are well placed to build on and apply these skills to digital preservation activities. However, it may require initially learning some new unfamiliar terminology (see Glossary), extending skill sets, and sometimes working in new ways.

Threats to digital materials

Keeping the data

Every digital file is formed from a series of zeros and ones, or bits (binary digits). These streams of bits need to be captured and retained over time, without loss or damage, to ensure the survival of digital materials. There are an array of threats to any attempt at preserving these bits. Storage media can decay over time, leading to corrupted files. Storage media may become obsolete and unsupported by contemporary computers and the software that understands and provides access to them. The bits may be ignored, abandoned, accidentally deleted or maliciously destroyed. Removable media could be left on a shelf and forgotten, files stored on a shared network drive might be left without an owner, or a third party cloud storage provider could go out of business.

Maintaining a systematic process for bit preservation remains a fundamental requirement in ensuring long term digital preservation. Storage media must be monitored and refreshed (See Legacy media). Redundancy must be introduced by replicating or backing up files, introducing diversity in dependent technologies and avoiding catastrophic disaster at a single geographical location (see Storage). Checksums must be generated and frequently recalculated to identify any loss and ensure that the integrity of the bits can be verified in an efficient and automated manner (see Fixity and checksums). The locations in which digital materials are stored should be carefully recorded, and responsibility for their preservation allocated.

Keeping the meaning of the data

Reconstructing the information that is encoded within a stream of a bits typically requires computer software that is designed to render, manipulate, analyse or otherwise interact with the particular encoding or format of the data. Over time, the encodings (or file formats) may change, and the software applications that interact with them may go in and out of favour. Although unusual for well known file formats, less well used file formats may become obsolete over time, as the software that renders them is no longer supported (see File formats and standards).

Understanding the technology on which particular digital materials are dependent enables appropriate action to be taken to ensure their preservation. A considered preservation planning process might result in the migration of digital files from format to format, the emulation of obsolete software, or the employment of alternative software applications to render the data (see Preservation action). Each of the options presents its own advantages and disadvantages and these need to be evaluated carefully, possibly on a case by case basis (see Preservation planning).

While file format obsolescence has not emerged as the overwhelming danger that was previously perceived, challenging subtleties remain. It may be possible to find a method for rendering an old file format (perhaps by emulating some obsolete software), but how accurate is the rendering, is it legal to run the software, and how much will this complex effort cost the preserver and the user?

Maintaining trust in the data

Digital materials have the potential to remain fluid over time, being edited or altered with ease, being damaged by media failure, or decoded into human readable information in an unreliable or inaccurate manner by rendering software. For an end user to have trust in the result of digital preservation work it requires careful consideration of the entire lifecycle of the digital materials and who or what has interacted with them over time. Information management systems need to be able to link to essential contextual information regarding the business procedures of the creating agency. Authenticity and integrity of digital resources can be equally important in other sectors. For example, scholars will need to feel confident that references they cite will stay the same over time, courts of law will need to be assured that material can withstand legal evidential requirements, government departments may well have legally enforceable requirements regarding authenticity, and so on. This issue overlaps with both legal and organisational issues and it may be one which is best resolved within individual sectors rather than through generic procedures.

The application of data integrity techniques and the maintenance of audit trails can provide confidence that a digital object has remained unchanged (except by necessary preservation action) since deposit in an archive (see Fixity and checksums, and Information security). Ultimately its authenticity to a user may depend much more on the broader trustworthiness of the preserving organisation as a whole. Maintaining high quality preservation processes based on current best practice and validated by appropriate audit and certification will be crucial (see Audit and certification).

Keeping the context of the data and its dependencies

The meaning of digital information can be dependent on additional information that may have been implicit within the context it was originally created or used in, but less clear when revisited at a later date. Identifying, understanding and capturing relevant contextual information can be vital to a successful preservation effort. This might be as simple as capturing the units of measurement used within a spreadsheet, the scale of a map, or the point of origin within a CAD drawing. As digital information continues to be created in a more complex and interconnected manner, it may be necessary to retain the place of particular digital materials within a wider context of associated information resources. What may be seemingly simple and stand alone documents may actually depend on related files, referenced fonts and may have pointers to related information on the web. What might be viewed as a simple web page may have been generated on the fly from live data sourced from different locations on the Internet.

Understanding the data, how it will be used, its dependencies and its context will enable it to be captured for preservation in an appropriate manner and documented in a sufficiently explicit manner to enable the intellectual content to be retained and understood on into the future (see Metadata and documentation).

Acting in a timely manner

Prioritising digital preservation activities and applying them in a timely manner can be crucial not just in avoiding loss but in ensuring the best use of limited resources. Where the opportunity exists to intervene early in the lifecycle, digital materials can be shaped to survive better into the future. The choice of file format, the capture of critical documentation or the description of key relationships in the metadata may require a small investment up front, but could deliver considerable savings further down the line (see Creating digital materials). Where this is not possible, and risks to the data have been identified, the best timing for preservation action can be unclear. Early intervention to head off technological obsolescence may provide greater confidence of long term sustainability but with the risk that intervention may not ultimately be necessary and resources were wasted. Just in time action may minimise unnecessary activity, but increase the effort needed to research obsolete technology in a particular case requiring specialist knowledge that is no longer current. Appropriate action should be taken on a case by case basis.

Coping with the data deluge

Research reported by David Rosenthal noted that the rate of data creation is expanding by about 60% per annum; that developments in data storage allow are expanding at about 25% per annum; and that data centre budgets are expanding at about 2% per annum (Rosenthal, 2014). While this places challenging pressures on selection policies and other organisational decision making it also poses technological questions. Simple preservation processes that function effectively at one level will not necessarily scale easily to work with very large volumes of data or perhaps very large individual files. The technology and understanding to work at scale is moving forward rapidly, with growing expertise for handling large audio visual collections, research data and web based archives (see Content-specific preservation). But some repositories still face significant challenges in developing and maintaining scalable architectures and procedures to handle growing quantities of data. The technical and managerial challenges in accessioning, managing and providing access to digital materials on this scale should not be underestimated. It can be important to remember that selection, appraisal and disposal are significant components in any digital management activity.

Organisational issues

While technological issues can be challenging, there are also numerous challenges which relate to organisational issues. These include how digital preservation is organised and delivered, or how those responsibilities change over both time and the lifecycle of digital materials. There are common digital preservation challenges faced across organisations, yet every organisational context will be different. It is vital to ascertain organisational drivers and tailor practical solutions to meet these needs. There is no one size fits all approach for digital preservation.

The creation, preservation and access for digital materials are widely distributed. As a result, there is an increasing need to go beyond the confines of individual organisations, or even countries, to maximise the benefits of the technology, address common issues, and to overcome the challenges cost-effectively.

In-house or outsource?

The decision whether to do all or part of digital preservation via a third-party or in-house, or perhaps a combination of the two, is often a complex one. Digital preservation may be undertaken in-house if there is sufficient staffing and infrastructure but outsourcing some activities or support can be cost-effective, and can leverage internal capabilities and capacity.

Outsourcing specific tasks or services from a repository is by no means a new phenomenon. Repositories have contracted out some of their operations for decades. Of critical importance is having and retaining sufficient knowledge to be able to prepare effective specifications and monitor performance. Outsourced work must be easily verified and quality checked, and this is best enabled via careful design of the specification, and the reporting providing by the 3rd party. Cost will clearly be a key consideration when deciding whether or not to contract out digital preservation but there are also other factors to consider such as legal issues. For example, legal provisions due to privacy or confidentiality may influence whether outsourcing is appropriate or not. The advantages and disadvantages of each option will need to be balanced in light of the individual organisation's mission and responsibilities (see Procurement and third party services and Cloud services).

Collaboration

There is a significant overlap in the digital preservation issues being faced by all organisations and across all sectors so it makes sense to pool expertise and experience. There are compelling reasons and, in some cases, political pressure, to engage in greater collaboration within and between organisations in order effectively to confront and overcome the challenges of digital preservation.

Most organisations readily acknowledge the benefits of increased collaboration but also indicate the potential difficulties that can arise in the form of differing agendas, timescales, or funding mechanisms. None the less, it is often possible to collaborate in specific areas or with different levels of intensity that moderate these potential difficulties. Some of the most high-profile and successful initiatives in digital preservation of recent times have been collaborative in nature (see Collaboration).

Organisational change

The modern digital world is a place of both rapid technological and organisational changes. Organisations re-organise internally, merge, or cease to operate with increasing frequency. Digital preservation is a long-term activity and the likelihood of it being affected by organisational change increases over time. This may affect a repository not only through changes to its parent organisation, but through changes to its major depositors and users, suppliers, or collaborators. Organisational change is therefore a major risk to be managed (see Risk and change management).

Organisational structures

The nature of the technology and dependencies in the preservation of digital materials are such that there are implications for organisational structures. Many of the activities converge, for example decisions about acquisition and preservation should sensibly be made at the same time. Organisational structures will need to cross boundaries in order to draw on the full range of skills and expertise required for digital materials. Assigning responsibility for preservation of digital materials acquired and/or created by an organisation will inevitably require involvement with personnel from different parts of the organisation working together. This can potentially present difficulties unless underpinned by a strong corporate vision which can be communicated to staff (see Collaboration, Advocacy, and Staff training and development).

Roles and responsibilities

There are some existing repositories which undertake responsibility for specific subject areas or specific formats. In the UK, for example, the UK Data Service undertakes responsibility for selected social science research data, while the British Library's National Sound Archive assumes responsibility for its collection of sound recordings. Each repository will need to consider its own collection policy and the broader landscape of collecting institutions and remits within which it sits.

The digital environment demands engagement with a large group of stakeholders. The lifecycle approach to digital preservation advocated in the Handbook has significant implications for the way organisations responsible for long-term preservation need to interact and collaborate with creators, publishers and other intermediaries, and each other.

Creators of digital materials need to be able to understand the implications of their actions in terms of the medium to long-term viability of the digital material they create. Whether it be a record created during the day-to-day business of the department, a digital copy of analogue collection material, or a "born digital" resource, guidance and support as well as an appropriate technical and organisational infrastructure will assist in facilitating greatly improved prospects for efficient management and preservation (see Creating digital materials).

Selection

The enormous quantity of information being produced digitally, its variable quality, and the resource constraints on those taking responsibility to preserve long-term access, makes selectivity inevitable if the objective is to preserve ongoing access.

In the digital environment non-selection for preservation may almost certainly mean loss of the item, even if it is subsequently considered to be worthwhile.

In cases where there may be multiple versions, decisions must be made in selecting which version is the best one for preservation, or whether more than one should be selected. Sampling dynamic resources as opposed to attempting to save each change, may be the only practical option but may have severe repercussions if the sampling is not undertaken within a well-defined framework and with due regard to the anticipated contemporary and future needs of the users.

Some consideration also needs to be given in the selection to the level of redundancy needed to ensure digital preservation. There needs to be a clear understanding of who will undertake that responsibility and for what period of time. Otherwise, even if several copies are stored in various repositories, all of those repositories might, for a variety of reasons, cease maintenance of the digital object at some point (see also Acquisition and appraisal).

Balancing security and access

There has always been a strong link between preservation and access. Repositories need to ensure that their digital materials are safe and secure, but most also provide access to a variety of users. Access by real users can provide a valuable steer to the design of preservation facilities, helping to avoid unnecessary actions but also validating and introducing a feedback cycle.

Many types of digital material selected for long-term preservation may contain confidential and sensitive information that must be protected to ensure they are not accessed by non-authorised users. In other cases there may be legal or regulatory obligations on the repository affecting access. There can be tensions between these two roles and a need to strike a balance between security and ease of access (see Access, and Information security).

Legal compliance

Legal issues are not simple in digital preservation. Multiple copies and derivative versions often exist of digital materials, and there may be associated software and metadata with them from different sources. Digital content is generated by a wider group of creators and incorporates more diverse formats and intellectual property rights (IPR) than applies in the analogue world. The law also often lags behind technological change and digital preservation needs. Some of the key legal issues that affect repositories in collecting, preserving, and providing access to digital materials are:

• Any legal requirements in terms of management, preservation, and access placed upon the repository and its parent organisation, by donors and funders via contracts and agreements or via legislation by Government (e.g. accessibility, availability, information security, retention, audit and compliance, Public Records, Legal Deposit, etc.);
• Those legal obligations relating to third party rights in, or over, the digital materials held by the repository (e.g. copyright, data protection); and
• The legal elements of any relationship between a repository and any third-party provider or providers (e.g. terms of service contracts and service level agreements).

For further guidance and resources to help address these issues and manage associated risks, see Legal compliance and Procurement and third party services sections respectively of the Handbook.

Resourcing issues

Budgets and costs

The cost of digital preservation cannot be easily isolated from other organisational expenses, nor should it be. Digital preservation is essentially about preserving access over time and therefore the costs for all parts of the digital life cycle are relevant. In that context even the costs of creating digital materials are integral in so far as they may need to include cost elements which will ultimately facilitate their long-term preservation (see Creating digital materials).

The ability to employ and develop staff with appropriate skills is made more difficult by the speed of technological change and the range of skills needed. It is also limited by resource constraints on organisations which may well need to manage growing traditional collections and digital collections without additional resources.

Nonetheless the exercise of calculating costs, however complex, is a valuable and necessary task to establish cost-effective practises and a reliable business model. The cost of the labour required for digital preservation will be the most significant by far and includes not only dedicated experts but varying proportions of effort from many staff such as administration, management, IT support, legal advisers etc.

Other major issues to impact costs include organisational mission and goals, including the type and size of collections, the level of preservation committed to, the quantity and level of access required, and time frame proposed for action. These are discussed in detail in the section on Business cases, benefits, costs, and impact.

The relationship of costs and institutional strategies and activities such as Collaboration, Procurement and third party services, Legal compliance, Staff training and development, or Standards and best practice are also discussed in the relevant sections of the Handbook.

Staffing and skills

Digital preservation involves a range of skills and organisational roles. Typically digital preservation draws on a range of skills which are not normally found in combination. That means larger organisations will likely need to assemble multi-disciplinary teams while in smaller organisations it will be necessary to rely on a distributed team or sources of support.

There are three main issues to consider with respect to staffing and skills:

• Firstly, although there have been considerable improvements in recent years, digital preservation teaching often lags behind current best practice or is wholly theoretical within relevant information management programmes for new entrants into the profession. So individuals with practical skills and experience are in high demand and staff can be hard to recruit.
• Secondly, job descriptions can be hard to script, especially when agencies are effectively starting from scratch with a new role. To this end a number of research projects have attempted to describe generic skills needed for digital preservation, using as a basis the assumption that different skills are required at different levels of an organisation. Tools like the DigCurv Skills framework allied to the Digital Preservation Coalition's Vacancies section can be very useful when describing new roles. Larger organisations with multi-disciplinary teams may be able to recruit to roles that are 'digital' variants of existing professional categories such as archivist, librarian or records manager, but for most organisations new types of roles must be created.
• Finally, staff working in digital preservation frequently report the need to engage in active career development. Given the expectation that technology and the needs of users develop through time, so the staff involved in meeting these changing requirements will need to find ways to have their skills constantly refreshed, such as through specialist briefings and professional networking (see Staff training and development).

Facilities

Effective digital preservation requires some basic facilities or infrastructure, typically technological in nature, on which operational workflows and the processing of digital material can be based. While these may be rudimentary or at least small scale in nature when an organisation takes its first steps in digital preservation, ramping up operations to address large quantities of data will require considerable investment in the facilities required to support it.

Storage

With the typical requirement of replicating preserved data to avoid loss, storage hardware remains amongst the most important digital preservation facilities. Storage technology has changed rapidly over recent decades. Archives widely used media such as CDs or DVDs for long term storage, but the rapid developments in magnetic media have brought fast and reliable storage that has made handheld media redundant. Enterprise storage systems now provide large storage volumes at reasonable cost. While they have finite lifespans, typically of around 4-8 years, they are easy to monitor and then replace when they reach end of life (see Storage).

Organisations may also wish to consider cloud services to "rent" preservation infrastructure. The flexibility of the cloud allows relatively rapid and low-cost testing and piloting. Cloud services can provide easy, automated replication to multiple locations and access to professionally managed digital storage and integrity checking. Repositories can add access to dedicated tools, procedures, workflow and service agreements, providing a digital repository system tailored for digital preservation requirements via specialist vendors (see Cloud services).

Digital repository systems

Many of the core requirements for preserving digital materials are provided in an automated fashion by dedicated digital preservation systems, or trusted digital repositories. A repository application will uniquely identify each digital object placed within it. It will manage the storage of that object, identify its characteristics and help a repository manager to plan its preservation. It will also facilitate access to the object. While basic preservation can be provided on an ad hoc basis at a small scale, a dedicated repository application is essential to managing digital materials effectively over time. The OAIS model provides a high level model for the functions required by a repository (see Audit and certification for more information on certification of trusted digital repositories and Tools for repository systems and components).

High performance computing

Increasing volumes of data require not only more storage but also greater computational power. Characterising and assessing the technical characteristics of data, indexing data to enable search and access, integrity checking and a host of other tasks require considerable computational performance. Those dealing with these big data, be it research data or web archives have typically looked to high performance computing, and technologies such as Apache Hadoop running on clusters of commodity hardware to meet this need.

Digital preservation laboratory

A number of larger organisations have developed lab environments within which an array of old and new technology can be applied for the stabilisation or ripping of data from obsolete media, and has been championed by organisations working with personal digital collections. Specialist drives for reading magnetic media, robots for processing large numbers of optical disks and write blockers for allowing access to hard drives without changing the bits in the process, are just some of the equipment that could be useful here. Media recovery companies offer an alternative approach that may be preferable in high volume cases, albeit with less control of the process and the need to move media offsite (see Digital forensics).

Resources

How Toy Story 2 Almost Got Deleted: Stories From Pixar Animation: ENTV

https://www.youtube.com/watch?v=8dhp_20j0Ys

Entertaining and informative story of how 'Toy Story 2' was almost deleted from Pixar Animation's computers during the making of the film and how the film was saved by one mom's home computer (2 mins 26 secs)

References

Rosenthal, D., 2014. Talk "Costs: Why Do We Care?", DSHR's Blog, Tuesday November 18 2014. Available: http://blog.dshr.org/2014/11/talk-costs-why-do-we-care.html

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

Introduction

This section together with Preservation issues is designed as a briefing for those new to digital preservation. It is structured into four inter-linked sub-sections. In addition it has a close relationship to the Getting started section, which is also particularly designed with those new to digital preservation in mind.

Digital preservation: the challenge of a generation

Any digital object can be considered in scope for digital preservation: born digital or digitised, corporate or personal, innovative or routine. Digital preservation can encompass texts and images, databases and spreadsheets, vectors or rasters, programs and applications, desktop files and enterprise systems, email and social media, games, movies, music and sound, entire web domains and individual tweets. Digital collections can derive from laptops or desktops or smart phones; from tablets, souped-up servers or hulking great mainframes. They can be snapped at the end of a selfie stick or beamed from sensors deep in space; they can be generated by tills and cash machines, by satellites and scanners, by tiny sensitive chips and massive arrays. They can be stored in repositories or data centres or USB sticks. There is no digital object or system that is not provisionally within scope for digital preservation.

Pervasive, changing and ubiquitous, digital technologies are a defining feature of our age. Digital materials are a core commodity for industry, commerce and government. They are fundamental for research, the law and medicine. The creative industries, cultural heritage and the media depend on reliable access to digital materials while families and friends extend and sustain their relationships through digital interactions.

But digital materials - and the opportunities they create - are fragile even if they also have the capacity to be durable through replication. Digital platforms change and the long chains of interdependence on which they depend are complicated and fluid. Their longevity and utility is threatened where contents or contexts are lost: engagement and exploitation are enabled when digital materials endure. The greater the importance of digital materials, the greater the need for their preservation: digital preservation protects investment, captures potential and transmits opportunities to future generations and our own.

Already we have made great strides in averting a "digital dark age". There are a growing number of repositories all over the world that can claim a long track record of keeping digital materials well over many decades (for example the UK Data Archive founded in 1967). This gives us a broad foundation of experience and collaborative professional networks to draw on.

It is a shared, generational challenge.

The always emerging digital preservation challenge

The unifying characteristic of digital materials is their machine-dependency. Information can only be accessed and functions can only be executed through a computer. As technology becomes more sophisticated this dependence becomes an ever more elaborate chain of inter-dependencies that are hard to track and tricky to maintain.

So long as the IT sector remains innovative in its provision of new tools and technologies, digital preservation managers will respond by devising effective strategies for ensuring the durability and usability of new digital materials, so digital preservation will remain an always-emerging challenge.

To ensure the value of digital materials in the long run we need to ensure access, which in turn means we need to understand and mitigate rapid changes in technology and organisations (see Preservation issues).

Digital material can often only be archived well in digital form: there is no non-digital equivalent such as paper that retains all the essential information and functionality it provides. Too often it has been necessary to print out digital material for archiving and then even re-digitizing the printed copy later because there has been no local capacity for managing born digital material.

Today we have a growing and effective body of approaches, experience, and collaboration to address the challenges. Digital preservation is an important, necessary and doable endeavour with simple first steps all can undertake (see Getting started).

What is in scope?

Simply because everything could be in scope for a digital preservation strategy does not mean that everything should be preserved.

The question is less what can be preserved so much as what should not be lost. Selection, appraisal and disposal are significant components in any digital management activity. In the context of an expanding digital universe, a determined effort to identify, process and retain digital material of enduring value means on one hand that the right material is available to the right people at the right time in the right format; and on the other hand material is identified that can be actively removed or benignly neglected.

Digital material provides profound new opportunities for access and use of repositories. If digital collections exist in a fast changing environment, then we should expect that our users do too. Users of digital materials are likely to be using technology that is not yet fully developed in ways that we cannot fully anticipate, in places we may never visit and for purposes that we may struggle to predict. So any meaningful answer to the question of 'how can we preserve digital materials' will rapidly resolve to 'what can we do to ensure that these digital materials can be used'? Preservation planning will only succeed when user needs are fulfilled.

All of this indicates a requirement that wherever possible the long term viability of digital materials should be defined early not late. Preservation action is needed at the start of the life of a digital object, not always at its end. Creation, management and archiving of digital materials are no longer at opposite ends of a process but are integrated all the way through. By extension, preservation is no longer simply a concern for memory institutions in the long term but for everyone interested in using and accessing digital materials.

Who needs to be involved?

The ability to preserve digital materials depends upon a wide range of stakeholders. Principal among these are the creators of digital content, whose involvement in their preservation might involve, for example, consideration of standards in terms of format and media, and ensuring enough contextual information is available to enable their management by others. Creators may often be unaware of their pivotal role. This could be for all kinds of reasons, but a vital part of any digital preservation effort is the effective dialogue with creators of digital materials to inform and advocate the value of their engagement (to them and others).

If the creators of digital materials have a responsibility to enable long term access, then this responsibility is borne even more fully by those who provide the infrastructure and environments in which they are created. In some cases this may be a corporate function, with the provision of corporate tools and services which are preservation ready. In other cases responsibility will be borne by external service providers who host digital infrastructure for clients.

The nature of digital technology dictates that it is not feasible simply to hand over stewardship of the resource at some point in the future, without having managed it sufficiently to facilitate sustainability.

In some cases, institutions will manage their own digital legacy: large institutions that create digital materials may most sensibly be the ones to manage them in the long term, thus maximising return on their initial investment. But in other contexts co-operative models for long-term preservation have emerged involving a number of organisations. Both subject specialist and expert centres have emerged offering specific preservation solutions for specific types of digital material.

For some organisations, it may prove more cost-effective to contract all or part of their digital preservation activities to a third party. Whilst it may be advantageous to outsource, it is important to remember responsibility remains with the organisation. Staff will need to be sufficiently aware of digital preservation issues, particularly as they relate to legal, organisational and contractual problems, to manage these third party contracts effectively.

Any institution which places value on digital resources in general needs to ensure the long-term preservation of digital materials. A significant number of institutions have not only taken that role on for themselves but have offered wider leadership in addressing the practical implications of digital preservation.

Ultimately however, digital preservation cannot be perceived as solely a concern for archives, libraries, museums and other memory institutions: it is a challenge for all who have an interest in creating, using, acquiring and making accessible, digital materials.

Resources

Why Digital Preservation is Important for Everyone

https://www.youtube.com/watch?v=qEmmeFFafUs&index=43&list=PLEA69BE43AA9F7E68

Short Library of Congress video produced in 2010 for the non-specialist audience explaining how traditional information sources such as books, photos and sculptures can easily survive for years, decades or even centuries but digital items are fragile and require special care to keep them useable. Rapid technological changes also affect digital preservation. As new technologies appear, older ones become obsolete, making it difficult to access older content. (2 mins 51 secs)

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

Who is it for?

Senior administrators (DigCurV Executive Lens), operational managers (DigCurV Manager Lens) and staff (DigCurV Practitioner Lens) within repositories, funding agencies, creators and publishers, anyone requiring an introduction to the subject.

Assumed level of knowledge

Novice.

Purpose

• To provide a strategic overview and senior management briefing, outlining the broad issues and the rationale for funding to be allocated to the tasks involved in preserving digital resources.
• To provide a synthesis of current thinking on digital preservation issues.
• To distinguish between the major categories of issues.
• To help clarify how various issues will impact on decisions at various stages of the life-cycle of digital materials.
• To provide a focus for further debate and discussion within organisations and with external audiences.

  Download a PDF of this section.

Save

Save

Save

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

Development of the Handbook

The Handbook was first created in 2001 as a print publication Preservation Management of Digital Materials: a Handbook, printed by British Library Publishing (Jones and Beagrie, 2001). A limited edition CD of the Handbook was produced in February 2002 for the launch of the DPC at the House of Commons. The online digital version of the Handbook was made available on the DPC website shortly after this in May 2002. An archived version of this "first online edition" of the Handbook is available in the UK Web Archive (UKWA, 2009).

For over a decade subsequently, the online edition of the Handbook has provided an internationally authoritative and practical guide to the subject of managing digital resources over time and the issues in sustaining access to them. However, only a few small additions have been made to the Handbook since it was made available online. The DPC set itself a strategic target to raise research grant funding to completely update the Handbook and integrate it more closely with the Technology Watch series to maintain its future currency and ease of updating.

In 2014, The National Archives (TNA) as lead funder with supplementary funding from the Archives and Records Association, British Library, Jisc, and National Archives of Scotland, provided a research grant to begin the full revision of the Handbook and promoting its use and uptake. This work was undertaken under the direction of William Kilbride Director of the DPC and the editor Neil Beagrie, and has been supported by an extensive advisory board and set of contributors.

The Digital Preservation Coalition (DPC) is a not-for-profit organisation that makes the Digital Preservation Handbook freely available as an online resource: it is heavily used for Continuous Professional Development, by university students, and for training in digital preservation. A review of digital preservation training by the APARSEN project has highlighted the narrow range of topics covered by most initiatives (APARSEN, 2012). It highlights current gaps in topic coverage that suggested an updated revised edition of the Handbook would meet ongoing needs.

As part of preparation for the 2nd edition, a mapping was undertaken between the Handbook and Aparsen recommendations and also to the DigCurV Curriculum Framework for Digital Curation (DigCurV, 2013). Extensive audience research informed the Handbook's proposed development and this included input from 285 members of the community via an online survey (DPC, 2014a) and a public consultation on a draft content outline (DPC, 2014b).

The Harvard System has been used for references, in the same style as the DPC Technology Watch Reports.

This focus on engaging the digital preservation community to ensure the Handbook represents the needs of a broad range of organisations and sectors, has continued through its revision via use of collaborative "book-sprints" and its Advisory Board.

Acknowledgements

The Second Edition of the Digital Preservation Handbook was delivered from a collaborative research project based on a consortium agreement between The National Archives, The Archives and Records Association, the British Library, Charles Beagrie Ltd, the Digital Preservation Coalition, and Jisc.  

• The project funding was initiated by The National Archives.  Aware of the digital preservation challenge faced by the archives sector and responsive to its role to provide strategic sector leadership, it offered 50% of the estimated required project funds (ie Gold Sponsorship) on the understanding that the DPC should either find the remaining funds by attracting additional sponsors or underwrite the costs directly.  
• The DPC Board, eager to revise the Handbook, agreed to the offer from The National Archives, offering DPC staff time and ensuring that material costs (such as the website) would be supplied at no cost. The DPC also made a cash contribution from member funds. 
• Charles Beagrie Ltd joined the consortium as the editorial partners to undertake original research, develop content and design, and co-ordinate quality assurance.  Charles Beagrie Ltd provided an in-kind contribution of staff time (equivalent to Bronze Sponsorship), helping to underwrite the final editorial tasks to complete the Handbook.  
• Jisc joined the collaboration early recognising that archivists and research data managers in higher education would be well placed to use the Handbook.  Jisc’s funds (Silver Sponsorship) enabled the  development of the three content specific preservation sections and the ‘book sprint’ methodology.  
• The British Library also joined the collaboration early, reflecting their role in the original Handbook.  Funds from the British Library (Silver Sponsorship) enabled a comprehensive needs assessment and survey which formed the basis of the revised table of contents.  
• After a period of content development and proto-typing, the Archives and Records Association (ARA) joined the collaboration offering funds (Silver Sponsorship) from their own research grants, ensuring that draft texts were systematically and comprehensively peer-reviewed prior to release, and that the DPC was able to take the Handbook ‘on the road’ in a series of training events accessible to ARA members.
• Finally funds were supplied from the National Records of Scotland (Bronze Sponsorship) to enable wide dissemination and timely launch.  

The DPC Board formally thanks the partners, noting also the many additional contributions, introductions and encouragements that they made to the successful completion of the project.

Funders

Gold sponsor

Silver sponsors

Bronze sponsors

digitalbevaring.dk has kindly given permission for use of the illustrations in the section headings of the Handbook and the icons in the Resources and Case Studies, produced by Jørgen Stamp. These are copyright of digitalbevaring.dk and shared under a CC BY 2.5 Denmark licence (illustrations) https://creativecommons.org/licenses/by/2.5/dk/deed.en_GB , and a CC0 1.0 licence (icons) https://creativecommons.org/share-your-work/public-domain/cc0.

Handbook Managing Editor: Sharon McMeekin (from June 2016)

Previous Managing Editor: Neil Beagrie (until June 2016)

Handbook Contributors

Management board

Advisory board

Peer reviewers

We would also like to record our thanks to the 26 individuals who have contributed to peer review of draft sections of the Handbook relevant to their areas of expertise and interest. We are grateful to members of the advisory board and in addition to Hugh Campbell (Public Record of Northern Ireland), Andrew Charlesworth (University of Bristol), Carey Clifford (Grosvenor Group), Lee Hibberd (National Library of Scotland), Neil Jefferies (University of Oxford), Catherine Jones (Science and Technology Facilities Council), Naomi Korn (Naomi Korn Copyright Consultancy Ltd ), Ingrid McDonald (Queensland State Archives), Laura Molloy (University of Oxford), Barbara Sierman (KB), Charlene Taylor (Worcestershire Archive & Archaeology Service), Kate Watson (Dorset County Council), Rebecca Webster (UCL Institute of Education), and Cathy Williams (The National Archives) for their reviews and suggestions that have made very significant contributions to the Handbook.

Save

Applying it in practice

The needs of institutions regarding the digital materials they create and acquire vary considerably. This Handbook is intended to provide a bridge between broad, high level overviews and explicit, detailed guidelines applicable to the needs of a specific institution. The strategic overviews are intended to link to operational activities in order to reinforce the need to develop practical procedures grounded firmly in the business mission of the institution. The Handbook provides pointers to where to find further guidance and to assist in developing policies and practices which are most applicable to the individual institution.

Ideally, the Handbook should be used to help focus thoughts, increase overall understanding, promote training, and act as a catalyst for further action. Nothing will preclude the need for each organisation ultimately to commit the necessary resources to an action plan but this Handbook is intended to oil the wheels of that process.

Handbook resources

The Handbook is advocating an overall approach to preserving digital resources based on sound principles and policies rather than prescriptive formulae. As the crucial importance of digital preservation becomes more widely recognised, an increasing number of valuable sources of guidance are appearing at a rapid rate. While potentially incredibly valuable, their proliferation can make it bewildering to decide which ones are likely to be most applicable for a given situation.

By selecting key Resources and Case Studies in each section, the Handbook should make it easier to navigate through existing sources of advice, guidance and options. As well as pointing to existing sources of guidance, we have used a combination of decision tree, summary checklists, selected exemplars and case studies, and commentary. These are intended to stimulate and promote further thought and discussion but above all, to stimulate action by institutions to develop digital preservation management policies and strategies appropriate to their needs.

Resources have been grouped into categories and denoted by the following icons:

Publications	Tools	Web resources	Video and webinars	Case studies

Introductory overviews

The Handbook is intended for a wide and diverse audience, from those who are only beginning to consider managing digital materials to practitioners who have already accumulated considerable theoretical and/or practical experience. It has been written with the intention of allowing quick and easy access to the most appropriate sections.

Each section is preceded by an 'at a glance' guide to its intended primary audience, their assumed level of knowledge, and the purpose of the section. The table below will help you decide which sections are likely to be most relevant. to you. It is however not intended to be rigidly prescriptive and anyone wishing to, can of course read the Handbook in its entirety!

All readers are encouraged to read the Introduction.

The Glossary provides concise explanations of key concepts and definitions of acronyms and initials used by organisations and projects throughout the Handbook.

Recommended sections and audiences

Re-use of the Handbook

Re-use in English

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

There is a rapidly increasing volume of information which exists in digital form.

Whether created as a result of digitising non-digital collections, as a digital publication or artwork, or as part of the day-to-day business of an organisation or individual, more and more information is being created digitally and the pace at which it is being created is accelerating. This activity is occurring in an environment in which there is a growing awareness of the significant challenges associated with ensuring continued access to these materials, even in the short term.

The combination of these two factors is both challenging and troublesome. On the one hand, there are considerable opportunities offered by digital technology to provide rapid and efficient access to information. On the other hand, there is a very real threat that the digital materials will be created in such a way that not even their short-term viability can be assured, much less the prospect that future generations will also have access to them.

The need for guidance

Guiding principles

The following principles have guided development of the Handbook that it should:

• Be consistent with the mission of the DPC by being vendor independent, encouraging pro-active participation by the community, and sharing knowledge and best practices.
• Be derived from the needs of the digital preservation community and deliver benefit to it. The community of current and potential users of the Handbook will be surveyed and their feedback used to inform its development. Future evaluations will be used to provide further user feedback and impact assessment.
  
• Be informed, current, concise and balanced. Chapters will provide concise overviews of topics with a selection of further reading and case studies providing opportunities to pursue topics in greater depth. Selection of further reading and case studies will be guided by the utility, currency and continuing relevance of their content.
  
• Lower the barriers to participation in digital preservation by being written in an informative but accessible style for a wide audience.
  
• Have peer-reviewed, authoritative content with explicit quality criteria as set out in the DPC Notes for Authors, Reviewers and Editors and in the Handbook project plan.
  
• Work together and achieve synergies with, reports in the DPC Tech Watch Series. The reports provide “deep dives” in either specific areas of content preservation (e.g. email) or topics (e.g. digital forensics) that can be cited or to source case studies in the Handbook.
  
• Promote other resources (e.g. the Community Owned digital Preservation Tool Registry -COPTR) that may be the most appropriate means of providing advice in areas of detail outside of the Handbook.
• Promote other resources (e.g. reports, guidance) that are accessible via the web and free to users, so wherever possible all selected external content is as accessible as the Handbook itself.
  
• Be developed for ease of maintenance, cost-efficiency, and sustainability in the long-term by the DPC.

Future development and support

On completion of the second edition, the DPC intends to integrate its maintenance and review with its Digital Preservation Technology Watch Report series. Oversight for commissioning of the revision of Handbook chapters and the Reports, or commissioning of new topics for Reports or content in the Handbook will be the responsibility of a DPC editorial advisory board.