Audit and certification

Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark

Introduction

Organizations are increasingly interested in evaluating their digital preservation infrastructures against an assessment framework, and audit, certification, and self-assessment are hot topics in digital preservation. It is worth taking a moment to consider the difference between a self-assessment exercise and an audit.

Audit and certification is a formal process commonly carried out and delivered by external service providers. It is often a time consuming experience with exactingly high requirements that demonstrate to an external audience that a particular standard is being complied with.

Self-assessment is a precursor, or alternative, to a full audit and is typically delivered by staff inside of the organization, and the results are usually of highest value to the organization being assessed (rather than an external audience). Self-assessments can be useful in identifying practices which are underdeveloped and require improvement, particularly if an organization is interested in pursuing full audit and certification at a later date.

Many of the benefits can be summarised as ensuring that a repository can be trusted. The concept of a trusted or trustworthy digital repository is now broadly recognised in the digital preservation community. The following section summarises the work that has taken place over the past 10 - 15 years to get us to this point.

Background to development of audit and certification frameworks

Audit and certification methods for digital preservation implementations have been in development for well over a decade with different organizations developing different methodologies in parallel. In Europe these are now coalescing under the European Framework for Audit and Certification of Digital Repositories.

The OAIS Reference Model (ISO, 2012a) (see Standards and best practice) influenced the development of the different methodologies, which began with the publication of Trusted digital repositories: Attributes and responsibilities (RLG/OCLC, 2002). This was refined as the draft publication An audit checklist for the certification of trusted digital repositories (RLG-NARA, 2005) before being finalised as TRAC (Trustworthy Repositories Audit & Certification: Criteria and Checklist) (CRL, 2007).

Equivalent activity was also taking place in both the Netherlands and Germany. The self-assessment process, Data Seal of Approval developed by DANS (Data Archiving and Networked Services), was released in 2008. Meanwhile, based on recommendations from a working group of nestor, the German Standards Committee (DIN) adopted DIN 31644 Information and documentation - Criteria for trustworthy digital archives.

Following their publication of the OAIS standard, and the later adoption of OAIS as an ISO Standard, in September 2011 the Consultative Committee for Space Data Systems released recommended practice on "Audit and certification of trustworthy digital repositories", This was subsequently adopted and published as ISO 16363 2012 Audit and certification of trustworthy digital repositories (ISO,2012b).

Current assessment options and the European Framework for Audit and Certification

The apparent proliferation of repository audit standards has been frequently cited as a barrier to participation. Consequently the European Commission has hosted a series of meetings to discuss a European-wide approach, and there is now a Memorandum of Understanding to define a European Framework for Audit and Certification of Digital Repositories. This memorandum effectively creates a tiered approach to certification, allowing an entry-level self-assessment and peer review based on the CoreTrustSeal (previously the Data Seal of Approval), a more extensive self- assessment (based on DIN 31644 or ISO 16363), and a full scale external audit based on ISO 16363.

1. Data Seal of Approval

The Data Seal of Approval (DSA, 2008) is a self-assessment process for digital archives, aimed specifically at those archives that hold data. Though an outlay of time is needed to apply for the DSA, it is far less onerous than ISO 16363, having only sixteen guidelines on which the organisation is assessed. The guidelines are based on the following five criteria:

• The data can be found on the Internet;
• The data are accessible (clear rights and licences);
• The data are in a usable format;
• The data are reliable;
• The data are identified in a unique and persistent way so that they can be referred to.

Though the DSA is on the surface a self-audit, this self-audit is then peer reviewed before a seal is awarded, thus adding a level of authority to the process. Openness and transparency are encouraged and institutions are asked to make their evidence (essentially documentation, policies and procedures) freely available online. Unlike an audit under ISO 16363, the peer reviewer is not required to visit the institution to see that the policies and procedures are working in practice, so this process is very much based on trust.

DSA are in the final stages of reviewing proposed amendments to the DSA Guidelines as a result of work with the World Data System through the Research Data Alliance. Details of when and how the transition to new guidelines will be managed will be released in due course, but in the meantime the current seal will be extended through 2017.

2. DIN 31644 Information and documentation - Criteria for trustworthy digital archives

The DIN Standards Committee in Germany adopted DIN 31644 Information and documentation - Criteria for trustworthy digital archives based on recommendations from a working group of the German competence network for digital preservation (nestor). The standard consists of requirements for a trustworthy digital repository structured in three sections:

The organisational framework requires that:

• The repository has defined goals for the selection of digital material and accepts the responsibility to preserve them over the long- term;
• The repository has a defined community for whom access and the ability to interpret digital materials will be provided;
• There is observation of legal and contractual rules between data creators and the digital repository;
• Sufficient organizational structures are provided in terms of personnel, finance, long-term planning and continuity of service;
• Processes and responsibilities are defined and documented.

Object management requires that:

• The integrity and authenticity of digital material are maintained;
• A strategic plan for digital preservation activities is in place;
• Information packages for ingest, storage and dissemination are defined;
• Adequate documentation is provided including permanent identifiers and sufficient structural, technical, rights and change metadata;
• The digital material and related metadata are packaged together for permanent preservation.

Infrastructure and security requires that:

• The IT infrastructure can deal with the digital material adequately and is secure.

DIN 31644 is in German but an English translation is provided by nestor on their website.

The extended certification process undertaken by nestor takes about three months. Guidance on this process, (nestor Certification Working Group, 2013) is available on their website. This certification process should not be confused with full external audit- this requires formal accreditation under ISO 16363.

3. ISO 16363 Audit and certification of trustworthy digital repositories

ISO 16363 is an evidence-based audit framework that uses the term 'repository' to mean the organisation responsible for digital preservation rather than just the technical infrastructure being used for storage. The criteria used in the standard look across the entire organisation and not just the technical system in which collection content is stored. Metrics are grouped into three areas:

• Organizational Infrastructure: including governance, organizational structure, staffing, procedural accountability, policy framework, financial sustainability and contracts, licensing and liabilities;
• Digital Object Management: including acquisition and ingest, preservation planning, creation and preservation of Archival Information Packages (AIPs), and information and access management;
• Infrastructure and Security Risk Management: including technical infrastructure, risk management and security risk management.

Terminology used in ISO 16363 is directly aligned with that of OAIS and the standard asks directly about both OAIS information packages and functional areas. A basic understanding of OAIS is therefore useful for those seeking to understand ISO 16363 and deliver an assessment against it.

With over 100 metrics spread across the three areas, undertaking an ISO 16363 audit or assessment is a significant commitment similar to many other ISO standards applied across organisations. A relatively small number of organisations have utilised the ISO 16363 standard since it was published. Some have sought certification by external auditors whilst others have undertaken self-assessments. Houghton (2015) acknowledges that even though a self-assessment is not an audit it is nonetheless a significant undertaking that should be tailored to organisational circumstances.

ISO 16363 follows ISO practice for certification which assumes that those carrying out the audit are themselves certified. Two other ISO standards support this:

• ISO 16919 Requirements for bodies providing audit and certification of candidate trustworthy digital repositories (ISO, 2011) that sets out the requirements for any organisation that certifies the auditors for ISO 16363; and
• ISO 17021 Requirements for bodies providing audit and certification of management systems (ISO, 2012a) provides a mechanism to audit accreditation bodies.

An agency called PTAB (Primary Trustworthy Digital Repository Authorisation Body) offers training for auditors and those preparing for audit. Other agencies including the Center for Research Libraries are also providing audits against these standards.

4. Other frameworks and tools for self-assessment

A useful entry level resource is the Levels of Digital Preservation from NDSA (NDSA, 2013). This is particularly useful for those institutions that are just starting on starting out and can be used to benchmark initial steps. The NDSA levels are used extensively in the Handbook (see Getting started, Fixity and checksums, Information security, and Storage). Risk assessment frameworks and tools can also contribute to audit assessments (see Risk and change management).

Which audit or assessment option should I choose?

The 2010 Memorandum of Understanding described above, effectively identifies a tiered approach to certification. The amount of effort required for each level increases, though so does the formality of the output. The choice of assessment framework for any given organisation should therefore take at least the following into consideration:

The choice of assessment framework should not be made independently and can often be directly influenced by the value that an assessment may have for other parts of the organization. Discussing the options with organizational peers and managers can be a useful first step in ensuring the right option is selected and support is secured from other areas of the organization from the outset.

How to run an audit or self-assessment

Once an appropriate methodology has been selected, a straightforward way to proceed is to develop the initiative as a project and produce a project plan. Advice on project planning is prolific online and you should consult this if your organization does not have an agreed process for project management. If it does have a process, then you should become familiar with it and plan your project using this methodology (or secure the assistance of a local project manager). Your plan should include at least the following sections:

• Scope: What content is in scope of the assessment?
• Timeframe: When will the assessment take place and when will it deliver results?
• Stakeholders: Who will deliver the assessment? Who else needs to be interviewed or consulted?
• Governance: Which group will have governance of the assessment and results?
• Communications: How will the process and outcome be communicated to stakeholders?
• Next steps: How will the results be implemented?

If you are running an ISO 16363 assessment you should consult the advice on the ISO 16363 Primary Trustworthy Digital Repository Authority Body website. The audit preparation page outlines the steps that should be taken when running a full audit and these can be adapted for a self-assessment. Similarly, the CoreTrustSeal website has an online self-assessment tool that will guide you through an assessment. PDF or HTML versions of the assessment manual guidelines are also available.

 Resources

APARSEN Report on Peer Review of Digital Repositories

http://www.alliancepermanentaccess.org/wp-content/uploads/sites/7/downloads/2014/06/APARSEN-REP-D33_1B-01-1_1_incURN.pdf

Lessons learnt to date from the process of repository certification have been usefully summarized by the APARSEN project in this report. It suggests although there has been considerable progress, arguably audit procedures are not yet fully bedded down and some issues remain for both auditors and repositories. (2012, 50 pages).

Digital Preservation Management tools: Principles

http://dpworkshop.org/workshops/management-tools/principles

For organizations that are committed to becoming a Trusted Digital Repositories (TDR), a formative step for developing a sustainable digital preservation and curation program is to adapt and adopt a set of standards-based principles as a foundation. The principles provide a frame for your program and adopting them is a positive (and hopefully easy) place to start.

Digital Preservation Management tools: Model document

http://dpworkshop.org/workshops/management-tools/policy-framework

Every Trusted Digital Repository needs to have a high-level policy document that explicitly states the scope, purpose, objectives, operating principles, and context of the organization's digital curation and preservation program. The DPM workshop team developed this model document to help organizations meet this objective. A model document identifies the recommended sections of a digital preservation policy framework with descriptions and examples for each section.

Digital Preservation Management tools: Self-assessment and peer review audit

http://dpworkshop.org/workshops/management-tools/self-assessment

TRAC (Trustworthy Repository Audit and Certification) Review tool developed for the DPM workshop.

The Open Archival Information System (OAIS) Reference Model: Introductory Guide (2nd Edition)

http://dx.doi.org/10.7207/twr14-02

This DPC Technology Watch Report from 2014 provides an accessible short guide to the OAIS standard. Terminology used in ISO 16363 is directly aligned with that of OAIS. The report will help provide a basic understanding of OAIS useful for understanding ISO 16363 and deliver an assessment against it.

Digital Preservation Capability Maturity Model (DPCMM)

http://www.securelyrooted.com/dpcmm/

DPCMM is a maturity model closely aligned with the OAIS  standard. It allows organisations to assess their digital preservation readiness across 5 levels for 15 components.

CoreTrustSeal

https://www.coretrustseal.org/

The Core Trust Seal is the first step in the global framework for repository certification. This repository assessment includes a 16 point checklist and can be used for self-assessment or peer review.

European Framework for Audit and Certification of Digital Repositories

http://www.trusteddigitalrepository.eu/

In 2010, the European Framework for Audit and Certification of Digital Repositories was established as a collaboration between the Data Seal of Approval (DSA) certification, the Repository Audit and Certification Working Group of the CCSDS, and the German Standards (DIN 31644) Working Group on Trustworthy Archives Certification. It aims to support an integrated framework for auditing and certifying digital repositories consisting of a sequence of three levels, in increasing trustworthiness.

Case studies

Preserving the H-Net Academic Electronic Mail Lists

http://files.archivists.org/pubs/CampusCaseStudies/Case11Final.pdf

Lisa M. Schmidt, Michigan State University, describes assessing the existing state of preservation for the H-Net e-mail lists using digital preservation theory and the Trusted Repositories Audit & Certification: Criteria and Checklist (TRAC) evaluation tool. Making recommendations and overseeing the implementation of improvements to make H-Net a trusted digital repository. Ensuring authenticity is the primary preservation issue. 2009 (15 pages).

ADS and the Data Seal of Approval – case study for the DCC

http://www.dcc.ac.uk/resources/case-studies/ads-dsa

Archaeology Data Service colleagues Jenny Mitcham and Catherine Hardman describe the ADS experience in applying for the Data Seal of Approval (DSA). They identify practical information about the DSA application process. They also outline issues ADS faced in undertaking the process and the potential benefits they envisage from DSA self-certification. 2011.

Self-assessment of the Digital Repository at the State and University Library, Denmark – a Case Study

https://ipres-conference.org/ipres14/sites/default/files/upload/iPres-Proceedings-final.pdf

In this iPres 2014 paper, the authors describe the process and the benefits of performing an audit based on self-assessment and ISO 16363 for the digital repository of the State and University Library in Denmark. (p.272-279 of 385).

TRAC Audit: Lessons

http://blog.dshr.org/2014/08/trac-audit-lessons.html

This is the third in a series of blog posts by David Rosenthal about CRL's TRAC audit of the CLOCKSS Archive. Previous posts announced the release of the certification report, and recounted the audit process. This post look at the lessons CLOCKSS and others can learn from their experiences during the audit.

Trustworthiness: Self-assessment of an Institutional Repository against ISO 16363-2012

http://www.dlib.org/dlib/march15/houghton/03houghton.print.html

In 2013, Deakin University Library undertook a self-assessment against the ISO 16363 criteria. This experience culminated in the current report, which provides an appraisal of ISO 16363, the assessment process, and advice for others considering embarking on a similar venture.

Managing an ISO 16363 Self-Assessment: A How-To Guide

http://www.dcc.ac.uk/sites/default/files/documents/IDCC16/18_Managing_ISO16363.pdf

A short poster presented at the International Digital Curation Conference (IDCC) in 2016 by Maureen Pennock and Caylin Smith of the British Library.

References

CRL, 2007. Trustworthy Repositories Audit & Certification: Criteria and Checklist. Available: http://www.crl.edu/sites/default/files/d6/attachments/pages/trac_0.pdf

DIN, 2012, DIN 31644 Information and documentation – Criteria for Trusted Digital Repositories. Available: http://www.nabd.din.de/cmd?level=tpl-art-detailansicht&committeeid=54738855&artid=147058907&languageid=de&bcrumblevel=3&subcommitteeid=112656173

Houghton, B., 2015. Trustworthiness: Self-assessment of an institutional repository against ISO 16363-2012. DLib Magazine, 21(3/4). Available: http://www.dlib.org/dlib/march15/houghton/03houghton.html

ISO, 2011. ISO 16919:2011 - Space data and information transfer systems - Requirements for bodies providing audit and certification of candidate trustworthy digital repositories. Available: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=57950

ISO, 2012a. ISO 14721:2012 - Space Data and Information Transfer Systems – Open Archival Information System (OAIS) – Reference Model, 2nd edn. Geneva: International Organization for Standardization. Available: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=57284

ISO, 2012b. ISO 16363:2012 - Space data and information transfer systems – Audit and certification of trustworthy digital repositories. Geneva: International Organization for Standardization. Available: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=56510

NDSA, 2013. The NDSA Levels of Digital Preservation: An Explanation and Uses, version 1 2013. National Digital Stewardship Alliance. Available: http://www.digitalpreservation.gov/ndsa/working_groups/documents/NDSA_Levels_Archiving_2013.pdf

nestor Certification Working Group, 2013. Explanatory notes on the nestor Seal for Trustworthy Digital Archives, nestor Materials 17, July 2013. Available: http://files.dnb.de/nestor/materialien/nestor_mat_17_eng.pdf

RLG/OCLC Working Group on Digital Archive Attributes, 2002. Trusted digital repositories: Attributes and responsibilities, Mountain View, California. Available: http://www.oclc.org/content/dam/research/activities/trustedrep/repositories.pdf

RLG-NARA Task Force on Digital Repository Certification, 2005. An audit checklist for the certification of trusted digital repositories, Mountain View. Available: https://web.archive.org/web/20051126181100/http://www.rlg.org/en/pdfs/rlgnara-repositorieschecklist.pdf

Save

Save