Disentangling Digital Preservation Risk with CHARM

CHARM represents the culmination of a four year PhD programme exploring the nature and complexity of digital preservation risk. Delivered as a practice-based PhD, the research sought to devise a holistic approach to digital preservation risk assessment that could integrate with enterprise-level risk management approaches. The resulting conceptual model and methods for implementation represent a significant new contribution to both our disciplinary knowledge of digital preservation risk and the tools with which it can be identified and assessed.

CHARM was developed using an innovative and creative blended methodology of risk science with design science. The requirements for the research were established through a review of tools in the existing solution space, balanced against specific needs of the British Library where the research was undertaken. The review explored several different assessment frameworks produced in the field over the past twenty-five years. It identified four main types of frameworks originating mainly from different projects or institutional settings, for assessment of file formats, collections, systems, or organizations. Comparative analysis indicated that whilst the frameworks often covered similar ground, particularly frameworks within a given type, they often understood and represented risk in very different ways. There was significant inconsistency in how each described different aspects of risk and in the measures used, suggesting a need for significant contextualization despite similar theoretical grounding. The wider literature also identified a degree of uncertainty about methodologies, particularly in relation to scoring and use of theoretical measures. Moreover, solutions typically addressed digital preservation in isolation rather than as an organizational endeavor, making them challenging to integrate into enterprise-level risk assessment management frameworks for review and consideration of digital preservation alongside more standard corporate risks.

The CHARM framework was developed to address these limitations and bring clarity into the nature and complexity of digital preservation risk with a clearly defined conceptual model and a practical framework for holistic and integrated digital preservation risk assessment. As a practice-based PhD it delivered both academic and practical outputs: the thesis represents an academic exploration into the subject and demonstrates the rigor of the research, whilst the practical outputs represent a series of usable artefacts derived from the research and produced for a wider, practitioner audience.

The practical outputs represent detailed solutions for three key types of design science artefacts: the construct (i.e. vocabulary), a series of models representing different aspects and layers of digital preservation risk, and a number of methods that demonstrate the utility of the models for risk identification and assessment. These are contained in the following documents:

• Conceptualising and Characterising Digital Preservation Risk: A Reference Model (CHARM)

• A How-To Guide for the CHARM Reference Model 

• The CHARM Risk Identification Framework (RIF)

• The CHARM Risk Assessment Spreadsheet (RAS)

The Reference Model was developed in a recursive design process that applied risk science techniques to the problem of digital preservation risk in order to develop a deep systematic and objective understanding of the problem. A signature feature of CHARM is its risk-science inspired distinction between characterized and conceptual risk, which led to the initial formation of a conceptual definition of digital preservation risk framed in terms of its undesirable outcomes (the potential for complete or partial loss of digital collection content), target values (authenticity, integrity, retrievability, accessibility, and longevity), and potential causes (sub-optimised risk sources within the managed organisational and technological environment). This not only clearly and meaningfully defines the concept of digital preservation risk but also provides the scope for the overall problem area and subsequent set of models. These consider for example different aspects of the risk concept, as well as target value concepts and their relationships and the most important significant conceptual feature of CHARM, that of the risk source.

The Risk Source concept is central to the CHARM model. The research developed an innovative generic and re-usable risk science model of the risk source that distinguishes between a number of related risk source entities: Risk Originating Entities, Risk Source Classes, Risk Source Instances and Instance Types, and Risk Factors. Collectively, these conceptual entities represent a re-usable schema through which to consistently and comprehensively express different aspects of the risk source concept across a given domain.

Figure 1: CHARM Risk Source Concept Model

Each of these entities is clearly and distinctly defined. They are then re-modelled into a re-usable structure for a domain-level risk source model, expressed in simple modelling notation (UML).

Figure 2: Abstraction of a Risk Originating Entity Family

This structural model is then implemented for three risk originating entities, each representing one of the conceptual high level risk source areas for the domain drawn from the definition of digital preservation risk: digital content, organizational infrastructure and technological infrastructure.

Figure 3: CHARM Risk Originating Entities and Risk Source Classes

Associated entities are identified and populated using knowledge of the domain drawn from over two decades of practitioner experience, with values tested and refined through use of the corresponding methods as described in the CHARM How-To Guide.

Figure 4: Populated example of the System Software class model

The How-To Guide demonstrates how the model can be used to inform scenario-based risk assessments. It outlines three suggested methods, two of which are focused primarily on risk identification whilst the third extends this to a full risk assessment process and characterization of risk through the formulation of risk statements. Templates for two methods are also provided. This third method is particularly compatible with enterprise-level risk management processes and this is explained in the How-To Guide, demonstrating how to apply additional concepts from the characterization process model and integrating additional and widely familiar risk management techniques such as risk matrices and risk appetites.

Model and methods were extensively and successfully tested as part of the formative design process, with summative evaluation thereafter against three sets of criteria: the CHARM requirements, the Design Science Artefact Criteria from March and Smith (1995) and the Design Science Guidelines from Hevner et al (2004). The evaluation confirmed the rigor of the approach, its utility and generality, the fidelity of the model with real world phenomena, its complete coverage, and a verifiable, innovative contribution to knowledge. All artefacts, including the thesis, have since been published on the University of Dundee institutional repository without embargo for circulation and engagement with the wider community.