Bit List 2023: If digital preservation is possible then data loss is a choice

William Kilbride is Executive Director of the Digital Preservation Coalition

Digits are born vulnerable.

Every single byte of data depends on a global infrastructure of technology, process and people for its meaning and purpose to be realized. Much data serves the moment: it is quickly forgotten in a continuous flow of process and interaction. Other data serve lengthier purposes, as evidence and outputs of transactions that have significant impacts and long duration, longer than the infrastructure and the institutions through which the data was created. Everything in the latter category falls into the scope of digital preservation—the series of managed activities necessary to ensure continued access to digital materials for as long as necessary, beyond the limits of media degradation, technical obsolescence, and organizational change.

The Bit List is not a paper exercise. It was originally conceived as a call to action based on the insight and authentic voice of the global digital preservation community, and it remains so.

The fourth edition of the Bit List  which was released today, can be summarized in five broad conclusions:

1. The reporting and reviewing process has been comprehensively overhauled, in line with the biennial review. In particular, the inauguration of the Bit List Council has involved many more partners and domain experts in assessment. The simple fact of coordinating such a large group means more effort has been expended. The addition of many new experts means not only that the conclusions are ever more authoritative, but that complacency and groupthink have been challenged. It is by some way the most robust and measured statement yet of the risks facing our digital legacy.

2. The Bit List is more robust than ever, but it is fluid and not intended as the final word. The introduction of a new entry for ‘First Nations Secret/Sacred Cultural Material is welcome because it demonstrates a more global perspective. More importantly it is not a new issue. There have been three previous editions of the Bit List and six reviews in total: it has taken seven years to make explicit acknowledgement of the digital preservation challenges that arise from indigenous data sovereignty. This has two implications. The community-based methodology which generated the list means that some themes are amplified while others are obscured, especially those of the global majority in the Southern Hemisphere. So there needs to be some acknowledgement that gaps are left. Moreover, simply because something has not been added to the list does not mean it is not at risk.

3. Digital preservation is achievable. Two entries have moved to lower risk classifications. The re-classifications of ‘Published Research Data Appended to Journal Articles’ and ‘Unpublished Research Data’ are made for sound reasons based on materials improvements. In these and the small number of improving trends the Council has been able to identify the impact of policy clarifications, and those places where effort and expertise which have been applied. If digital preservation is possible then data loss is a choice.

4. But these are very few examples against a very the extensive list. The most noticeable conclusion is how little has changed. The expanded Bit List Council was invited to start over, with a wide range of new voices and subject matter expertise. Even so, the Council has made only marginal changes from recommendations in 2021. In this sense, 2023 has validated the broad conclusions of previous years, updating them rather than setting them aside. With a few honourable exceptions, there has been little or no improvement in the overall risk profile of digital assets.

5. The findings of the Third Edition of the Bit List in 2021 were largely valid, but its conclusions and recommendations have been largely ignored.

This much can be summarized from the report. The conclusions sit in a context, accurately described by the organizers of iPRES 2023 as ‘Disruptive Times’. Harper-Collins introduced the word ‘permacrisis’ as word of the year in the 2022 edition of their English Dictionary: the persistent sense of instability and insecurity arising from a continuing series of catastrophic events.

The Bit List 2023 is published in the shadow of a global pandemic, during a land war in Europe and a time of heightened tension and possible war in the Middle East. These are threats to digital content and coincide with a crisis of knowledge and fog of disinformation. Cyber-warfare can make battlefields and hostages of almost any connected device and data. Technical inter-dependency means that economic shocks threaten the digital memory of the world in ways we have barely begun to comprehend.

This is the year in which Twitter and its communities have effectively been destroyed. This is the year in which an American president was indicted for corruptly concealing documents and corruptly altering, destroying, mutilating, or concealing a documents and records. This is the year in which a former UK Prime Minister forgot the passcode to his phone and the messages that were stored there; and a serving UK Prime Minister defied a High Court ruling to disclose messages to a public enquiry. This is the year in which the Prime Minster of Cambodia deleted his Facebook Account, including live-streamed speeches threatening violence against his opponents. This is the year in which the bombing of a hospital in Gaza, apparently misreported, prevented a summit of Arab leaders with the US President in a rapidly escalating conflict.

We cannot afford to be complacent about the loss of bits and bytes. The preservation of authentic digital materials cannot be taken for granted.

Evidence matters. Justice, transparency, accountability, community, creativity and knowledge are at stake. Digital preservation has never been more important.

Recommendations

It seems apt to begin with an update and restatement of recommendations for action made in the last Bit List 2022 report:

“The DPC calls on our members, partners and colleagues globally to take four steps:

• Enable and support the preservation of digital materials emerging as a form of protest in the context of political upheaval, and to do so urgently as the threats to the loss of such materials are pernicious and immediate.

• To address the gaps in treaty provision that would enable the preservation of digital cultural heritage at a time of conflict, such as envisaged in the 1954 UNESCO Convention on the Protection of Cultural Property in the Event of Armed Conflict and related. Recognizing that cyberwarfare can make a battlefield of any connected device, digital resources are inevitably and already enmeshed in numerous conflicts, hot and cold. Digital cultural heritage should be granted special legal protection from electronic warfare such that attacks on the memory of the world may be treated not simply as cybercrimes, but in relevant conditions also as war crimes and in cases of cultural genocide as crimes against humanity.

• To remove and reduce barriers to the preservation of social media, enabling reasonable preservation actions by trusted and legitimate actors.

• Continue and expand the research and provision of digital preservation capability.”

This year, we further call on auditors, regulators, legislators to formulate plans that:

• Demand a higher standard of competence and attention to digital preservation in the context of regulated industries and public authorities to prevent data loss, recognizing the reputational and real harms to stakeholders, to themselves and to future generations that arise, and which are entirely avoidable.

• Ensure higher standards of data management are achieved through the introduction of systematic benchmarking of the digital preservation competence of regulated and statutory agencies that rely on data.

• Challenge short-termism within the sectors and agencies they regulate so that minimum retention periods are not misinterpreted as maximum, and ensuring value can be obtained from data gathered at great expense.

We call on data controllers, chief technology officers and corporate audit committees to:

• Use the Bit List to understand the risks they face, taking steps to remove, reduce or mitigate them.

• Fold digital preservation policies into cyber-resilience strategies.

• Recognize that long term commitments cannot be met solely on a project basis, and therefore to fold short term and exploratory digital preservation projects into longer term strategic plans.

• Prioritize the deployment technologies, the development of policy and the employment of staff to address digital preservation risks and the harms that result from data loss.

• Benchmark their digital preservation maturity as an organization and ensure that the capacity of staff and paid for services match their stated goals and aspirations.

• Assess and adopt new technologies based on the total cost of ownership, recognizing that longer term solutions may have larger up-front costs but will save money in the long term, avoid the accrual of technical debt and enable longer term exploitation of data and services.

We call on courts and law enforcement to:

• Use the full extent of the law to prosecute data losses that arises from criminal negligence or malfeasance, especially those deletions that expose professional misconduct or defeat public accountability.

We call on the global digital preservation community, especially our members, supporters and partners to:

• Help promulgate the Bit List, and report ways in which it has been used to advocate for and prioritize digital preservation within their institutions.

• Advise about omissions and updates from the Bit List to enable a continuous and informed cycle of revision to help maintain its currency and authority.

• Use the Bit List as a tool to roadmap the development of tools and solutions that will tackle some of the more intractable technical challenges reported here.

• Use the Bit List to guide and support students and new entrants into the digital preservation workforce so that they are suitably prepared for the material challenges of their chosen profession.