Cloud Storage

   Vulnerable small

Materials routinely copied or backed up to an independently managed, off site data storage facility and able to be restored under contractual terms

Group: Storage

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: New Entry

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors

Effort to Preserve

It would require a small effort to preserve materials in this group, requiring the application of proven tools and techniques.

Examples

Remote network storage provided by a third-party service under contract, such as DropBox, Amazon, Microsoft Azure, Dell EMC, Google Cloud Platform, Google Drive, IBM, Intel, Rackspace, Iron Mountain, SAP and others

‘Endangered’ in the Presence of Aggravating Conditions

Encryption; lack of routine maintenance; lack of storage replication; over-dependence on single supplier; insufficient documentation; lack of local alternative; political or commercial instability; overly aggressive compression; poor information security; lack of integrity-checking; lack of strategic investment; lack of migration plan; lack of exit strategy; unenforceable penalties; unstable pricing; unpredictable removal costs

‘Lower Risk’ in the Presence of Good Practice

Backup to different technology; backup to diverse locations; documentation of assets; integrity checking; preservation planning; export functionality; resilient to hacking; version control; resilient funding; technology watch; enforceable contract; disaster planning and documentation, stable pricing; budgeted removal costs.

2019 Review

The judges introduced this entry to ensure that the range of media storage is properly assessed and presented. 

Additional Jury Comments

The history of digital preservation suggests that the risk of vendors going out of business or shutting down services is the key issue here, over and above any specific technical solutions or risks.

See also:

Digital Preservation Coalition (2016) ‘Storage’ in Digital Preservation Handbook (Second Edition), online at https://dpconline.org/handbook/organisational-activities/storage

Read More

Content on cloud video services produced by the service provider

   Vulnerable small

Video materials - films and television programs – that are produced by companies that maintain their own distribution platforms and are exclusively available through these platforms.

Group: Sound and Vision

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to preserve materials in this group, including the development of new tools or techniques.

Examples

Netflix, Amazon Prime

‘Endangered’ in the Presence of Aggravating Conditions

Lack of corporate preservation planning; lack of preservation capability; conflating backup with preservation; loss of original recordings; lack of preservation voice at executive level; poor planning and roadmap for infrastructure; slapdash procurement or migration to new systems; mergers and acquisitions; profusion of corporate systems; complex intellectual property rights; single point of failure; technical protection measures that inhibit reasonable preservation actions.

‘Lower Risk’ in the Presence of Good Practice

backup and documentation; use of open formats and open source software; data management planning; licencing that enables preservation; corporate preservation capability; resilient to hacking; authenticity and integrity managed; recognition of preservation functions at executive level; technology watch; preservation audits; participation in preservation community.

2019 Review

This is a new entry from the open nomination, representing collections that are highly significant in cultural and social terms. It was adopted as the Jury was unclear whether content could be played outside of the producers’ publication platform. That introduces technical dependencies between content and software which are amplified by rights management.

Additional Jury Comments

This entry has five aspects.  1. It falls outside of the scope of traditional regulatory frameworks and archiving has not yet been included in any legislative framework, unlike broadcast tv where there is a designated archive in most developed nations. 2. As a result, the collecting and preservation of the content from the online platforms is underdeveloped, and the content remains unavailable in public archives.  3. These risks are mitigated by the fact that the commercial archives are technologically advanced, with mature digital ecosystems and skills, and much of the content has a ‘long tail’ business model and as commercial products these have value, so preservation incentives are clear. 4. However these are often stored at scale on LTO taps and so specific issues arise with the obsolescence of LTO tape technologies for the broadcast sector.  5. Nonetheless, issues remain around archiving the rushes, and other relevant assets which may not be valued by the production company. 

Read More

Current Hard Disk Technologies

   Vulnerable small

Materials saved to storage devices with a variety of underlying magnetic or solid-state (flash) technologies that are hardwired into a computer still under warranty or supported: typically hard disks that are less than five years old.

Group: Storage

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: New Entry

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a small effort to preserve materials in this group, requiring the application of proven tools and techniques.

Examples

Direct Attached Storage (DAS) such as Magnetic or solid-state drives integrated into individual laptop or workstations and into smaller scale storage facilities.

‘Endangered’ in the Presence of Aggravating Conditions

Encryption; poor handling; poor storage; lack of consistent replication; failure of external (dependencies eg suppliers, security); political or commercial interference; failure of internal dependencies (e.g. power supply, disk controller); overly aggressive compression; poor information security; lack of integrity-checking; lack of strategic investment; lack of warranty; unenforceable warranty; encryption.

Lower Risk in the Presence of Good Practice

Backup to different technology; backup to diverse locations; documentation of assets; integrity checking; preservation planning; refreshment planning; export functionality; resilient to hacking; selection and appraisal criteria; version control; resilient funding; technology watch; enforceable warranty; disaster planning.

2019 Review

The judges introduced this entry to ensure that the range of media storage is properly assessed and presented. 

Additional Jury Comments

See also:

Digital Preservation Coalition (2016) ‘Storage’ in Digital Preservation Handbook (Second Edition), online at https://dpconline.org/handbook/organisational-activities/storage

Read More

Digital recordings published via cloud-based music sharing platforms

 

 Vulnerable small

Music licensed and playable through corporate platforms protected by rights management and subscription revenues and presented as compressed single-track recordings.

Group: Sound and Vision

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: 2019

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to preserve materials in this group, including the development of new tools or techniques.

Examples

Spotify, iTunes, Bandcamp, SoundCloud

‘Endangered’ in the Presence of Aggravating Conditions

Lack of preservation capability within corporate systems; conflating backup with preservation; loss of original multi-track recordings; lack of preservation voice at executive level; poor planning and roadmap for corporate infrastructure; slapdash procurement or migration to new systems; mergers and acquisitions; profusion of corporate systems; complex intellectual property rights; single point of failure; technical protection measures that inhibit preservation actions; encryption.

‘Lower Risk’ in the Presence of Good Practice

Strong backup and documentation; use of open formats and open source software; data management planning for preservation; licencing that enables preservation; corporate preservation capability; resilient to hacking; authenticity and integrity managed; recognition of preservation functions at executive level; technology watch; regular preservation audits; accreditation and participation in professional preservation community.

2019 Review

The 2017 BitList included an entry for Digital Music Production and Sharing which included this entry.  In 2019 the Jury decided to split this initial nomination into four, recognising the different challenges faced.  This entry is particularly concerned with the music industry at scale and the services that connect the vast majority of artists to their audiences.  These are typically large and well-funded, and typically recognise the value of the content they publish.  But this is not without risks and it is perhaps surprising that the music industry does not yet have any equivalent to the non-print legal deposit regime that applies to other types of publication, including sheet music in some jurisdictions.

Additional Jury Comments

The preservation of recorded music is one of our generations most important jobs but it’s unclear where responsibility lies. There are commercial incentives to do so but there are also incentives to reduce costs. National collecting organizations may need to develop a role to address this. Whilst public archives are permitted to keep this material in some jurisdictions they typically do not have the resources to do so.  Consequently, there is an expectation that rights holders will maintain their own archival copies which they may not be able to do.

Read More

Geomagnetic Data and Software

   Vulnerable small

Data, software and supporting paradata relating to Earth's magnetic field and Earth-Sun environment, including current declination, geomagnetic field surveys and observations, field models and magnetic indices, and data from geomagnetic observatories.

Group: Research Outputs

Trend: New Entry

Unanimous Decision

Added to List: 2017

Last update: 2017

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a major effort to preserve materials in this group, including the development of new tools or techniques.

Examples

Outputs from geomagnetic scanning and measurement sensors both terrestrial and satellite-based, as well as supporting calibration and reference data and software to enable assembly and rectification to co-ordinate systems

‘Endangered’ in the Presence of Aggravating Conditions

Bespoke or undocumented software; undocumented data cleaning; disassociation with documenting, especially instrument calibration data; Lack of conformance or validation against standards; lack of preservation commitment or planning; no identified repository or lack of capability in repository; lack of storage replication; political or commercial interference; loss of version control; poorly managed intellectual property rights; poor metadata; loss of authenticity.

Lower Risk in the Presence of Good Practice

backup and documentation; use of open formats and open source software; data management planning before fieldwork; licencing that enables preservation; preservation capability in designated repository; resilient to hacking; selection and appraisal criteria; version control; authenticity; resilient funding and recognition of value; technology watch.

2019 Review

This entry was introduced in 2017 but was outside the competence of the judges to assess at that time.  It was scheduled to be assessed in 2019 and additional expertise was invited to the panel to support this assessment: but it was also independently introduced through the open nomination process again in 2019. It is presented as an example of research data, though noting the competences of agencies involved in preserving geo-magnetic data and software it has been classified accordingly.

Additional Jury Comments

Institutions that collect and manage geomagnetic data should strongly consider an assessment against the Core Trust Seal which now incorporates many of the characteristics previously measured by the World Data System.  Researchers intending to create data should examine data management planning tools; and researcher intending to deposit data should assess whether their chosen repository meets the Core Trust Seal Requirements.

Read More

Local Network Storage

 

 Vulnerable small

Materials routinely copied or backed up to locally managed data storage facility and able to be restored under institutional service arrangements

Group: Storage

Trend: New Entry

Consensus Decision

Added to List: 2019

Last update: New Entry

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a small effort to preserve materials in this group, requiring the application of proven tools and techniques.

Examples

Institutional or departmental network storage and institutional data centres based on technologies such as (NAS) Network Attached Storage, (RAID) Redundant Array of Independent Disks, (SAN) Storage Area Networks, JBOD (Just a bunch of disks), SPAN and related

‘Endangered’ in the Presence of Aggravating Conditions

Encryption; lack of routine maintenance; lack of storage replication; over-dependence on single supplier, technology or technician; insufficient documentation; single point of failure; political or commercial interference; failure of dependencies (eg power supply, controller software); overly aggressive compression; poor information security; lack of integrity-checking; lack of strategic investment; lack of warranty; unenforceable warranty, encryption.

Lower Risk in the Presence of Good Practice

Backup to different technology; backup to diverse locations; documentation of assets; integrity checking; preservation planning; refreshment planning; export functionality; resilient to hacking; selection and appraisal criteria; version control; resilient funding; technology watch; enforceable warranty; disaster planning and documentation.

2019 Review

The judges introduced this entry to ensure that the range of media storage is properly assessed and presented. 

Additional Jury Comments

See also:

Digital Preservation Coalition (2016) ‘Storage’ in Digital Preservation Handbook (Second Edition), online at https://dpconline.org/handbook/organisational-activities/storage

Read More

PDF/A

 

 Vulnerable small

Materials contained within PDF/A, a format derived from the PDF 1.4 variant but with additional requirements that reduce external dependencies. Includes more recent variants (PDF/A2 and PDF/A3)

Group: File Formats

Trend: New Entry

Consensus Decision

Added to List: 2017

Last update: 2017

Previous category: Of Concern

Imminence of Action

Action is recommended within five years, detailed assessment within three years

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a small effort to preserve materials in this group, requiring the application of proven tools and techniques.

Examples

Documents stored offline, or online in repositories or EDRMS including as reports, agenda, minutes, correspondence, contracts, essays, articles, or research papers.

‘Endangered’ in the Presence of Aggravating Conditions

Lack of conformance or validation; Lack of preservation commitment or planning; Lack of storage replication; Poor management of data protection or intellectual property rights; Political or commercial interference; loss of version control; use beyond the design capability of the standard; complacency; poor metadata; disassociation from context; loss of authenticity; encryption.

‘Lower Risk’ in the Presence of Good Practice

Backup and documentation of media assets; validation on creation; part of preservation plan; export functionality; licencing enables preservation; preservation capability in repository or EDRMS service; resilient to hacking; selection and appraisal criteria; version control; authenticity; resilient funding and recognition of value; technology watch

2019 Review

The judges have introduced this entry as a subset of a previous entry for ‘PDF’, emphasizing the different threats faced by the two main types of PDF.  PDF/A explicitly reduces dependencies and thus curtails preservation risks for certain types of content. This alone is not sufficient to ensure preservation and users are warned against complacency.  PDF/A has sometimes been misunderstood or misrepresented as a generic solution to all digital preservation requirements, whereas in the eyes of the judges it can only offer a preservation solution when embedded within a wider preservation infrastructure.

Additional Jury Comments

Vulnerability also depends on if the PDF file conforms to the specific PDF/A standard or not. This is caused by a combination of 1) not conforming to the standard and 2) collection managers assuming that the file is resilient simply because it purports to be a PDF/A.  This risk is less with format and more with the understanding and experience in data management. Moreover, materials embedded in orattached to PDF/A-2 and PDF/A-3 may be at risk.

See also:

·        Fanning, B (2017)  Preserving with PF/A (Second Edition), DPC Technology Watch Report 17-01 online at http://dx.doi.org/10.7207/twr17-01

Read More

Pension, Mortgage and Insurance Records

   Vulnerable small

Records of transactions for long-lived financial products and services contracted between individuals and corporations.  These records typically contain or depend on significant amounts of personal information and outlast the infrastructure on which they were created.

Group: Sensitive Data

Trend: New Entry

Consensus Decision

Added to List: 2017

Last update: 2017

Previous category: Of Concern

Imminence of Action

Action is recommended within three years, detailed assessment within one year.

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors

Effort to Preserve

It would require a small effort to preserve materials in this group, with the deployment of proven tools or techniques.

Examples

Applications, correspondence and ancillary records relating to pensions, mortgages and insurances and other contracts of long duration. This includes corporate databases, email, web archives and EDRMS and may require some co-ordination of paper, microfiche, born digital and digitized records.  Records will likely include the scope and duration of the contract as well as any agreed changes during the lifetime of the product.  It may also include evidence of misselling or other sharp practice which only becomes apparent after the fact.  This entry pertains to the corporate records rather than personal records. 

‘Endangered’ in the Presence of Aggravating Conditions

Lack of corporate preservation planning; lack of preservation within procurement of corporate systems; companies conflating backup with preservation; loss of integrity and authenticity; loss of context and connections to provide meaning; lack of preservation capability within agencies; lack of preservation voice at executive level; poor planning and roadmap for corporate infrastructure; proliferation of legacy systems; slapdash procurement or migration of new systems; mergers and acquisitions leading to confusion of corporate systems; lack of compliance, audit or accountability at operational levels; encryption.

‘Lower Risk’ in the Presence of Good Practice

backup and documentation; use of open formats and open source software; considered data management planning; licencing that enables preservation; preservation capability in designated repository; resilient to hacking; selection and appraisal in place; authenticity and integrity of records managed; resilient funding and recognition at executive level; technology watch; regular preservation audits; accreditation and participation in professional preservation community.

2019 Review

This entry was introduced in 2017 but was outside the competence of the judges to assess at that time.  It was assessed in 2019 with additional expertise invited to the panel to support this assessment.

Additional Jury Comments

This is a diverse category which may need to be sub-divided again. 

Read More

Published Research Papers

   Vulnerable small

Completed research papers published in serials, monographs or theses which fall under specific collecting policies of research libraries or archives and are managed through dedicated repository infrastructures.

Group: Research Outputs

Trend: Improving

Strong Consensus Decision

Added to List: 2017

Last update: 2017

Previous category: Endangered

Imminence of Action

Action is recommended within five years, detailed assessment within three years

Significance of Loss

The loss of tools, data or services within this group would impact on people and sectors around the world.

Effort to Preserve

It would require a small effort to preserve materials in this group, with the deployment of proven tools or techniques.

Examples

Scholarly E-Books, Electronic Journals, E-theses, Electronic Monographs.

‘Endangered’ in the Presence of Aggravating Conditions

Lack of documentation; lack of clarity with respect to intellectual property; embedded complex objects; unstable funding for repository; lack of strategic investment; complex external dependencies; lack of persistent identifiers; bespoke formats; lack of legal deposit mandate.

‘Lower Risk’ in the Presence of Good Practice

Strong documentation including intellectual property rights; clarity of preservation path and ensuing responsibilities; credible preservation plan; proven capacity of repository; legal deposit preservation copying; post-cancellation access service; persistent identifiers used consistently; non-proprietary formats used and validated; minimal or well managed external dependencies.

2019 Review

Published research outputs were introduced in 2017, though without reference to the capacity of the repository infrastructure.  The jury has amended this category slightly to presume the existence of repository infrastructure and noted that the aggravating conditions which introduce risks and good practice enhancements which reduce it are mostly relevant to repository operations.

Additional Jury Comments

See also:

·        Beagrie, N (2013) Preservation, Trust and Continuing Access for E-Journals, DPC Technology Watch Report 13-04, online at: http://dx.doi.org/10.7207/twr13-04

·        Morrissey, S and Kirchhoff, A (2014) Preserving E-Books, DPC Technology Watch Report 14-01, online at: http://dx.doi.org/10.7207/twr14-01

Read More

Recently Commissioned or Completed Media Art

   Vulnerable small

Media art currently displayed in a gallery or in the process of being displayed.

Group: Media Art

Trend: New Entry

Consensus Decision

Added to List: New Entry

Last update: New Entry

Previous category: New Entry

Imminence of Action

Action is recommended within three years, detailed assessment within one year

Significance of Loss

The loss of tools, data or services within this group would impact on many people and sectors.

Effort to Preserve

It would require a major effort to preserve materials in this group, including the development of new tools or techniques.

Examples

Media art recently acquired by galleries that utilizes specific hardware and software in order to be accessed or exhibited.

‘Endangered’ in the Presence of Aggravating Conditions

Lack of documentation to enable maintenance; lack of clarity with respect to intellectual property; complex interdependencies on specific hardware, software or operating systems; lack of capacity in the gallery or workshop; lack of strategic investment; complex external dependencies

Lower Risk in the Presence of Good Practice

Strong documentation; clarity of preservation path and ensuing responsibilities; proven preservation plan; capacity of workshop to support art work at de-installation; capacity of gallery to conserve after de-installation; capacity of gallery to re-install work

2019 Review

Media Art was introduced in 2017, though with particular reference to historical media art which was categorised as ‘critically endangered’. The Jury introduced this entry to ensure greater specificity in its recommendation.  It is intended to represent works commissioned in the last five years where there is a reasonable expectation that documentation has been produced or could still be obtained.

Additional Jury Comments

By the time digital art, time-based media etc. has entered into the 'permanent' care of a stewarding institution many of its technologies are already end-of-life, unsupported, or the hardware components have deteriorated. Often the expertise to maintain these many interacting components sits outside the host organisation, with a technical supplier to the gallery, and this is in itself vulnerable to business change. Although there are a few exceptions, there is need for greater capacity within the museum and gallery sector to address the challenges that arise.

Read More

Scroll to top