As realities erode: 3d4ever?

My eroding realities

Alternative facts are the cliché du jour but let me pitch a problem that is categorically larger: alternative realities. Not just alternative, but alternative and obsolete realities. I spoke about this at a DPC briefing day at the end of 2016 – which already seems a life time ago. It was a surprisingly hard programme to assemble because there is precious little evidence that those involved in producing 3d data sets for the cultural heritage sector have any capacity – and in many cases no evident concern – to ensure the accessibility of their virtual realities outside of tightly constrained and poorly documented delivery mechanisms. Therefore, in relatively short order and despite much rhetoric to the contrary, interactions degrade, effort is wasted and new kinds of cultural disenfranchisement are engineered. 3d data faces – it already has – an endemic crisis of obsolescence, resource discovery and corporate abandonment.

This blog is a cry for help. I want to be wrong. I want to be corrected and contradicted on the assumption that there must be a better story. Some clever and generous person is going to collate and deploy the evidence to show how the 3d data community has a concern and the capacity to ensure a longer-term viability to their virtual realities. They will gently take the megaphone off me and reassure me that the tremendous opportunities associated with 3d data are being met with a commensurate capability to preserve them.

The next big thing?

3d scanning - and 3d printing - has been the ‘next big thing’ for a long time. With origins in the 1960s it is an established technology and has enjoyed a prominent place in the imagination: but has remained been a rather niche concern in practice. But as scanning equipment become cheaper and more user-friendly, and as ubiquitous processing power improves, so the barriers to creating, sharing and accessing highly detailed 3d models are eroding. Simultaneously a price crash in personal immersive devices means that objects can finally be distributed with confidence, while 3d printing is beginning to look like a seriously disruptive technology within any number of industrial and commercial sectors. It may have taken fifty years, but in the last few months we have witnessed the first proper scandal of appropriated 3d data when a team claimed to have surreptitiously scanned the bust of queen Nefertiti at the Neues Meseum in Berlin. 3d visualisation companies are being purchased by global tech firms eager to add 3d capabilities to their stable. The value and utility of the 3d scanning is going up as the cost and barriers to production are dropping.   It’s hard to avoid the conclusion that 3d scanning is, finally, the next big thing.

As DPC regulars know, digitisation creates, by default, a digital preservation challenge. We have argued for almost 2 decades now over formats for capture, preservation and dissemination. Complex metadata requirements have been specified and embedded into workflows; preservation modules have been integrated into image management systems; compression techniques have been devised, demonstrated and denounced. Perhaps the long heritage of analogue photography means we implicitly value of 2d images more; perhaps the rapid evolution of 3d scanning means we haven’t taken time to establish its worth.

It’s not as if no one has been voicing concern about the long-term sustainability and preservation of 3d data. In 1996, the wonderful and sadly-missed Nick Ryan challenged the lack of transparency as a problem in the generation and use of 3d models in archaeology. In 2002 the ADS published a ‘Guide to Good Practice’ for creating Virtual Reality. Principles 5 and 6 of the 2006 ‘London Charter’ encouraged public agencies developing 3d visualisations to act as good stewards of the public investment by ensuring sustainability. Have these warning been overlooked?

What, if anything, do we have to show from the last 50 years of 3d scanning?; and what do we need to do to ensure that the next 50 years is better?

An unlikely event

As director of the Digital Preservation Coalition I enjoy an unusual perspective on digital preservation. It is no longer really my role to offer specific technical insights or provide training: rather more to facilitate the exchange of expertise; to track, translate and amplify; and to encourage and improve new approaches. I quite enjoy my role of making the coffee and serving the sandwiches because it affords me the privilege of sharing my days with the brightest and best. So, it is unusual for me to speak directly to DPC audiences other than as compere and host. And on this occasion, I feel compelled to speak.

There is no shortage of interesting work in 3d data creation and distribution – from many different sectors with many different approaches. But in preparing the workshop it was striking how hard it was to put together a programme on preservation and re-use of data after the initial creation. It’s all the more surprising since most of the agencies commissioning 3d data do understand and frequently articulate notions about the long-term value of data. There is no shortage of fine words in documents like the London Charter which mentions preservation in 2 of its 6 principles. It is a very dynamic and creative field which is relatively well resourced, enjoys a high - perhaps disproportionate – public profile, and attracts some highly talented developers and organisations.

These commitments to preservation the dynamism and the undoubted capabilities do not seem to have so far offered much in the way of practical digital preservation know-how. The 3d data community seems not to have learned much from the wider digital preservation community. Nor, for that matter has the digital preservation community, which is noted for its habit of borrowing good ideas, been able to scavenge ideas or approaches from the 3d data community. Perhaps this is simply a coincidence. Perhaps the second Friday of December (when we hosted the workshop) was the day that the world’s 3d data preservation community was otherwise engaged. Perhaps there is no long-term use case for 3d data? Or perhaps there is a disconnection between the 3d data community and the digital preservation community: and perhaps that’s to our mutual disadvantage.

Say it once, say if often: digitization is not digital preservation

It doesn’t help that language is sometimes misapplied or muddled. Digital preservation is a relatively challenging field concerned with the preservation of digital content. It means ‘the series of managed activities necessary to ensure continued access to digital materials for as long as necessary.’ It refers broadly to ‘all of the actions required to maintain access to digital materials beyond the limits of media failure or technological and organisational change.’ This is definitively not laser scanning, or computed tomography or photogrammetry or ground penetrating radar, or LIDAR, or side-sweeping sonar, or ultrasound or digital x-ray, or any other of the techniques used to render explore or generate 3d images of real world objects and places. These are interesting and worthy things: but digital preservation they are not. This misappropriation is confusing and sometimes contrived. Digitization, can sometimes be described as ‘preservation by record’. It creates an implied digital preservation requirement, especially when the real-world objects in question have since been lost, altered or destroyed.

Nor for that matter is digital preservation uniquely about data: it encompasses tools, paradata, metadata and all the applications and technology stacks necessary to ensure a faithful, authentic and credible interaction with data. It’s about much more than just data storage, and for the avoidance of doubt, it doesn’t mean we should try to save everything. On the contrary we should be encouraged to dispose of the things we don’t need and prioritise the parts that matter.  

It is not unusual to find different and at times competing expectations and opportunities with data: and it is therefore not necessarily clear what the long-term use case for data will be. In some cases, 3d scanning is presented as a novel means of access to collections, sometimes for research and sometime to monitor the impact of erosion or other natural processes. In other cases, the drive to 3d scanning is for the production and sale of replicas, and there is a high-profile and genuine effort to deploy 3d scanning techniques to create point-in-time records of cultural objects that face destruction.

These different use cases mean there will not be a single digital preservation solution: a 3d representation may be good enough for sales but useless for research and vice versa. But that doesn’t mean there is no long-term requirement. Instead it means the 3d data community could be leading a debate on which techniques work best for different requirements and how preservation actions might be optimised to multiple use cases. That’s a debate the digital preservation community badly needs. So both communities are missing an important opportunity.

A preservation voice in the technological desert?

Honestly, if I get reincarnated and if I get any choice in the matter, I am going to ask to come back as an old testament prophet. Maybe John the Baptist. The DPC has given me plenty of practice.

Why does the digital preservation community find it hard to engage with the 3d data creation community? Perhaps because there is no such thing: in describing a 3d data creation community I am reducing a dynamic and diverse set of approaches and agencies involved in different parts of a workflows which are themselves quite distinct. It’s also a fast-changing topic which is technically complex and produces large quantities of data. It’s no wonder that the digital preservation community finds it hard to engage, hard to pin anything down, and hard to resolve the manifest complexity into practical advice.

These different barriers to dialogue are not imagined, but nor are they insurmountable. Other sectors with large volumes of data – larger in many cases than those produced by 3d technologies – seem quite capable of engaging coherently with their digital preservation challenges. The audio-visual preservation community for example has a sizeable digital preservation challenge but also a sizeable digital preservation literature. Similarly, there is probably no digital theme more dynamic, commercially sensitive or diverse than the aggregation and interrogation of social media: and yet we manage a meaningful and at times highly productive discussion. Digital preservation can bring incredibly detailed expertise to bear and has used this to learn from and contribute to most of the sectors orthogonal to 3d data generation such as AV, CAD, geo-sciences and every aspect of data visualisation. This doesn’t mean to imply that in each – indeed in any case – the digital preservation challenges have been resolved. But there is at least some elementary understanding of the issue and practical know how that gives confidence, as evidenced in publications like the DPC Technology Watch Report series. It is hard to believe that the 3d data generation community is so exceptional or so exotic or so fiercely competitive as to be able to ignore the pressing challenge of preservation, a challenge of its own making.

It’s because I care

It may seem that I am throwing cold water at 3d digitisation with snide, 2d recalcitrance. That could not be farther from the truth. In fact this article was drafted precisely because I believe the opportunities and impending impact of 3d data are very real. 3d data and associated technologies really will be disruptive and transformative: anyone who doubts this should consider what might be termed a 3-fold prize crash in 3d data. Laser printing remains expensive as a consumer technology, but it is within reach of small businesses and geeks or devotees in the way that PC’s were in the mid 1980’s. If I were in the auto-motive parts industry, I’d be selling up. High quality 3d headsets, mostly thanks to the gaming industry, are now a real presence Christmas lists and the infrastructure to deliver them over the internet is widespread overcoming the decades of extrinsic rubbishness which we suffered since the 1980’s. And scanning technology is as ubiquitous as the smart phone. So 3d, which had been touted as the ‘next big thing’ for decades, is now at last the next big thing.

A problem and a proposal

The cultural heritage sector will be welcomed to this future with open arms. It has remarkable treasures to share and remarkable opportunities to exploit. But if it’s going to be a meaningful and productive interaction which brings value back to the memory institutions that look after the real world from which the 3d is derived, then those institutions need to think about the long-term implication of this change. And that starts with some relevant and credible thinking about digital preservation: there needs to be much more robust and straightforward questioning about the long term viability of the digital materials that are generated. More generally if we have a vision of disruptive technologies evolving structural changes in the economy, but have not discussed how those technologies will sustain their own outputs, then we risk envisioning an economy built on sand.   3d data faces – in fact it already has – an endemic crisis of obsolescence, resource discovery and corporate abandonment.

It was exactly this fear of obsolescence that brought the DPC into existence and it is a mature platform and partner with which those generating 3d data can progress the necessary discussion. There are two possible responses. Either my description of the problem and the lack of solution is entirely wrong: in which case lets assemble the necessary evidence and facilitate a better and more effective discussion around the lessons learned. Or my description is broadly accurate: in which case we need to prioritise some actions.

Both conclusions in the same direction: it’s time to develop a roadmap for the development of digital preservation solutions for 3d outputs and how they can be incorporated early into object or project lifecycles. Let’s also assess the current crop of 3d projects and select a few that would we like to be able to use in 10 years. Let’s experiment on how to preserve them and refine those techniques not just by talking but by doing. And in this work, let the DPC be a partner, guide and forum for debate introducing leading edge thinking about digital preservation to the 3d community; and helping the digital preservation community learn from the experience of 3d.

Two messages

In planning this workshop on how to preserve 3d data, two key messages emerged for those who generate 3d models and the many companies, developers and engineers that support them.

Firstly, it’s challenging enough for the digital preservation community to deal with the massive quantities of data that are coming our way – even the ones which are well known and clearly within scope for archives and libraries. These quantities of data are only going to increase in size, complexity and importance and the task we face becomes ever more challenging. So something will be left behind. Although the digital preservation community would like to ensure 3d data can be managed through the challenges of media failure, technical obsolescence or organisational change, don’t expect archivists and records managers to sort out a data mess that is exotic in form and liminal to institutional missions.

Secondly, and perhaps more fundamentally, it behoves the 3d data community to show the world that it values its own outputs. If those creating 3d models and the tools to construct them don’t take them seriously, and don’t think they are of long term value, then it’s going to be hard to persuade archivists and others to take them seriously too.