Annie Stille is a Metadata Specialist at Sydarkivera in Sweden
Illustrations from Magnus Heimonen, Web editor and media producer, Sydarkivera
The challenges of web archiving in a rapidly changing world at Kommunalförbundet Sydarkivera described with focus on the digits. Sydarkivera is a municipality organisation founded in the south of Sweden in 2015. Preserving information as an archival authority organisation in times of change is a great challenge. Some of these challenges will be presented in this blog post, written by Annie Stille, metadata specialist and responsible for web archiving at Kommunalförbundet Sydarkivera.
Sydarkivera is a provider of long-term digital preservation services. It acts as the archival authority for several municipalities in southern and central Sweden who are members of Sydarkivera. The purpose of the organisation is to preserve the member’s information flows for future generations.
In today’s society, information flows are rapidly changing and increasing in volume. This puts the onus on actors dealing with digital preservation to be prepared and outward-looking. This is, however, challenging because it is not always possible to predict how the information will be used in the long term. This places considerable technical demands on organisations working with digital preservation in terms of potential formats, technical solutions, metadata, storage and data management. Alongside these technical demands are changes to attitudes towards new situations that require quick action from a digital preservation perspective. Doing nothing or acting too slowly risks information loss.
For this year’s “Digits: For Good” theme I will consider the web archiving of Sydarkivera’s member’s websites. This is an area that faces many challenges. For our members, 2020 has brought considerable changes in working methods because of COVID-19, so their websites have evolved much more rapidly than usual.
Sydarkivera has had to adapt quickly to create a different working process and increase the number of web crawling sessions. In this process, we have changed our working methods and routines, extend our knowledge and resources and improved our flexibility. Consequently, in 2021, Sydarkivera will increase the number of website crawls from one to four per year. Tests to incrementally harvest smaller fractions of the websites have been ongoing since mid-2020. These have focused primarily on showing changes to start pages once per month. Sydarkivera also intends to investigate the preservation of social media on a more frequent basis. The benefit of these changes is that the member municipalities do not have to worry about preserving their websites on their own; they can, if they wish, delegate all responsibility for website preservation to Sydarkivera.
Since 2016, Sydarkivera has preserved its member’s websites once a year. Until now, this annual schedule has fulfilled the mission of website preservation. But, there have been indications that this is no longer enough, and we have foreseen that the interval needs to increase. The frequency of website preservation is dependent on how often a website is updated; if the information on the website is considered to be worth preserving and whether the organisation behind the website has any special requirements.
It is apparent that the 2020 COVID-19 pandemic has led to fundamental changes in society; with implications for the way public authority is executed, and research directions. Now, more than ever, archival organisations need to be agile so that they can spot changing needs and react quickly. Analysis of the local, national and international contexts is critical to stay up-to-date.
In addition, the work procedures developed need to be well thought through. There is little use in preserving information that will benefit future generations. It is therefore essential to carefully choose technical solutions and metadata labels that are sustainable in the long term. Of course, the challenge is that it is impossible to know precisely how the information will be used in the future, which technical solutions will be available or which working methods which will apply. At present, we can only guess and preserve as much as possible to create the best opportunities for the long-term information supply. But it is not just technical solutions that are important. The attitude within the organisation is critical. Today, organisations need to be open and attentive, with the ability to spot existing needs and predict long-term requirements and have the flexibility to act quickly when required. By combining technological insight with the right attitude, we can hopefully meet future challenges in a way that benefits future users of our data.