Authors: Lynn Bruce, Web Archivist; Laura Gould, Government Records Archivist; Barbara Fuentes, Web Continuity Assistant; Joanne Watson, Digital Capacity Planning Lead & Garth Stewart, Head of Digital Records Unit from the National Records of Scotland.
Introduction
In a year like no other, so many aspects of our society have changed in the wake of Covid-19; how we work, how we play, how we govern, how we live.
At the pandemic’s onset, National Records of Scotland - as keeper of Scotland’s national archive, set out to ensure that we captured a record of the unprecedented changes facing Scotland’s government and public health administrations. This required bundles of energy, focus, and engagement by colleagues, which at times significantly tested our resilience. We’ve adapted admirably to this challenge: read on to explore our key successes and thoughts on the future.
Part 1 – understand your depositors, record their needs
The early months of lockdown were particularly busy for NRS Archive Depositor Liaison: our archive service ‘front-end’ for depositors. We focused our energies on understanding the effect which Covid-19 was having on recordkeeping across Scotland, and how this could influence the shape and format of the future national archive.
To piece this together, we supplemented our usual depositor engagement with additional conversations, often via Skype for the first time. In addition, our colleagues in the NRS Public Records (Scotland) Act team carried out a national survey of the recordkeeping sector across Scotland, which gave us a broad-based understanding of these changes.
This engagement showed how some depositors had to provide brand new functions; alter their services, and innovate their ways of working: all of which led to the creation of new types of records, mostly born digital. We also learned that websites and social media played a key role in conveying decisions and service changes, and our Government Records Collecting Policy was updated to explicitly include COVID-19 records, as we strive to ensure that Scotland’s response to the pandemic is preserved.
In order to capture this intelligence systematically – be it tacit, or new, knowledge – we also devised an information collation tool: our ‘Maxi-client list’. The Maxi-client list enables us to record depositor information in a highly structured way e.g. key contact information, terms of deposit, format of archival records etc. With well over 200 depositors to manage, we had long aimed to create this type of resource, but the shift to remote working added impetus. The Maxi-client list is now used by all colleagues working with depositors, and is already supporting our future depositor engagement and archive capacity planning.
Part 2 – our collecting responses
The Web Continuity Service, established in 2017 and staffed by the NRS Digital Records Unit, provides NRS with the means to archive snapshots of our depositors’ websites. From the beginning of Covid-19 - and supported by our findings above - we realised that web archiving would be fundamental to capturing a record of the pandemic’s impact. Following an internal business case, and additional funding, since May, we’ve significantly increased our service’s operations – see our graphic below.
Higher crawl figures are one thing, but we’ve supported this with enriched levels of collaboration with all of the Service’s stakeholders. Colleagues from across NRS – in particular, Archive Depositor Liaison - have been key partners in Covid web archiving appraisal and selection, ensuring a joined-up approach.
As we acquire permission for web archiving ahead of crawl, and as many websites which we selected for archiving as part of our Covid web archive were new to us, we needed to significantly upscale our engagement with depositors. We also took the opportunity to streamline our permissions process, so that all Covid-focused websites could be captured and preserved quickly.
We’ve benefitted from working closely with our commercial supplier, Internet Archive, to anticipate and resolve technical challenges, and the greater volume of crawls has certainly expanded our technical trouble-shooting know-how!
Our joined-up approach has strengthened our relationships with depositors. We’ve worked hard to make our web archiving end-to-end process very swift: archived content is normally available online within a few days or weeks from point of crawl. Due to this, our depositors have been able to see the results of our service very quickly. This has powerfully elucidated our service’s value during a fast-paced, frenetic period, and has provided opportunities for depositors to incorporate archive content into their websites’ business-as-usual procedures.
Part 3 – Sustaining the service
The significant increase in our web archiving due to Covid-19 put pressure on our service operations. Our key challenge was quality assurance (QA) - how were we to QA so many more sites and so many more crawls? We place great emphasis on QA, but it is still very manually-orientated (read time-consuming!). We simply didn’t have enough capacity to apply our business-as-usual levels to our Covid crawls.
Working from home provided the opportunity for us to assign some of our wonderful colleagues to help with this task. Perhaps unusually for web archives, these colleagues are Archive Attendants, whose pre-Covid roles centred on paper records e.g. van transportation, document retrieval, acid-free box manufacture etc.
With our new workforce established, we faced Challenge Number Two; How to provide training? And how to do this remotely via Skype?! To streamline this, we produced and shared a guidance document ahead of training – this included a pared-down, more sustainable QA process. Despite the perils of web archiving technical jargon, the training went well - particularly as it was refined as we went along - and colleagues applied their existing collections’ knowledge to web archiving during sessions.
Our end result? A new ‘army’ of Web Archiving QA Experts, a scalable, sustainable QA process, and the development of new digital skills by colleagues. J
Figure 1: snapshot of Ready Scotland in the NRS Web Archive, taken 24 September 2020, previous version of website at https://webarchive.nrscotland.gov.uk/20200904190448/https://www.readyscotland.org/
Part 4 – A new service: Twitter archiving
NRS have long recognised that social media accounts are a public ‘face’ of our depositor organisations, and therefore an element of the archival record. However, resourcing, technical challenge, and information sensitivity risks had hitherto prevented us from collecting this format.
Covid-19 made us reconsider our approach, and provided the impetus to identify a solution that would allow us to safely and efficiently capture content from selected Twitter accounts. To that end, we collaborated with Internet Archive on the launch of a Twitter-archiving service, built on Social Feed Manager technology. It’s enabling us to capture a missing piece of the national public record, and we’re really excited about how this nascent service will develop.
Conclusion
NRS is now capturing a record of Covid-19’s impact on Scotland’s government and public health administrations. Our expanded web archive service is capturing the online record of this crisis as it happens, whilst our on-going engagement with depositors will ensure, in due course, that the many other aspects of the national record will be made available for future generations.
Working from home has forced us to develop a robust, resilient, coherent and broadly-based series of services. We’ve established clearer communication channels between teams across NRS, such as Procurement and IT, and with our wider stakeholders. We feel, despite the adversity of Covid-19, we are well placed to plan the next phase in our digital archive’s development. We’re pleased to reflect on progress on this very special day, happy #WDPD!