This blog was written by Tara Majoor and Allison Graycon at Museums of History New South Wales
Museums of History NSW manages and preserves the State Archives Collection for the State of New South Wales, Australia, as well as place-based collections of objects and materials associated with our portfolio of historic houses and the Caroline Simpson Collection of publications, objects and materials related to homes, gardens and domestic life in Australia.
Need for automation Teams across Museums of History NSW (MHNSW) shared a vision of a more efficient digital ingest solution for the NSW State Archives Collection. Through an incremental approach and collaboration with industry partners a JIRA ingest tool has now been implemented to improve digital preservation.
MHNSW’s digital preservation system, Rosetta, is integrated with Axiell Collections and Primo, collectively forming the State Archives Management System (SAMS). Rosetta is a highly customisable system. When it was first implemented, several workflows were developed to ingest digital files using Rosetta’s OAI harvester, as well as a CSV method of linking ingested files to item records in Axiell Collections. This initial process involved several steps and required systems knowledge but served its purpose. But as ingest needs for born digital and digitised records grew over time, areas for improvement were identified, and the internal SAMS steering committee of staff from multiple teams advocated for greater automation.
Partnership work delivers Version 1.0 With support from NSW Treasury, a preservation-driven digitisation project began in early 2021 with $4.25 million funding, allowing the development of an ingest tool. We knew colleagues at the State Library of NSW had successfully implemented an ingest tool for Rosetta and were willing to share their experiences. The tool would need to not only process the hundreds of thousands of digital images (TIF, JPEG & PDFs) the project would create, but also provide capacity for future projects and business as usual – and have the potential to accommodation other digital file formats, including born digital State archives.
We established a working group of digitisation project team members, archivists, and technical staff, with an internal system developer leading development. Given previous success with JIRA software for other Axiell Collections applications, JIRA (on-premises) was selected as the prime software. The working group collaborated to set functionality requirements and outcomes, and development of JIRA and an API commenced. Following more joint work on user testing, Version 1.0 of the ingest tool was put into production. Media record CSV lists could now be attached to JIRA tickets instead of working within Rosetta’s OAI harvester – users didn’t even need a Rosetta login. The process removed several manual steps, including validation of CSV lists – bulk ingest of image file formats was now simplified.
Covid impacts on the digitisation project meant compressing ingest activities into a short timeframe and running the tool at maximum capacity – attempting processing of over 70,000 digital assets a week. We identified some opportunities to make the process more efficient and reduce errors, but the tool proved capable and ultimately enabled the successful ingest of over 900,000 digital assets created through the digitisation project. Version 1.0 was then re-purposed for further ingest of digitised records.
Wider interest and Version 2.0 The tool sparked interest from other areas of MHNSW, particularly the team working with NSW Government agencies on digital transfer – and dealing with increasing transfers of born digital State Archives and a growing need to accommodate audio-visual records. The teams collaborated to develop a successful business case for a scalable, more flexible Version 2.0 that could accommodate a wider range of file formats.
Industry partner Kororo was engaged to develop the second iteration in JIRA cloud, leveraging the technical components and working closely with the internal developer of Version 1.0. Kororo developers worked with MHNSW staff representing different business needs to determine the requirements for Version 2.0, including data validation and identification of potential technical errors and the ability to accommodate formats such as MP3, MP4, PDF, MSG, JPG and STL. The collaborative development and design process resulted in a more streamlined JIRA experience.
The result Business case objectives were met, with the tool increasing MHNSW’s capability and capacity to ingest large amounts of digital material at one time. Scalability was achieved through increased processing power, a holding area and queue scheduling. Flexibility was achieved through a new material flow that could accommodate identified digital files formats. The JIRA design provides a simplified user interface, and its labels and customisable lists empower teams to self-configure new projects. The result was a multi-function tool for scalable automation across the organisation.