HSBC’s Global Digital Archive (GDA) system, built on a 2012 server architecture, was becoming outdated and hard to maintain. The team faced increasing challenges in patching vulnerabilities, upgrading server components and ensuring descriptive and technical metadata was correctly synchronized across two separate platforms for cataloguing and digital storage and preservation. Compatibility issues led to skipped upgrades and multi-vendor priority divergence, making it difficult to maintain user experience. The unique (within a bank) software and associated hardware infrastructure created knowledge gaps or did not conform to wider IT processes, prolonging maintenance and system development. All this necessitated planning for an architecture renewal and upgrade project. This was seen as an opportunity to reappraise the goals of the GDA and sought to move to a second generation of the system – a new system that was developed to improve efficiency, sustainability and the holistic management of digital, digitised and physical archival assets.
Key project work streams:
-
Uplift of over 175,000 individual digital objects and their associated technical and descriptive metadata to a new cloud (AWS) based digital repository and the associated decommissioning of the old architecture.
-
Migration of all descriptive metadata for all physical records to exist alongside the digital record metadata within the new cloud platform.
-
Migration of the old digital preservation platform’s database to a new database system.
-
Development of an ISAD(G) standard descriptive metadata management tool designed to sit within the digital preservation platform with the primary vendor, Preservica.
-
Improvement of search functionality to assist broader archival searching (in addition to specific digital asset searching).
-
Creation of an automated mechanism for transferring content from within the HSBC managed network to a third party cloud hosted preservation platform.
-
Establishing a global transfer agreement for digital records across HSBC, ensuring legal agreements for the transfer of records into the system from all 62 countries and territories that HSBC currently operates in.
-
Developing a security model that would allow digital assets with high security ratings to be managed on a third party shared cloud platform, adhering to the multilayered security standards required by a global financial services company.
Main achievements of this project:
-
Ensured the long-term future and sustainability of HSBC’s digital collections.
-
Demonstrated that holistic management of archival records, both digital and physical, is both possible and beneficial.
-
Created and integrated a new cataloguing/metadata management tool for all users of the vendor’s platform. This was agreed at the outset of development and, whilst not open source, having an effective metadata management tool in-place within a trusted digital repository and preservation platform will benefit a wide range of users. Without this project there would be no digital preservation platform that allows users to easily catalogue their content in a way that fulfils the demands of traditional, hierarchical archival arrangement.
-
Developed enhanced search options that provide search results that can be viewed and refined in a way that has greater relevance to archival users. The team focused on work with the vendors to offer a range of new search features that allow the end user to customise their results screen. This enables users to search and make accurate assessments of records based on the metadata fields displayed in the results. It also provides greater contextual information from the record’s hierarchy that would not typically be visible within the search function of a standard DAM.
-
Developed a seamless and more efficient transfer mechanism for automating the upload of content for the digital archive system out of the HSBC managed network to the third party cloud hosted preservation platform (Preservica).
-
Produced a digital preservation and metadata management model that can be replicated by other businesses and large organisations that have significant security concerns over the management of data by third parties. This model has been shared and ensured the bespoke features developed with the vendor are available to all other customers.
-
Showcased a versatile framework that can be adopted by other sizable organisations seeking to implement a digital preservation system, particularly those with limited capital budgets and an inability to maintain an in-house solution.
-
Strengthened HSBC Archives' commitment to a sustainable archive service. Overall energy consumption has been reduced by transitioning from a self-hosted digital preservation system on separate physical servers across two data centres to a cloud-based solution powered by 100% renewable energy. The cloud platform's highly efficient server equipment delivers superior performance per watt, ensuring that the digital preservation program continues to become increasingly energyefficient through the unique benefits of a shared cloud infrastructure.
Conclusions:
The innovative approach taken in consolidating the management of physical and digital records, along with their metadata, has revolutionised the archival cataloguing and digital preservation system. By streamlining processes and adopting a user-friendly interface, the HSBC team has successfully managed to integrate the descriptive metadata of both digital and physical assets within a single, unified system. Physical and digital assets associated with the same catalogue entry are held alongside each other, having different technical metadata profiles, but they share descriptive metadata (such as its title, description, reference number etc…) which is stored in the parent folder that holds both assets. This groundbreaking solution not only offers a more powerful and stable platform for search, cataloguing and hierarchical browsing of archival tree structures, but also serves as a model for third party cloud-based holistic archival management and digital preservation.
As the team continue to share our experiences and support other organisations in their quest to simplify metadata management for hybrid collections, they are confident that the upgraded HSBC Global Digital Archive system will pave the way for large organisations with preservation needs, low data-risk appetites, and limited on-site resources or expertise. By embracing this cutting-edge model, they are ensuring the long-term preservation and accessibility of our hybrid collections, ultimately benefiting both HSBC and the wider community.