DPC

The Digital Preservation Coalition has examined JPEG 2000 in a report published today.  The report concludes that JPEG 2000 represents a great stride forward for the archival community.  The format now allows for greater compression rates and a recompression rate that is visually lossless.

The findings come as the Digital Preservation Coalition launch its latest 'Technology Watch Report' written by Dr. Robert Buckley, a Research Fellow with Xerox, 'JPEG 2000 - a practical digital preservation standard?'.  The report looks in-depth at the new format and the challenges it has to cope with.  JPEG 2000 is widely used to collect and distribute a variety of images from geospatial, medical imaging, digital cinema, and image repositories to networked images. Interest in JPEG 2000 is now growing in the archival and library sectors, as institutions look for more efficient formats to store the results of major digitisation programmes.

Programme

Background

The Digital Preservation Coalition (DPC), now in its sixth year, seeks to be a proactive membership organization fostering joint action to address the urgent challenges of securing the preservation of digital resources in the UK.

At the start of the new chapter in its history, the DPC would like to hear from its members and to provide them with an opportunity to shape its next, hopefully equally successful and productive, phase. So please do attend this event which will go some way to inform DPC activities and priorities.

About the Day

The morning session will comprise of presentations from the five short-listed DP Award projects. This will provide you with an opportunity to learn more about the diverse and fascinating projects that were showcased at the Conservation Awards ceremony in September. We will also hear the Judges' perspective into how they evaluated the projects. You may even pick up some tips for the 2009 DP Awards!

http://www2.digitalpreservationcoalition.co.uk/advocacy/2007-shortlist.html

This session will be chaired by Kevin Ashley, the Chair of the Judging Panel.

After a nutritious lunch and an initial briefing you will break up into groups to focus on your selected topics. We will endeavour to ensure that you are assigned to your preferred group but we cannot always guarantee this and indeed may have to assign you to another group. There will however be an opportunity to contribute views on all issues that will be covered during the day. Full details of your group allocation will be sent to you before the event.

The key recommendations, comments, issues which arise from the discussion will be shared with other participants through a 'speed reporting' session (more about this on the day) followed by the final summing up of all those 'big ideas'.

Breakout Group Themes

The following are the themes which the groups will consider. The bullet points are merely suggestions to kick start the discussions - you may wish to consider other issues within your area - so please do feel free to! Further details will be provided before the event. Please indicate your preferences for the groups, in 1-5 order where (1) is your preferred option.

1. 'Top 5' for 2008 - Planning for the Near Future:

   1. Consider what are the most (5) important issues in the digital landscape over the coming year
   2. How will these impact on digital preservation?
   3. What are the threats and opportunities that these issues will bring to digital preservation
   4. How can the DPC contribute to these discussions?
   5. How can the DPC take forward these issues on behalf of its members?
2. DPC Governance:
   1. Look at the mission - is this still relevant? If not why not? [A copy of the published goals etc will be provided]
   2. How would you improve the mission statement?
   3. Look at the goals, aims and principles. Could these be recast, reshaped?
   4. If so how, why etc.
3. DPC Activities:
   1. What type of event do you find useful e.g. workshops, conferences?
   2. What activities would you like the DPC to run that they have not previously done?
   3. Is the DPC 'Handbook of Digital Preservation' still useful?
   4. Would you like to see the Handbook updated?
   5. Would you like to see a follow up to 'The Mind the Gap' report?
   6. Would you like to see more training programmes offered through the DPC?
4. Outreach:
   1. Website - how could this be improved?
   2. What information would you like to see on the website?
   3. Awareness alerting tools. What tools work best for you? For example the 'What's New' list on the website, RSS feeds, mailing lists etc.
   4. Create some strap/tag lines for the DPC. Why are these meaningful?
   5. Suggest 5 pragmatic ways how the DPC could continue to serve its members.
5. Future Gazing - Making Ready for the Mid Future :
   1. Picture star date 2017, what will the digital landscape comprise of then for all our constituencies e.g. learners, citizens, researchers, government etc?
   2. What emerging technologies, trends (social, organizational etc), developments etc will impact on digital preservation?
   3. How can we ensure that the digital preservation component is kept on the agenda of other allied digital initiatives?
   4. How can the DPC empower its members to be fit for the 'digital preservation purpose' now and in the future?

Who Should Attend?

Colleagues from member organizations whatever your role and interest is in digital preservation or related digital asset management and stewardship areas. Your views and comments would be most welcome.

Benefits of Attendance

• A chance to learn from exemplars of good practice from the DP Award short-listed projects
• To network and engage with colleagues from other member organizations
• An opportunity to air your views on matters digital preservation
• A chance to dabble in 'digital gazing'
• To contribute to, and take forward the work and aims of the DPC
• Finally to become a shaper of the DPC

Introduction

The JPEG2000 image compression technique has been cited by experts as a new archiving format for digital images. It is both a preservation and delivery format, and has been seen as a possible alternative to the TIFF format which most institutions use as a long-term archiving standard. Produced by both imaging experts and the Joint Photographic Experts Group, it is now a recognised ISO standard. The standard JPEG file format which is so widely in use is not yet an ISO standard.

JPEG 2000 allows a wide range of uses and can support a wide range of formats and multiple resolutions. It can also offer both lossy and lossless compression modes. Most importantly it is a flexible file format which allows metadata to be built in to the file, a vital element of the digital preservation process.

However, the standard is not yet widely in use and there is as yet no native support for it in internet browsers. More investigation and practical implementations of the standard are yet to be seen but it could be used as a potential archival standard.

The workshop

This forum will look more into the details of the standard and expert speakers who are familiar with the standard or have implemented it will share their experiences. The forum will also include industry experts to talk about the creation of the file formats. Delegates will learn about the benefits of the standard, especially with regard to digital preservation and whether it is worth implementing it within their own institutions as an image storage format. Guest speakers include Bill Comstock, Harvard College Library, Christoph Becker, Vienna University of Technology, Manfred Thaller, Cologne University and Jim King, Adobe.

Who should attend the workshop?

Digital Repository Developers, Library and Archives professionals, Digital Preservation (technology watchers) future trends, Information Managers (public sector) and anyone with large numbers of images within their institutions or companies.

Cost of the workshop is £60 for DPC members and £100 for non members

Programme

The workshop will be chaired by Professor Roger James, Head of Digital Library Infrastructure, British Library.

Afternoon session

Article on Overview of E-journal Archiving and Preservation Workshop, UKSG, Serials 20 (3) November 2007

Overview and Presentations

The DPC, The British Library and JISC held a 1 day workshop looking at the issue of e-journal archiving and preservation. The workshop brought together international stakeholders from the information world, including publishers, librarians and representatives from archiving solution providers. The day was chaired by Hazel Woodward of Cranfield University and provided an opportunity to review what initiatives are currently in place and what more developments are needed.

The first morning session saw Anne Kenney (PDF 519KB) of Cornell University provide an overview of the e-journal archiving landscape, drawing on work that was done in 2006 to pull together the opinions of library directors and also provide a detailed review of 12 archiving solutions. This was followed by presentations from 4 different providers of archiving and preservation services. Victoria Reich (PDF 13MB) , Director of the LOCKSS program, gave an overview of LOCKSS and CLOCKSS and emphasised that the more archived copies of an item the safer it is. Eileen Fenton (PDF 593KB), Executive Director of Portico, described the service provided by Portico which archives peer review scholarly journals and to date has more than 5,800 journals from 30 publishers. Erik Oltmans (PDF 377KB), Head of Acquisitions and Cataloguing at the National Library of Netherlands (Koninklijke Bibliotheek) spoke of the e-deport service which from a starting point in 1995 is now ingesting 50,000 items a day. The UK national library perspective was given by Richard Boulderstone (PDF 418KB), Director of e-Strategy at The British Library, who showed that the traditional scholarly communication process is changing and this requires new processes and durable technical architecture to ensure that archiving solutions are able to satisfy requirements now and in the future.

The afternoon session saw presentations from the publisher and university library perspectives. Steven Hall (PDF 156KB), Commerical Director at Wiley-Blackwell, demonstrated that publishers take the issue of archiving seriously but there needs to be more clarification when it comes to access. Preservation is not the same as access and access is not the same as open access. Paul Ayris (PDF 440KB), Director of Library Services at UCL, said that UCL were moving towards the e-only delivery of journals and therefore were reviewing what solutions need to be in place to ensure continued access. Paul also highlighted the potential costs of archiving.

The workshop finished with a question and answer session which enabled delegates to pose questions to the expert panel and Hazel Woodward brought the day to an end by summarising the themes of the day.

Morning session: The E-Journal Preservation Landscape
Chair: Hazel Woodward

Afternoon session: High-level Principles of E-journal Archiving and Preservation Services
Chair: Hazel Woodward

  
This one day forum was jointly organised by CILIP in Scotland and the Digital Preservation Coalition and was aimed at library and information professionals who work with digital resources.  The forum explored whether content is being locked away  behind technical protection measures and asked how we can make content accessible while upholding the rights of intellectual property owners. The forum also highlighted digital preservation issues in order to ensure long term accessibility of digital resources.

Programme

Chair: William Kilbride, Glasgow City Council Museums

The DPC held a Briefing day on different implementations of digital repositories. This event was aimed at people who are in the process of planning digital repositories and who want to find out more about available tools and the benefits of each model. The day highlighted different approaches to and aspects of repository models and was useful for those who wish to take a more modular approach to repository building. The day was also useful to people who are starting to move from strategic planning to actual repository implementation.

Sayeed Choudhury emphasised that institutional repositories should provide both an institutional service and long-term custodianship of digital academic output. He also highlighted however how there was a definite lack of discussion on the digital preservation element within repositories; preservation is very hard indeed, and there need to be more case studies. An important theme was that there is no one-size-fits-all solution and institutions should focus on their needs rather than on their systems.
Sayeed Choudhury - Johns Hopkins University: Digital Repository Models (PDF 123KB)

Dave Thompson of Wellcome Trust outlined why they chose Fedora as a system. The outstanding reasons were that it was cheap and versatile. He outlined the implementation of the relationship builder and the metadata extraction tool. As a test bed Dave chose to archive an email spam collection and the methodologies for doing this provide a good case study. The overall message was that a good approach is to manage the information and metadata, not the technology.
Dave Thompson - Wellcome Trust: Fedora at the Wellcome Library, progress and work to date (PDF 150KB)

Paul Bevan continued on the Fedora theme and how they have used it at the National Library of Wales for their digital asset management system. Their interesting project has mapped OAIS elements into the archive. Paul emphasised the importance of cross-organisation buy-in, and the technical challenges of moving digital objects into a managed environment.
Paul Bevan - National Library of Wales: Implementing an Integrated Digital Asset Management System: FEDORA and OAIS in Context (PDF 1.1MB)

Steve Hitchcock outlined the history of institutional repositories and how they have gradually developed out of the open archival initiative. The highlighted the results of a survey of repositories; for example, how only one out of the eighteen surveyed even have a preservation policy for their repository, which is a cause for concern.
Steve Hitchcock - Southampton University: Repository models and policies for preservation (PDF 581KB)

Andrew Wilson's talk outlined the SherpaDP project which investigates a distributed preservation model. This comprehensive work flow is to be the basis of a business model, not a free service; almost an exemplar for outsourced preservation services. The project will also create a handbook. He emphasised that there is a need for 'object mobility' within the model, and how each of the detailed workflow modules map to the OAIS model.
Andrew Wilson - AHDS: SHERPA-DP: Distributed Repositories/Distributed Preservation (PDF 153KB)

The Digital Preservation Coalition (DPC) and Digital Curation Centre (DCC) delivered a two-day workshop to explore the range of policies required to manage, preserve, and reuse the information held within digital repositories over time. This event was co-sponsored by the Oxford Internet Institute (OII) and held at Wolfson College at the University of Oxford on 3rd and 4th of July, 2006.

Developing and implementing a range of policies is vital for enabling the effective management, discovery, and re-usability of information held within digital repositories. This workshop provided concrete examples of the range and nature of the policies required and shared real-life experiences in implementing these policies through a series of case studies and panel discussions.

Monday July 3rd, 2006

Setting the Scene

• Digital information life-cycles and policies (PDF 31KB)
  David Ryan, Director of Records, The Royal Household
• Data Sharing and Curation Policies across the UK Research Councils (PDF 56KB)
  Mark Thorley, NERC

Session One: This session explored issues including: roles and responsibilities in developing policies, relationships with other institutional policies, workflow issues, key themes of specific policies, problems encountered during development.

• Developing the NERC Data Policy (PDF 382KB)
  Mark Thorley, NERC
• Developing a digital preservation strategy (PDF 173KB)
  Heather Needham, HRO
• TNA's custody of digital records policy (PDF 116KB)
  Malcolm Todd, TNA
• Digital repositories and Grids (PDF 1.2 MB)
  Bob Jones, CERN

Tuesday July 4th, 2006

Session Two: Implementating Curation and Preservation Policies

• Ingest at the Archaeology Data Service: a case study (PDF 3.3MB)
  Jennifer Mitcham, Archaeological Data Services (ADS)
• Implementing selection and appraisal policies at the UK Data Archive (PDF 41KB)
  Kevin Schurer, UKDA
• The Case of the Disappearing Documents: the Capture and Curation of Current University Records (PDF 245KB)
  Susan Graham, Records Manager, the University of Edinburgh

Session Three: Evaluating and Reviewing Curation and Preservation Policies

• The public consultation process for Operational Selection Policies (PDF 108KB)
  Howard Davies, TNA
• Preserving, Sharing and Re-using Biomedical Research Data: Policy and Practice (PDF 245KB)
  Allan Sudlow, Medical research council (MRC)

The DPC held a one-day web archiving forum at the British Library. The first DPC web archiving forum was held in 2002 to promote the need to archive websites given their increasing importance as information and cultural resources.

Four years on, this event again brought together key web archiving initiatives and provided a chance to review progress in the field. The day provided an in-depth picture of the UK Web Archiving project as well as European initiatives. Technical solutions and legal issues were examined and the presentations encouraged much debate and discussion around different strategies and methodologies. The event made clear that the field has moved on tremendously from four years ago. The debate has broadened and so have the tools and methodologies.

The first presentation was from Philip Beresford, Web Archiving Project manager at the British Library [BL]. He spoke about the BL's involvement with UKWAC, the tools the project had built, the challenges they have had with the PANDAS software and the overall constraints of web archiving, especially as it is such a technology dependent discipline. Philip also outlined the web curator tool developed with the National Library of New Zealand and the next version of PANDAS. UKWAC - the first two years [PDF 33 KB]

Adrian Brown, Head of digital preservation at the National Archives followed on from Philip's talk as he outlined the future of UKWAC and its recent evaluation report. Adrian outlined the collection methods at the National Archives as well as database preservation and transactional archiving. He touched on one rather overlooked aspect, that of the long term preservation of the actual content. Collecting and preserving web content [PDF 401KB]

John Tuck spoke in the second session about the legal BL's deposit bill. He touched on issues regarding collection, capture, preservation and access to non-print collections. Of interest is how the legal deposit bill translates into the e-environment and web archiving; should web archiving extend to UK-related sites, not just UK-domain sites and are national boundaries less relevant now? He outlined the BL's two different strategies - taking a twice yearly snapshot of the entire UK web and the second being a more selective approach of sites that are deemed to be of national and cultural interest. He also stressed the lengthy permissions process that gathering each web site entails. Collecting, selecting and legal deposit [PDF 42KB]

Andrew Charlesworth highlighted the complexity of the UK legal framework regarding web archiving. An emerging theme throughout the day was the debate about whether archives should ask for permission before or after they have collected websites. Andrew stressed the importance of understanding the regulatory framework. The field has moved on in that we know more today about the risks and benefits regarding web archiving than we did a few years ago. Any web archiving project probably needs to carry out risk analysis and to have insurance, in particular with regard the defamation law, ensure that they don't hold anything in their archive that could be used as legal evidence. Archiving of internet resources: the legal issues revisited [PDF 33KB]

Julien Masanes spoke about the European Web Archive [PDF 530KB] . He presented an interesting approach to web archiving - the information architecture of the web is such that its archiving should follow the natural structure of the web. Julien reminded the audience that web content is already digital and readily processable and that the web is cross-linkable and cross-compatible, a good foundation for an archive. He also stressed that web archiving requires functional collaboration. What is needed is a mutualisation of resources which combines competence and skills. Internet preservation: current situation and perspectives for the future [PDF 530KB]

Paul Bevan outlined the UKWAC project to archive the 2005 UK election sites. He described how three national libraries collaborated on this web archive. He touched on selection, collection remit for each library and frequence of snapshots. Did the general election classify as an event or as a known instance? Paul stressed the difficulties involved in obtaining permissions to archive electoral websites and the difficulties in identifying candidates websites. On a technical level the slowness of the gathering engine was also highlighted. Archiving the 2005 UK General Election [PDF 129KB]

Catherine Lupovici of the International Internet Preservation Consortium [IIPC] outlined the activities of the IIPC and outlined all the life-cycle tools that the team are working on such as ingest and access tools. She stressed the importance of collaboration in web archiving and it is clear that both UKWAC and IIPC do this successfully. IIPC activity: standards and tools for domain scale archiving [PDF 149KB]

The panel session was most productive. The panel leaders stressed that we are still in the early days of web archiving. We can never be fully sure that the techniques employed are correct, but we have to make a start. However, more research needs to be carried out into the preservation techniques of the actual content. Access issues are also critical; searching a digital web archive won't employ the same search and retrieval tools as a traditional archive would and crucial access tools need to be developed for successful use of web archives.

On a technical note, we need to be aware of issues of browser compatibility in the future; there was a debate about whether it was an acceptable solution to obtain the source code of browsers in order to assist rendering pages in the future. It was highlighted that we have to be aware of unknown plugins which could hinder the readability of web pages. The importance of the ingest stage was stressed and the transformation of the digital object that should occur at this point to ensure readability. There may be legal issues to consider here however in transforming from one format to another.

Web archiving is not an isolated activity - so many web formats are now available as well as different content delivery mechanisms e.g. blogs and chat rooms. These formats make archiving even more challenging.  There was a recognition that the community needs smarter tools to make web archiving scalable. There is definitely a need to semi automate quality assurance and selection. The question was raised whether or not we still need manual and selective archiving which is both time-consuming and costly compared to automatic sweeping of the web? The general consensus was that both methodologies should still be employed. The overall conclusion and recurring theme of the day is that collaboration is essential and no single organisation can carry out web archiving on its own. Projects such as UKWAC, IIPC and the European Web Archive demonstrate that much can be achieved in terms of solutions and methodologies.

The DPC held a briefing day on the OAIS model on 4^th April 2006 at the York Innovation Centre. The purpose of the day was to examine the model and provide an informal but in-depth look at the practical application of the Open Archival Information System [OAIS] model in various UK institutions. OAIS is a high-level reference model which describes the functions of a digital archive and has been used as a model for a number of digital archiving repositories. It is now a recognized and highly-prominent international standard.

There were four presentations in total, all of which presented interesting case-studies and examples of OAIS implementation in a variety of institutions, giving a valuable overview of how it has been interpreted and applied.

Najla Semple, Digital Preservation Coalition, began the day with an overview of OAIS and her experience of implementing the model at Edinburgh University Library in 2002 (Overview of OAIS PDF 1MB). She gave a summary of the pilot project and how she used the model to digitally archive the online University Calendar. Each of the six OAIS processes were examined and used as part of the archival workflow. She also gave an overview of the detailed OAIS metadata scheme that was implemented.

Jen Mitcham, Curatorial Officer at the Archaeology Data Service [ADS] presented next (Working with OAIS PDF 2.6MB). Her approach to OAIS was different from that of Edinburgh University Library as ADS already have a digital archive up and running. At ADS they have applied the model to their existing digital archive, which is both an interesting and practical way to approach the model. Her talk identified which areas of her organization the model could be applied to, and she clarified this by including photographs of the actual staff involved in each of the OAIS processes. The issue of registration and access to online archives was debated.

Andrew Wilson, Preservation Services and Projects Manager at AHDS spoke in the afternoon (Sherpa-DP and OAIS PDF 300KB) about the use of OAIS in the Sherpa DP project http://ahds.ac.uk/about/projects/sherpa-dp/. They are using a disaggregated model for implementing the model throughout the university-based institutional repositories and he indicated that they will share an AHDS preservation repository. He then initiated the question, 'What does OAIS compliance mean?', an interesting question with regard to institutions setting up their own archives. He touched on the OAIS audit process developed by RLG and what this will mean for future implementation of the model. A certification process might lead to the assumption that the model will have to be implemented in a certain prescriptive way and perhaps this goes against the 'open' spirit of OAIS. Some of the processes are 'deliberately vague' therefore they shouldn't be set in stone as to how one applies them. This issue initiated much lively debate amongst the delegates.

The final presentation of the day was a joint effort by Hilary Beedham of the UK Data Archive and Matt Palmer of the National Archives (Mapping to the OAIS PDF 500KB). They gave an interesting insight into two archives that are both assessing their existing organizational structure against the OAIS model. Interestingly, they both arrived at similar conclusions and found certain shortcomings with OAIS. A couple of areas that they struggled with were management of the Dissemination Information Package, as well as the metadata model which they thought could perhaps be made more detailed to include access controls and IPR concerns.

Matt also pointed out that it is fairly easy to be compliant with OAIS as most of the functions and processes are core to any digital archive. Both the TNA and UKDA Designated Communities are wide-ranging and they indicated that it might be the case that the model assumes a homogenous user community. However, this point was disputed by the audience as indeed the issue of the Designated Community is a very important feature of OAIS and establishing who you are preserving the information for is crucial. The Representation Information metadata field assumes that you will include an appropriate detailed technical description according to who will read the data in the future.

Hilary Beedham concentrated on their recently published report, 'Assessment of UKDA and TNA Compliance with OAIS and METS Standards' http://www.jisc.ac.uk/uploaded_documents/oaismets.pdf. The JISC-funded report was written partly to assist regional county-councils apply the model and simplify it.

The discussion at the end of the day proved very fruitful, and overall conclusions were as follows:

• That it was really useful to have some real-life examples and case-studies.  
• OAIS vocabulary and terminology is now recognised as really useful across a range of institutions.  The value of having OAIS-compliant repositories will also enable a 'seamless transfer' of data between archives.
• While the model may be vague in its prescription, it certainly indicates what to think about when setting about creating a digital archive.
• One delegate suggested that the starting point should be to look at your own organization first, analyse the processes involved and apply OAIS processes accordingly.
• A practical guide as to how to set up an OAIS repository would be very useful, especially one which indicated different communities and organizational-specific interpretations.  This guide could ideally take the form of 'OAIS-lite'.