Bringing Light to the "Digital Dark Age" Preserving Information for the Long Term (2011)

Many organizations are facing the serious challenge of economically preserving and retaining access to a wide variety of digital content for dozens of years. Long-term digital information is vulnerable to issues that do not exist in a short-term or paper world, such as media and format obsolescence, bit-rot, and loss of metadata. Ironically, as the world becomes digital, we may be entering a "Digital Dark Age" in which business, public and personal assets are in ever greater danger of being lost. The SNIA Long Term Retention (LTR) Technical Working Group works with key stakeholders in the preservation field, to develop the Self-contained Information Retention Format (SIRF) and enable applications to interpret stored data, independent of the application that originally created it. SIRF is a logical container format for the storage subsystem appropriate for the long-term storage of digital information. SIRF consists of preservation objects and a catalog containing metadata relating to the entire contents of the container as well as to the individual preservation objects and their relationships. It makes it easier and more efficient to provide many of the processes that address threats to the digital content at a lower level of the system stack and can be performed close to the data using more robust, efficient, and automatic methods. Easier, more efficient preservation processes in turn lead to more scalable and less costly preservation of digital content. SIRF will be examined in a new European Union integrated research project, called ENSURE – Enabling kNowledge, Sustainability, Usability and Recovery for Economic Value. ENSURE creates a preservation infrastructure for commercial digital information built upon cloud storage and virtualization enablement technologies. It explores issues such as evaluating cost and value for different digital preservation solutions, automation of preservation processes, content-aware long term data protection, new and changing regulations, and obtaining a scalable affordable solution by leveraging cloud technology. The presentation will cover use cases, requirements, and the proposed architecture for SIRF as well as its potential usage in ENSURE storage services.

Learning Objectives

Recognize the challenges in the long-term preservation of digital information, and understand why best practices and optimal solutions must address different levels of available time, money and effort.
Identify the need, use cases, requirements, and proposed architecture of SIRF. Also, review the latest activities in SNIA LTR technical working group and the development of SIRF.
Discuss the usage of SIRF in the ENSURE cloud infrastructure that draws on actual commercial use cases from health care, clinical trials, and financial services.