|
Although corporate and legal issues have recently brought data archiving to the light of day, the problems associated with preserving digital information are not new. Archiving for a few years is hard enough, but when requirements dictate that data be retained for longer, problems with media deterioration and technology obsolescence can seem insurmountable.
- Source: Galen Schreck - Forrester Aug. 2005
Long term preservation of digital content is a big challenge in the Information Society era, digital information in any form is at risk to be lost forever. Technology on which digital content relies becomes obsolete and application versions and files formats change, making data soon inaccessible. Even if content is coded in the simplest format, such as ASCII code, storage media degradation and obsolescence could make it disappear. Even on-line information such as web pages and databases, are vulnerable as much as their web structure become complex thanks to (aging) hyperlinks and cross references.
- Source: Alfredo M. Ronchi - Medici Framework
The Digital Crisis
- Risk of losing digital information over time
- Growing cost and complexity of physical and logical migration
- Overwhelming volume of digital information to preserve long-term
- Increased legal, business, and security risk
What is the state of best practices today?
Standards for Electronic Archives: Long-term electronic archive designs should consider incorporation of national or international specifications such MoReq or Open Archival Information System (OAIS). Standards such as ISO 154895 establish guidelines for records management policies and systems but fall short of specifying functional details of automated systems.
How do you cost-effectively manage and automate the preservation of petabytes of information forever, growing at 50% to 100% per year in the data center? How do you migrate multiple PBs per year logically and physically? The answer is you can’t. The IT archive process and system is technologically broken. It doesn’t scale. - Michael Peterson, Chief Strategy Advocate, SNIA-DMF
Migration is Required: NARA says "If information is on disk drives, migrate it every 3 years and if on tape, every 5 years." But, what about the ability to read and interpret the information long term?
Physically, long-term storage is not about media life (because migration is required) but about periodic migration to newer media. But, in a large repository, how do you migrate multiple pedabytes every year and at what cost? At some point migration becomes overwhelming and the digital retention process is broken - it does not scale.
Logically, long-term retention is about the ability to read the information and to be able to use it. Applications have a relatively short life and rarely have the ability to read information older than a few revisions. Even standard formats evolve, change, and become obsolete. At some point, information has to be migrated periodically to a new standard logical format.
Then what about issues such as compliance, integrity, authenticity, privacy, and discovery? How are you guaranteeing these over the retention period?
The 100 Year Archive Task Force believes that it has identified solutions to these two "grand challenges" of long-term retention. Join us and help progress the work.
Standards and Resources:
Long-term electronic archive designs should consider incorporation of national or international specifications such MoReq or Open Archival Information System (OAIS). Standards such as ISO 154895 establish guidelines for records management policies and systems but generally fall short of specifying functional details of automated systems. However, DoD 5015.2-STD and MoReq contain useful information defining functional requirements for electronic record archives. Both of these also define selected metadata elements required for an electronic records archive. Either document would be appropriate as a starting point for acquisition or construction of an electronic archive system. Finally, both ARMA International and the National Archives Records Administration (NARA) provide planning and guideline documents at their respective web sites
| 
