Long Term Retention
SNIA’s Long Term Retention Technical Working Group (TWG) developed a SNIA standard for a logical container format called the Self-contained Information Retention Format (SIRF). This SNIA standard has been approved by ISO as a new international standard, ISO/IEC 23681:2019. SIRF enables long-term hard disk, cloud, and tape-based containers a way to effectively and efficiently preserve and secure digital information for many decades, even with the ever-changing technology landscape.
The SIRF standard addresses the technical challenges of long-term digital information retention & preservation for both physical and logical preservation. SIRF is a storage container of digital preservation objects that provides a catalog with metadata related to the entire contents of the container, individual objects, and their relationships. This standardized metadata help interpret the preservation objects in the future
Value to the Industry:
- SIRF serialization for adapted industry technologies is provided in the specification.
- SIRF serialization for the cloud is supported using OpenStack Swift object storage and SIRF serialization for tapes is supported using the LTFS standard.
- SIRF is media agnostic, vendor agnostic, extensible, self-described and self contained and plays a key role in the preservation and retention of critical data, and because it is interpretable by future data preservation systems, it has the benefit of greatly reducing the associated costs of digital preservation.
Watch our BrightTALK webcast on-demand:
Long-term Data Retention: Challenges, Standards and Best Practices
Download the slides from the webcast: Long-term Data Retention: Challenges, Standards and Best Practices
More information about the SIRF standard
The SIRF standard is published as a SNIA technical position, and is available here.
OpenSIRF is an open source implementation of the SIRF format and it is available under the MIT license. OpenSIRF can be found here: https://github.com/opensirf
Aspects of SIRF serialization for the cloud are being experimented with OpenStack Swift object storage and the stored data is designed to be understandable and does not need specialized software to interpret it.
- SIRF is interpretable by future data preservation systems, thus reducing the associated costs of digital preservation.
- SIRF plays a key role in the preservation and retention of important data.
- Typical use cases for SIRF would be the preservation of data attached to medical records, libraries and in the Movie/Film industry.
SIRF technical documents are available on the SNIA site and we encourage everyone to review and comment on the current specification v1.0.
- Self-contained Information Retention Format (SIRF) Specification v1.0
- ISO/IEC 23681:2019 Information technology -- Self-contained Information Retention Format (SIRF) Specification
- SIRF Use Cases and Functional Requirement Document
SIRF Scientific papers:
- Simona Rabinovici-Cohen, Roger Cummings and Sam Fineberg, "Self-contained Information Retention Format For Future Semantic Interoperability", Proceedings of the 4th International Workshop on Semantic Digital Archives (SDA), September 2014, London, UK
[paper] - Simona Rabinovici-Cohen, Mary G. Baker, Roger Cummings, Sam Fineberg, and John Marberg, "Towards SIRF: Self-contained Information Retention Format”, Proceedings of the Annual International Systems and Storage Conference (SYSTOR), May 30-June 1, 2011, Haifa, Israel
[paper]
Selected LTR public presentations:
- A SNIA tutorial "SIRF: Self-contained Information Retention Format", presented by Sam Fineberg and co-authored by Simona Rabinovici-Cohen (IBM), Roger Cummings (Antesignanus) & Phillip Viana (IBM) at the 2015 USENIX Conference on File and Storage Technologies (FAST), February 2015, Santa Clara, CA, USA;
- A SNIA tutorial "Combining SNIA Cloud, Tape and Container Format Technologies for Long Term Retention", presented by Sam Fineberg and co-authored by Simona Rabinovici-Cohen (IBM) & Roger Cummings (Antesignanus) at the 2014 SNIA Data Storage Innovations Conference (DSI), April 2014, Santa Clara, CA, USA;
- A SNIA tutorial "Combining SNIA Cloud, Tape and Container Format Technologies for the Long Term Retention of Big Data", presented by Sam Fineberg and co-authored by Roger Cummings (Antesignanus) & Simona Rabinovici-Cohen (IBM) at the 2013 SNIA Analytics and Big Data Summit, September 2013, Santa Clara, CA, USA;
- A webcast "Digital Content ‐ Challenges of Long‐term Storage", presented by Mary Baker (HP) to the Society of Motion Picture and Television Engineers (SMPTE) on May 10, 2012;
- A SNIA tutorial "Bringing Light to the “Digital Dark Age” – Preserving Digital Information for the Long Term", presented by Roger Cummings (Symantec) at the Spring 2012 StorageNetworkingWorld conference (April 2-5) in Dallas, Texas, USA;
- A presentation entitled "Long Term Information Retention" given by Sam Fineberg (HP) & Simona Rabinovici-Cohen (IBM) as part of the SNIA 2011 Storage Developer Conference (September 19-22) in Santa Clara, CA;
- A tutorial "Long Term Preservation of Digital Information" that has been presented by Mary Baker (HP) and Roger Cummings (Symantec) at a number of StorageNetworkingWorld conferences in the US and elsewhere for the last two years;
- An extended webinar version of the above tutorial;
- A presentation "Long Term Retention Format" made by Sam Fineberg (HP) and Simona Cohen (IBM) at the 2009 SNIA Storage Developers Conference;
- A keynote "Long Term Retention Work @ SNIA" presented by Roger Cummings (Symantec) at the ECMA 1st International Workshop on Standards and Technologies in Multimedia Archives and Records (STAR), plus a video of the presentation;
References for Long-Term Retention & Preservation:
- Long term Retention related work from MSST 2013
- LTR cost modeling work from UC Santa Cruz
- Dagstul Seminar report, "Is the future of preservation cloudy?"
- Links related to the economics of long term retention
- LTR Related papers from the 2012 SNIA Storage Developers Conference
- Alliance for Permanent Access 2012 Conference Presentations
- 2012 Creative Storage Conference http://www.creativestorage.org/
- US Workshop on Roadmap for Digital preservation Interoperability Framework
- OpenSIRF has a developer Google Group hosted here.
Liaison Organizations with LTR TWG:
- EU FP7 Integrated Project ForgetIT- Concise Preservation by combining Managed Forgetting and Contextualized Remembering
- The TIMBUS project: http://timbusproject.net
- EU FP7 Integrated Project ENSURE - Enabling kNowledge, Sustainability, Usability and Recovery for Economic Value
- SNIA Cloud Archive and Preservation Special Interest Group
- Preservation and Archiving Special Interest Group