Spiffy: Enabling File-System Aware Storage Applications

Author(s)/Presenter(s):
Library Content Type:
Publish Date: 
Thursday, September 27, 2018
Event Name: 
Focus Areas:
Abstract: 

Many file system applications such as defragmentation tools, file system checkers or data recovery tools, operate at the storage layer. Today, developers of these storage applications require detailed knowledge of the file system format, which takes a significant amount of time to learn, often by trial and error, due to insufficient documentation or specification of the format. Furthermore, these applications perform ad-hoc processing of the file-system metadata, leading to bugs and vulnerabilities.

We propose Spiffy, an annotation language for specifying the on-disk format of a file system. File system developers annotate the data structures of a file system, and we use these annotations to generate a library that allows identifying, parsing and traversing file-system metadata, providing support for both offline and online storage applications. This approach simplifies the development of storage applications that work across different file systems because it reduces the amount of file-system specific code that needs to be written.

We have written annotations for many modern Linux file systems, and developed several applications for these file systems, including a type-specific metadata corruptor, a file system converter, and an online storage layer cache that preferentially caches files for certain users. Our experiments show that applications that use the library to access file system metadata can achieve good performance and are robust against file system corruption errors.

Learning Objectives:
1. Understanding potential bugs that arise with ad-hoc processing of file system metadata
2. The Spiffy annotation language provides the missing information that allows for full specification of the format of the file system
3. Using the Spiffy generated library to help simplify development of file-system aware applications and also to make them more robust

Watch video: