Data Integrity Support for Silent Data Corruption in Gfarm File System

Data stored in storage are often corrupted silently without any explicit error. To cope with the silent data corruption, the file system level detection is effective; Btrfs and ZFS have a mechanism to detect it by adding checksum in each block. However, data replication is required to correct the corrupted data, which waste storage capacity in local file system. Gfarm file system is an open-source distributed file system, which federates storages even among several institutions in wide area. It supports file replicas to improve access performance and also fault tolerance. The number and the locations of file replicas are specified by an extended attribute of a directory or a file. This presentation describes a design and implementation of data integrity support for silent data corruption in Gfarm file system. Due to native and required feature of file replicas in Gfarm file system, the silent data corruption can be detected efficiently and the corrupted data can be corrected without any waste storage capacity.