File Systems and File Management

Material on this page is intended solely for the purpose of content review by SNIA members. Tutorial material may be read and commented upon by any SNIA member, but may not be saved, printed, or otherwise copied, nor may it be shared with non-members of the SNIA. Tutorial managers are responsible for responding to all comments made during the open review period. No responses will be given to comments made outside the open review period.

Jump straight to an abstract:

Object Storage Technology

The Abstracts

Object Storage Technology
Brent Welch
Download

Object Storage is a generalized data container with uses in cloud storage, HPC file systems, and custom applications that roll their own indexing and metadata layers over objects. This tutorial provides a survey of these different kinds of objects, their APIs, and the applications that use them. The OSD (Object Storage Device) command set for SCSI provides a secure, general purpose mechanism used in HPC file systems via the NFSv4.1 pNFS (parallel NFS) protocol. The Amazon S3 object interface uses a web-based transport to provide cloud-based storage, and there are a number of similar interfaces in the open source community such as Swift and Eucalyptus. The differentiating features of object storage systems include the access protocols (SCSI, RPC, REST), performance (high-speed LAN or WAN), security mechanisms, replication and reliability schemes, metadata and indexing services. As a result, different application domains have evolved their own Object Storage ecosystems.

Learning Objectives

Understand the OSD and NFSv4.1 pNFS standards
Understand web objects and their “REST-ful” APIs
Understand the tradeoffs between POSIX file system semantics and cloud storage semantics