Lessons Learned Implementing Cross-protocol Compatibility Layer

Over the past year, we have integrated our storage solution with a number of cloud and object storage APIs, including Amazon S3, WebDAV, OpenStack, and HDFS. While these protocols share much commonality, they also differ in meaningful ways which complicates the design of a cross-protocol compatibility layer. In this presentation, we detail how the various storage protocols are the same, how they differ, and what design decisions were necessary to build an underlying storage API that meets the requirements to support all of them. Further, we consider the lessons learned and provide recommendations for developing cloud storage APIs such as CDMI.

Learning Objectives

Features of the various popularly used cloud storage protocols
Differences between the protocols, how they authenticate, how they handle multi-part objects, how they guarantee integrity
Overview of the design decisions we made to support all these protocols on top of the same underlying storage