Massively Scalable File Storage (2016)

Internet changed the world and continues to revolutionize how people are connected, exchange data and do business. This radical change is one of the cause of the rapid explosion of data volume that required a new data storage approach and design. One of the common element is that unstructured data rules the IT world. How famous Internet services we all use everyday can support and scale with thousands of new users added daily and continue to deliver an enterprise-class SLA ? What are various technologies behind a Cloud Storage service to support hundreds of millions users ? This tutorial covers technologies introduced by famous papers about Google File System and BigTable, Amazon Dynamo or Apache Hadoop. In addition, Parallel, Scale-out, Distributed and P2P approaches with Lustre, PVFS and pNFS with several proprietary ones are presented as well. This tutorial adds also some key features, such erasure coding, essential at large scale to help understand and differentiate industry vendors offering.

Learning Objectives

Understand various technologies around File Storage at megascale
Anticipate the recent technology wave around distributed storage with design based on Google or Amazon research papers
Receive key elements and arguments to select the right solution for various needs