Challenges with Persistent Memory in Distributed Storage Systems

Persistent memory will significantly improve storage performance. But these benefits can be harder to realize in distributed storage systems such as Ceph or Gluster. In such architectures, several factors mitigate gains from faster storage. They include costly network overhead inherent for many operations, and deep software layers supporting necessary services. It is also likely that the high costs of persistent memory will limit deployments in the near term. This talk shall explore steps to tackle those problems. Strategies include tiering subsets of the data or metadata in persistent memory, incorporating high speed networks such as infiniband, and the use of tightly optimized “thin” file systems. The talk will include discussion and results from a range of experiments in software and hardware.

Learning Objectives

Understand bottlenecks in scale-out storage when persistent memory is used
Learn the benifits and limits of faster networks in such configurations
Understand the current stacks and where improvements can be made