Accelerated Erasure Coding: The New Frontier of Software Defined Storage

Author(s)/Presenter(s):
Library Content Type:
Publish Date: 
Tuesday, September 25, 2018
Event Name: 
Focus Areas:
Abstract: 

Efficient storage is critical to the success of datacenters and the functioning of enterprises. The exponential growth in the volume of data is forcing CIOs to rethink their storage strategies. One challenge they face is finding a replacement for aging RAID technology, which falls short in extreme I/O performance, data protection and resiliency.

A solution is erasure coding (EC), which is becoming the preferred choice for data protection in large datacenters. Erasure codes have evolved from traditional Reed-Solomon algorithm to more sophisticated locally recoverable and regenerating codes that can perform more efficient data recovery. Erasure codes are compute-intensive and impose higher resource cost on distributed storage system solutions.
Our approach to above problem is a hardened Ceph based distributed storage solution based on modular and scalable EC-offload-engine (ECoE) library. The ECoE, comprising of new age EC algorithms on general-purpose graphics processing units (GPUs) and can provide improvements of up to 40% in encode/decode processes depending on the algorithm. This presentation will discuss the Ceph erasure code plugin infrastructure alongside the design of ECoE parallel algorithms on GPUs. It will as show the performance and storage characteristics of ECoE algorithms in comparison to traditional EC algorithms.

Learning Objectives:
1. Erasure codes and Ceph erasure code plugin infrastructure
2. ECoE erasure code algorithms and implementation on GPUs
3. Performance and cost analysis of ECoE algorithms on Ceph

Watch video: