Accelerated Erasure Coding - The new frontier of Software Defined Storage

Efficient storage is critical to the success of datacenters and the functioning of enterprises. The exponential growth in the volume of data is forcing CIOs to rethink their storage strategies. One challenge they face is finding a replacement for aging RAID technology, which falls short in extreme I/O performance, data protection and resiliency. A solution is erasure coding (EC), which is becoming the preferred choice for data protection in large datacenters. Erasure codes have evolved from traditional Reed-Solomon algorithm to more sophisticated locally recoverable and regenerating codes that can perform more efficient data recovery. Our approach to above problem is a hardened Ceph based distributed storage solution based on modular and scalable EC-offload-engine (ECoE) library. The ECoE, comprising of new age EC algorithms on general-purpose graphics processing units (GPUs) and can provide improvements of up to 40% in encode/decode processes depending on the algorithm.