New Encoding Technique to Reform Erasure Code Data Overwrite

Besides data creation and recovery, data overwrite is also a foundation of erasure code. , When encoding for data overwrite, existing erasure code will cause large data transmission, heavy computation and complex software logic. It already becomes a barrier to broaden erasure code’s usage in the industry. This talk will reveal a distinctive read-modify-write encoding technique, we call it as overwrite-encoding. We will explain the underlying mechanism about how this technique avoids full stripes read and re-encode when overwriting stored data. We will also give a guidance on how to use it to simplify software logic of data overwrite.
The new method is compatible with kinds of erasure code. It can be applied into existing storage applications, e.g. Swift, Ceph, and then enhance their erasure code modules. As a result, more than half of data transmission, major of computation and complexity software stack are reduced. Overwrite-encoding would be a key for erasure code to break its cage of backup or archive storage.