Big Data Analytics on Object Stoage - Hadoop Over Ceph Object Storage with SSD Cache | SNIA

Abstract

Cloud object store provides the ability to store objects across multiple datacenters over a straightforward HTTPS REST API. The namespace is hierarchical and can be searched. Objects can be arbitraiy large and numerous. The deployment can also be done on a commodity-harware based. This makes them an attractive option for archiving large amounts of data that are produced in science and industry. To analyze the data, advanced analytics such as MapReduce can be used. However, copying the data from the object store into distributed file system that the analytics system requires directly on object stores greatly improves usability and performance. In this work, we study the possibility of running Hadoop over Ceph Object Storage and identify common problems.