Ozone, the story of a billion keys

Object stores are known for ease of use and massive scalability. Unlike other storage solutions like file systems and block stores, object stores are capable of handling data growth without increase in complexity or developer intervention. Apache Hadoop Ozone is a highly scalable Object Store and is a spiritual successor of HDFS. It can store billions of keys and hundreds of petabytes of data. With the massive scale there is a requirement for it to have very high throughput while maintaining low latency.

This talk discusses the Ozone architecture and design decisions which were significant for achieving high throughput and low latency. With petabytes of data and billions of keys Ozone has a scalable metadata layer. The talk will detail how Ozone supports this layer without compromising throughput or latency. Such a massive scale requires Ozone to be scalable in terms of client connections and amount of data read and written to the store. The talk will discuss the challenges faced and the corresponding design solutions. Also it would touch upon Ozone’s goal of reaching trillion objects and possible challenges.