SNIA Developer Conference September 15-17, 2025 | Santa Clara, CA
Lokesh Jain
Senior Software Engineer
Cloudera Inc
Lokesh Jain is a Senior Software Engineer at Cloudera. He is an early developer of Apache Ozone(Object Store) and Apache Ratis project(Raft Consensus Protocol implementation) and has been contributing for the past 4 years. He holds committer and PMC privileges for Apache Hadoop, Apache Ozone and Apache Ratis. He has pursued M.Sc.(Hons.) Mathematics and B.E.(Hons.) Computer Science from BITS Pilani. Lokesh has experience with distributed computing, data replication, data pipeline and storage systems.
Related Sessions
Ozone - Architecture and Performance at billions’ scale
Object stores are known for ease of use and massive scalability. Unlike other storage solutions like file systems and block stores, object stores are capable of handling data growth without increase in complexity or developer intervention. Apache Hadoop Ozone is a highly scalable Object Store which extends the design principles of HDFS while maintaining a 10-100x scale compared to HDFS. It can store billions of keys and hundreds of petabytes of data. With the massive scale there is a requirement for it to have very high throughput while maintaining low latency. This talk discusses the Ozone architecture and design decisions which were significant for achieving high throughput and low latency. It will detail how Ozone supports metadata and data layer without compromising throughput or latency. Further the talk discusses the hardships and challenges related to resource management and good performance in Ozone. It would cover some major pain points and present the performance issues in broad categories.
Apache Ozone - Balancing and Deleting Data At Scale
Apache Ozone is an object store which scales to tens of billions of objects, hundreds of petabytes of data and thousands of datanodes. Ozone not only supports high throughput data ingestion but also supports high throughput deletion with performance similar to HDFS. Further with massive scale the data can be non-uniformly distributed due to addition of new datanodes, deletion of data etc. Non-uniform distribution can lead to lower utilisation of resources and can affect the overall throughput of the cluster. The talk discusses the balancer service in Ozone which is responsible for uniform distribution of data across the cluster. It would cover the service design and how the service improves upon HDFS balancer service. Further the talk discusses how Ozone deletion matches the HDFS deletion performance of deleting 1 Million keys / hour but can scale much more. Simple design and asynchronous operations enable Ozone to achieve the scale for deletion. The talk would dive deeper into the design and performance enhancements.