Object Store Performance: Benchmarking & Troubleshooting

Author(s)/Presenter(s):
Library Content Type:
Publish Date: 
Friday, December 4, 2020
Event Name: 
Event Track:
Focus Areas:
Abstract: 

Object storage is increasingly getting used on premises. The reasons tend to vary across use cases, but generally include performance and security concerns. In this talk we will first briefly talk about some of the use cases that customers are using object storage on premises and the corresponding performance expectations.

Next we talk about our journey to identify the right set of tools for benchmarking an Object Store and why we ended up developing a home grown benchmark. We will talk about its capabilities and why we developed them. Our tool Poseidon benchmark is fully distributed, supports a wide variety of configurations to simulate different workloads. This allows us to control the compressibility of data generated, object size distribution, and different IO patterns. It also allows adding pluggable modules that can talk to various internal layers of the software stack to debug performance bottlenecks in different layers.

In the third part of the talk, we will discuss the performance bottlenecks that were found out using the Poseidon Benchmark and the optimizations we did to address the bottlenecks. The optimizations include vNIC performance improvements, golang garbage collection and associated high cpu utilization, reducing memory fragmentation, and caching.

Watch video: