Fine-Grained Latency Measurement in Large-Scale Storage Systems

Benchmarks lie. They report aggregates that misrepresent what’s really going on within a storage system.
Modern storage controllers have complex data paths that include myriad peripherals like DMA controllers, co-processors, heterogeneous data buses and other data processing engines. Optimizing these data paths can be a challenge given that each peripheral presents its own unique timing to the system, creating hidden performance bottlenecks. Aggregate benchmarks just don’t provide the kind of information needed to identify the bottlenecks and correct the design.

This presentation discusses a different solution to latency measurement within a mixed microprocessor/FPGA storage controller. The presentation will show the tools used and provide examples of practical and applicable representations of latency data. It will discuss lessons learned and performance gains realized by our developers through precise data plane latency measurements.