SNIA Podcasts

Storage Developer Conference Podcast Series

Every week the Storage Developer Conference (SDC) podcast series presents important technical topics to the Storage Developer community. Each episode is hand selected by the SNIA Technical Council from the presentations at SDC.

This week’s highlighted Podcast: #122: 10 Million I/Ops From a Single Thread by Benjamin Walker, Software Engineer, Intel (MP3, Video, PDF of Slides)

One of the most common benchmarks in the storage industry is 4KiB random read I/O per second. Over the years, the industry first saw the publication of 1M I/Ops on a single box, then 1M I/Ops on a single thread (by SPDK). More recently, there have been publications outlining 10M I/Ops on a single box using high performance NVMe devices and more than 100 CPU cores.

This talk will present a benchmark of SPDK performing more than 10 million random 4KiB read operations per second from a single thread to 20 NVMe devices, a large advance compared to the state of the art of the industry. SPDK has developed a number of novel techniques to reach this level of performance, which will be outlined in detail here. These techniques include polling, advanced MMIO doorbell batching strategies, PCIe and DDIO considerations, careful management of the CPU cache, and the use of non-temporal CPU instructions. This will be a low level talk with real examples of eliminating data dependent loads, profiling last level cache misses, pre-fetching, and more. Additionally, there remains a number of techniques that have not yet been employed that warrant future research. These techniques often push devices outside of their original intended operating mode, while remaining within the bounds of the specification, and so often require collaboration between NVMe controller and device designers, the NVMe specification body, and software developers such as the SPDK team.

Learning Objectives:
1. Optimal use of NVMe devices
2. Optimal use of PCIe and MMIO in a storage stack
3. Leveraging advanced x86-64 CPU instructions and making best use of the CPU cache

The Podcast can be downloaded at iTunes. Download a copy of the original presentation slides as well.



Note: SNIA Podcasts are provided as information only. Articles and guests that appear on any SNIA programs express the opinions, experience, technical positions, and research of those authors and guests and not necessarily those of the SNIA.

SNIA Podcasts Copyright 2007 -  2019. All rights reserved. No part of any SNIA Podcast may be reproduced, stored in a retrieval system, or transcribed, in any form or by any means, photocopying, electronic, mechanical recording or otherwise for personal or professional financial gain, without prior permission in writing of the publisher, The Storage Networking Industry Association, 4410 ArrowsWest Drive, Colorado Springs, CO 80907-3444 USA