SPDK Schedulers – Saving CPU cores in a polled mode storage application

Polled mode applications such as the Storage Performance Development Kit (SPDK) NVMe over Fabrics target can demonstrate higher performance and efficiency compared to applications with a more traditional interrupt-driven threading model. But this performance and efficiency comes at a cost of increased CPU core utilization when the application is lightly loaded or idle. This talk will introduce a new SPDK scheduler framework which enables transferring work between CPU cores for purposes of shutting down or lowering frequency on cores when under-utilized. We will first describe the SPDK architecture for lightweight threads. Next, we will introduce the scheduler framework and how a scheduler module can collect metrics on the running lightweight threads to make scheduling decisions. Finally, we will share initial results comparing SPDK NVMe-oF target performance and CPU efficiency of a new scheduler module based on this framework with the default static scheduler.

• Understand how work is scheduled in an SPDK polled mode application such as the NVMe over Fabrics target
• Understand how SPDK scheduler modules under the new framework can decide if and when to move work between CPU cores
• Understand how performance + CPU efficiency compare between the default scheduler and scheduler implemented in the new framework