Low-Overhead Flash Disaggregation via NVMe-over-Fabrics

In this presentation we revisit NVMe-SSD disaggregation, using NVMe-over-Fabrics (NVMe-oF) as the remote storage protocol. We show that NVMe-oF reduces the overheads of remote access to a bare minimum, thus greatly increasing the efficiency and viability of NVMe disaggregation. Specifically, while recent works showed that SSD storage disaggregation via iSCSI degrades application-level throughput by 20%, we show negligible performance degradation with NVMe-oF.

We will present detailed performance characterization, using synthetic and real-world application. First, we use synthetic stress-tests to characterize and break down NVMe-oF latency and throughput overheads, and compare them to those of iSCSI. Then, we characterize real-world application disaggregation (RocksDB, MySQL) and show that they experiences minimal performance degradation when running on remote storage with NVMe-oF. We also show that NVMe-oF’s low processing overhead enables better scaling at the storage server-side compared to iSCSI, further improving cost efficiency.