Achieving 10-Million IOPS from a single VM on Windows Hyper-V

Many server workloads, for example OLTP database workloads, require high I/O throughput and low latency. With the industry trend of moving high-end scale-up workloads to virtualization environment, it is essential for cloud providers and on-premises servers to achieve near native performance by reducing I/O virtualization overhead which mainly comes from two sources: DMA operations and interrupt delivery mechanism for I/O completions. The direct PCIe NVMe device assign techniques allow a VM to interact with HW devices directly and avoid using traditional Hyper-V para-virtualized I/O path. To improve interrupt handling in the virtualization environment, Intel introduces Posted Interrupts (PI) as an enhanced method to mitigate interrupt delivery overhead in a virtualized environment, bypassing hypervisor involvement completely. In this talk, we will present Microsoft implementation and optimization of Intel PI and Hyper-V direct PCIe NVMe access on Windows platform. The results showed that we were able to achieve more than 10-Million IOPS from a single VM for the first time in the industry using an Intel Skylake based HPE commodity server with these techniques.