SNIA Developer Conference September 15-17, 2025 | Santa Clara, CA
The rapid advancement of AI is significantly increasing demands on compute, memory and the storage infrastructure. As NVMe storage evolves to meet these needs, it is experiencing a bifurcation in requirements. On one end, workloads such as model training, checkpointing, and key-value (KV) cache tiering are driving the need for line-rate saturating SSDs with near-GPU and HPC attachment. On the other end, the rise of multi-stage inference, synthetic data generation, and post-training optimization is fueling demand for dense, high-capacity disaggregated storage solutions — effectively displacing traditional rotating media in the nearline tier of the datacenter. This paper explores the architectural considerations across both ends of this spectrum, including Gen6 performance, indirection unit (IU) selection, power monitoring for energy efficiency, liquid cooled thermal design, and strategies for enabling high capacity through form factor and packaging choices. We demonstrate how thoughtful design decisions can unlock the full potential of storage systems in addressing the evolving challenges of AI workloads.
The rapid advancement of AI is significantly increasing demands on compute, memory and the storage infrastructure. As NVMe storage evolves to meet these needs, it is experiencing a bifurcation in requirements. On one end, workloads such as model training, checkpointing, and key-value (KV) cache tiering are driving the need for line-rate saturating SSDs with near-GPU and HPC attachment.
Abstract With its introduction in 2011, the NVMe storage protocol has allowed CPUs to handle more data with less latency. This in turn has significantly improved the CPU's ability to manage parallel tasks with multiple queues while improving CPU utilization rates. More recently, the growing relevance of GPUs in AI training and inference has led to innovations that illustrate enabling NVMe storage access directly from GPUs. In this presentation we discuss some challenges with doing this efficiently.
Should SSDs supporting power states higher than the maximum TDP dissipation supportable in a system? Many industry standards for drive form factors are targeting =25W, but will Gen6 SSDs be viable in a x4 configuration or will these form factors be abandoned? What is proposed is a framework currently supported in NVMe and OCP's Datacenter NVMe SSD Specification of allowing enhanced latencies in cases where there is thermal margin above the maximum TDP of the form factor using either host orchestrated NVMe power state management or device orchestrated Host Controlled Thermal Management.
