SNIA Developer Conference September 15-17, 2025 | Santa Clara, CA
Standards-based Data Platforms for HPC and AI Data Architectures
Abstract
The rapid evolution of GPU computing in the Enterprise has led to unprecedented demand for robust and scalable data platforms. This session will explore the critical role that standardized frameworks and protocols play in optimizing data creation, processing, collaboration and storage within these advanced computing environments. Attendees will gain insights into how adopting standards can enhance interoperability, move data to compute, facilitate efficient data exchange, and ensure seamless integration across diverse systems and applications. By leveraging case studies and real-world implementations, the session will demonstrate how standards-based approaches can drive significant performance improvements and operational efficiencies in HPC and AI data architectures.
Learning Objectives
Related Sessions
MAINSTAGE "Industry Visionary" Presentation: Maximizing the Benefits of QLC Flash with Pure's Hyperscale Solution for Applications Ranging from AI/ML to HDD Displacement
As data growth accelerates in the age of AI, hyperscalers demand higher-capacity storage solutions that deliver a balanced combination of performance, power, and cost-effectiveness. In this session, we will present how QLC NAND-based Direct Flash Modules (DFM), in conjunction with Pure’s advanced software architecture, allows hyperscalers to take advantage of high-density, higher-performance, and reliable SSD storage compared to HDDs. Pure’s intelligent flash management software abstracts the host software from the underlying NAND technology, enabling faster adoption of newer NAND technologies and providing flexibility with different NAND vendors. With a configurable solution, Pure’s offering enables the adoption of the same solution across multiple use cases, from AI/ML to HDD displacement.
Will AI Stay Dumb for the Lack of Memory and Storage?
The rapid evolution of artificial intelligence (AI) technologies is precipitating a profound transformation in data storage requirements, highlighting a potential bottleneck in AI advancement due to insufficient memory and storage capacities. This presentation examines the interplay between AI development and data storage technologies, focusing on the growing disparity between their respective growth rates.
Current AI clusters are experiencing a doubling in computing speed approximately every two months, a pace that starkly contrasts with the 3-5 year doubling time of contemporary data storage technologies. This discrepancy is generating significant challenges, as the demand for data storage surges in tandem with the proliferation of AI applications. Notably, recent trends indicate that the price per terabyte (TB) for solid-state drives (SSD), hard disk drives (HDD), and tape storage has increased by 15-35%, driven by the burgeoning appetite for data storage solutions spurred by language-based generative AI models such as ChatGPT.
The demand for data storage is further exacerbated by AI's capacity to generate vast quantities of images and videos, necessitating even greater storage capabilities. AI systems, particularly those involved in training complex models, require fast-access SSDs for efficient data processing and HDDs for mid-term data retention (up to five years) to continuously refine and improve these models. Additionally, long-term cold storage is becoming increasingly critical for AI applications that rely on extensive historical data, such as autonomous driving. Here, training data must be preserved for decades, encompassing development, production, and operational phases.
Beyond autonomous driving, other AI applications, including drug design, healthcare, and aviation, also necessitate long-term data storage solutions. These fields require the retention of vast datasets over extended periods, underscoring the critical need for advancements in storage technology to keep pace with AI's accelerating computational demands.
This presentation aims to shed light on the urgent need for innovative storage solutions to sustain AI's growth trajectory and explores potential avenues for bridging the gap between AI's computational power and data storage capabilities. By addressing these challenges, we can ensure that AI continues to evolve and unlock its full potential without being hindered by storage limitations.
Disaggregated Memory for High Performance Computing; Architectures and Opportunities for CXL
HPE pioneered the concept of "memory-driven computing," spurred by its invention of memristor ReRAM, practical mid-board optics technology, and the GenZ interconnect. This presentation looks at some of that research and the direction it has taken since the consolidation of OpenCAPI, CCIX, and GenZ into the CXL specification and the failure of storage-class memory to gain a sustainable commercial foothold. We will examine current HPC architectures, including current HPE Cray EX systems such as the Frontier system at Oakridge National Laboratory and the Aurora system at Argonne National Laboratory--presently the #1 and #2 systems on the Top500 list--and look at future opportunities for emerging technologies, such as CXL memories and computational storage, in next-generation HPC architectures.
Project Silica: The Future of Sustainable Archival Data Storage
Sustainable and cost-effective long-term storage remains an unsolved problem. The most widely used enterprise storage technologies today are magnetic (hard disk drives and tape). They use media that degrades over time and has a limited lifetime, which leads to inefficient, wasteful, and costly solutions for long-lived data. This talk presents Silica: the first cloud storage system for archival data underpinned by quartz glass, an extremely resilient media that allows data to be left in place indefinitely. The hardware and software of Silica have been co-designed and co-optimized from the media up to the service level with sustainability as a primary objective. This design follows a cloud-first, data-driven methodology underpinned by principles derived from analysing the archival workload of a large public cloud service. Silica can support a wide range of archival storage workloads and ushers in a new era of sustainable, cost-effective storage.
Standards-based Data Platforms for HPC and AI Data Architectures
The rapid evolution of GPU computing in the Enterprise has led to unprecedented demand for robust and scalable data platforms. This session will explore the critical role that standardized frameworks and protocols play in optimizing data creation, processing, collaboration and storage within these advanced computing environments. Attendees will gain insights into how adopting standards can enhance interoperability, move data to compute, facilitate efficient data exchange, and ensure seamless integration across diverse systems and applications. By leveraging case studies and real-world implementations, the session will demonstrate how standards-based approaches can drive significant performance improvements and operational efficiencies in HPC and AI data architectures.
