In a little over a month, more than 1,500 people have viewed the SNIA Cloud Storage Technologies Initiative (CSTI) live webinar, “Ceph: The Linux of Storage Today,” with SNIA experts Vincent Hsu and Tushar Gohad. If you missed it, you can watch it on-demand at the SNIA Educational Library. The live audience was extremely engaged with our presenters, asking several interesting questions. As promised, Vincent and Tushar have answered them here. Given the high level of this interest in this topic, the CSTI is planning additional sessions on Ceph. Please follow us @sniacloud_com or at SNIA LinkedIn for dates. Q: How many snapshots can Ceph support per cluster? Q: Does Ceph provide Deduplication? If so, is it across objects, file and block storage?  A: There is no per-cluster limit. In the Ceph filesystem (cephfs) it is possible to create snapshots on a per-path basis, and currently the configurable default limit is 100 snapshots per path. The Ceph block storage (rbd) does not impose limits on the number of snapshots.  However, when using the native Linux kernel rbd client there is a limit of 510 snapshots per image. There is a Ceph project to support data deduplication, though it is not available yet. Q: How easy is the installation setup? I heard Ceph is hard to setup.  A: Ceph used to be difficult to install, however, the ceph deployment process has gone under many changes and improvements.  In recent years, the experience has been very streamlined. The cephadm system was created in order to bootstrap and manage the Ceph cluster, and Ceph also can now be deployed and managed via a dashboard.  Q: Does Ceph provide good user-interface to monitor usage, performance, and other details in case it is used as an object-as-a-service across multiple tenants? A: Currently the Ceph dashboard allows monitoring the usage and performance at the cluster level and also at a per-pool basis. This question falls under consumability.  Many people contribute to the community in this area. You will start seeing more of these management tool capabilities being added, to see a better profile of the utilization efficiencies, multi-tenancy, and qualities of service. The more that Ceph becomes the substrate for cloud-native on-premises storage, the more these technologies will show up in the community. Ceph dashboard has come a long way.   Q: A slide mentioned support for tiered storage. Is tiered meant in the sense of caching (automatically managing performance/locality) or for storing data with explicitly different lifetimes/access patterns? A: The slide mentioned the future support in Crimson for device tiering. That feature, for example, will allow storing data with different access patterns (and indeed lifetimes) on different devices. Access the full webinar presentation here.  Q: Can you discuss any performance benchmarks or case studies demonstrating the benefits of using Ceph as the underlying storage infrastructure for AI workloads?    A: The AI workloads have multiple requirements that Ceph is very suitable for:

• Performance: Ceph can provide the high performance demands that AI workloads can require. As a SDS solution, it can be deployed on different hardware to provide the necessary performance characteristics that are needed. It can scale out and provide more parallelism to adapt to increase in performance demands. A recent post by a Ceph community member showed a Ceph cluster performing at 1 TiB/s.
• Scale-out: Scale was built from the bottom up as a scale out solution. As the training and inferencing data is growing, it is possible to grow the cluster to provide more capacity and more performance. Ceph can scale to thousands of nodes.
• Durability: Training data set sizes can become very large and it is important that the storage system itself takes care of the data durability, as transferring the data in and out of the storage system can be prohibitive. Ceph employs different techniques such as data replication and erasure coding, as well as automatic healing and data re-distribution to ensure data durability
• Reliability: It is important that the storage systems operate continuously, even as failures are happening through the training and inference processing. In a large system where thousands of storage devices failures are the norm. Ceph was built from the ground up to avoid a single point of failure, and it can continue to operate and automatically recover when failures happen.

Object, Block, File support: Different AI applications require different types of storage. Ceph provides both object, block, and file access. Q: Is it possible to geo-replicate Ceph datastore? Having a few Exabytes in the single datacenter seems a bit scary.  A: We know you don’t want all your eggs in one basket. Ceph can perform synchronous or asynchronous replication. Synchronous replication is especially used in a stretch cluster context, where data can be spread across multiple data centers. Since Ceph is strongly consistent, stretch clusters are limited to deployments where the latency between the data center is relatively low. For example, stretch clusters are in general, shorter distance, i.e., not beyond 100-200 km.  Otherwise, the turnaround time would be too long. For longer distances for geo-replication, people typically perform asynchronous replication between different Ceph clusters. Ceph also supports different geo replication schemes. The Ceph Object storage (RGW) provides the ability to access data in multiple geographical regions and allow data to be synchronized between them. Ceph RBD provides asynchronous mirroring that enables replication of RBD images between different Ceph clusters. The Ceph filesystem provides similar capabilities, and improvements to this feature are being developed.  Q: Is there any NVMe over HDD percentage capacity that has the best throughput?  For example, for 1PB of HDD, how much NVMe capacity is recommended? Also, can you please include a link to the Communities Vincent referenced? A: Since NVMe provides superior performance to HDD, the more NVMe devices being used, the better the expected throughput. However, when factoring in cost and trying to get better cost/performance ratio, there are a few ways that Ceph can be configured to minimize the HDD performance penalties. The Ceph documentation recommends that in a mixed spinning and solid drive setup, the OSD metadata should be put on the solid state drive, and it should be at least in the range of 1-4% of the size of the HDD. Ceph also allows you to create different storage pools that can be built by different media types in order to accommodate different application needs. For example, applications that need higher IO and/or higher data throughput can be set to use the more expensive NVMe based data pool, etc. There is no hard rule.  It depends on factors like what CPU you have. What is seen today is that users tend to implement all Flash NVMe, but not a lot of hybrid configurations. They’ll implement all Flash, even object storage block storage, to get consistent performance. Another scenario is using HDD for high-capacity object storage for a data repository. The community and Ceph documentation have the best practices, known principles and architecture guidelines for a CPU to hard drive ratio or a CPU to NVMe ratio. The Ceph community is launching a user council to gather best practices from users, and involves two topics: Performance and Consumability If you are a user of Ceph, we strongly recommend you join the community and participate in user council discussions. https://ceph.io/en/community/ Q: Hardware RAID controllers made sense on few CPU cores systems. Can any small RAID controller compete with massive core densities and large memory banks on modern systems? A: Ceph provides its own durability, so in most cases there is no need to also use a RAID controller. Ceph can provide durability leveraging data replication and/or erasure coding schemes. Q: I would like to know if there is a Docker version for Ceph? What is the simple usage of Ceph? A: Full fledged Ceph system requires multiple daemons to be managed, and as such a single container image is not the best fit. Ceph can be deployed on Kubernetes via Rook. There have been different experimental upstream projects to allow running a simplified version of Ceph. These are not currently supported by the Ceph community. Q: Does Ceph support Redfish/ Swordfish APIs for management? Q: Was SPDK considered for low level locking? A: Yes, Ceph supports both Redfish and Swordfish APIs for management.  Here are example technical user guide references. https://docs.ceph.com/en/latest/hardware-monitoring/ https://www.snia.org/sites/default/files/technical_work/Swordfish/Swordfish_v1.0.6_UserGuide.pdf To answer the second part of your question, SeaStar, which follows similar design principals as SPDK, is used as the asynchronous programming library given it is already in C++ and allows us to use a pluggable network and storage stack—standard kernel/libc based network stack or DPDK, io_uring or SPDK, etc.  We are in discussion with the SeaStar community to see how SPDK can be natively enabled for storage access Q: Are there scale limitations on the number of MONs to OSDs, wouldn’t there be issues with OSDs reporting back to MON’s (epochs, maps) etc management based on number of OSDs? A: The issue of scaling the number of OSDs has been tested and addressed. In 2017 it was reported that CERN tested successfully a Ceph cluster with over 10,000 OSDs. Nowadays, the public Ceph telemetry shows regularly many active clusters in the range of 1,000-4,000 OSDs. Q: I saw you have support for NVMe/TCP.  Are there any plans for adding NVMe/FC support? A: There are no current plans to support NVMe/FC. Q: What about fault tolerance? If we have one out of 24 nodes offline, how possible is data loss? How can the cluster avoid request to down nodes? A: There are two aspects to this question: Data loss: Ceph has reputation in the market for its very conservative approach to protect the data. Once it approaches critical mass, Ceph will stop the writes to the system. Availability: This depends on how you configured it. For example, some users spread 6 copies of data across 3 data centers. If you lose the whole site, or multiple drives, the data is still available. It really depends on what is your protection design for that. Data can be set to be replicated into different failure domains, in which case it can be guaranteed that, unless there are multiple failures in multiple domains, there is no data loss.  The cluster marks and tracks down nodes and makes sure that all requests go to nodes that are available. Ceph replicates the data and different schemes can be used to provide data durability. It depends on your configuration, but the design principle of Ceph is to make sure you don’t lose data. Let’s say you have 3-way replication. If you start to lose critical mass, Ceph will go into read-only mode. Ceph will stop the write operation to make sure you don’t update the current state until you recover it. Q: Can you comment on Ceph versus Vector Database? A: Ceph is a unified storage system that can provide file, block, and object access. It does not provide the same capabilities that a vector database needs to provide. There are cases where a vector database can use Ceph as its underlying storage system.  Q: Is there any kind of support for parallel I/O on Ceph? A: Ceph natively performs parallel I/O. By default, it schedules all operations directly to the OSDs and in parallel. Q: Can you use Ceph with two AD domains? Let say we have a path /FS/share1/. Can you create two SMB shares for this path, one per domain with different set of permissions each? A: Partial AD support has been recently added to upstream Ceph and will be available in future versions. Support for multiple ADs is being developed. Q: Does Ceph provide shared storage similar to Gluster or something like EFS?  Also, does Ceph work best with many small files or large files? A: Yes, Ceph provides shared file storage like EFS. There is no concrete answer to whether many small files are better than large files. Ceph can handle either.  In terms of “what is best”, most of the file storage today is not optimized for very tiny files. In general, many small files would likely use more metadata storage, and are likely to gain less from certain prefetching optimizations. Ceph can comfortably handle large files, though this is not a binary answer.  Over time, Ceph will continue to improve in terms of granularity of file support. Q: What type of storage is sitting behind the OSD design?  VMware SAN? A: The OSD device can use any raw block device, e.g., JBOD.  Also, the assumption here is that every OSD, traditionally, is mapped to one disk. It could be a virtual disk, but it’s typically a physical disk. Think of a bunch of NVMe disks in a physical server with one OSD handling one disk. But we can have namespaces, for example, ZNS type drives, that allow us to do physical partitioning based on the type of media, and expose the disk as partitions. We could have one OSD to a partition. Ceph provides equivalent functionality to a VSAN. Each ceph OSD manages a physical drive or a subset of a drive. Q: How can hardware RAID coexist with Ceph? A: Ceph can use hardware RAID for its underlying storage, however, as Ceph manages its own durability, there is not necessarily additional benefit of adding RAID in most cases.  Doing so would duplicate the durability functions at a block level, reducing capacity and impacting performance. A lower latency drive could perform better. Most people use multiple 3-way replication or they just use erasure coding. Another consideration is you can run on any server instead of hard-coding for particular RAID adapters.              

SNIA Compute, Memory, and Storage Summit is where solutions, architectures, and community come together. Our 2024 Summit – taking place virtually on May 21-22, 2024 – is the best example to date, featuring a stellar lineup of 30 speakers in sessions on artificial intelligence, the future of memory, sustainability, critical storage security issues, the latest on CXL®, UCIe, and Ultra Ethernet, and more. “We’re excited to welcome executives, architects, developers, implementers, and users to our 12^th annual Summit,” said David McIntyre, Compute, Memory, and Storage Summit Chair and member of the SNIA Board of Directors. “Our event features technology leaders from companies like Dell, IBM, Intel, Meta, Samsung – and many more – to bring us the latest developments in AI, compute, memory, storage, and security in our free online event.  We hope you will attend live to ask questions of our experts as they present and watch those you miss on-demand.“ Artificial intelligence sessions sponsored by the SNIA Data, Networking & Storage Forum feature J Michel Metz of the Ultra Ethernet Consortium (UEC) on powering AI’s future with the UEC,  John Cardente of Dell on storage requirements for AI, Jeff White of Dell on edgenuity, and Garima Desai of Samsung on creating a sustainable semiconductor industry for the AI era. Other AI sessions include Manoj Wadekar of Meta on the evolution of hyperscale data centers from CPU centric to GPU accelerated AI, Paul McLeod of Supermicro on storage architecture optimized for AI, and Prasad Venkatachar of Pliops on generative AI data architecture. Memory sessions begin with Jim Handy and Tom Coughlin on how memories are driving big architectural changes. Ahmed Medhioub of Astera Labs will discuss breaking through the memory wall with CXL, and Sudhir Balasubramanian and Arvind Jagannath of VMware will share their memory vision for real world applications. Compute sessions include Andy Walls of IBM on computational storage and real time ransomware detection, JB Baker of ScaleFlux on computational storage real world deployments, Dominic Manno of Los Alamos National Labs on streamlining scientific workflows in computational storage, and Bill Martin and Jason Molgaard of the SNIA Computational Storage Technical Work Group on computational storage standards. CXL and UCIe will be featured with a CXL Consortium panel on increasing AI and HPC application performance with CXL fabrics and a session from Samsung and Broadcom on bringing unique customer value with CXL accelerator-based memory solutions. Richelle Ahlvers and Brian Rea of the UCI Express will discuss enabling an open chipset system with UCIe. The Summit will also dive into security with a number of presentations on this important topic. And there is much more, including a memory Birds-of-a-Feather session, a live Memory Workshop and Hackathon featuring CXL exercises, and opportunities to chat with our experts! Check out the agenda and register for free! The post 30 Speakers Highlight AI, Memory, Sustainability, and More at the May 21-22 Summit! first appeared on SNIA Compute, Memory and Storage Blog.

Measuring power efficiency in datacenter storage is a complex endeavor. A number of factors play a role in assessing individual storage devices or system-level logical storage for power efficiency. Luckily, our SNIA experts make the measuring easier! In this SNIA Experts on Data blog series, our experts in the SNIA Solid State Storage Technical Work Group and the SNIA Green Storage Initiative explore factors to consider in power efficiency measurement, including the nature of application workloads, IO streams, and access patterns; the choice of storage products (SSDs, HDDs, cloud storage, and more); the impact of hardware and software components (host bus adapters, drivers, OS layers); and access to read and write caches, CPU and GPU usage, and DRAM utilization. Join us on our final installment on the  journey to better power efficiency - Part 4: Impact of Storage Architectures on Power Efficiency Measurement. And if you missed our earlier segments, click on the titles to read them:  Part 1: Key Issues in Power Efficiency Measurement,  Part 2: Impact of Workloads on Power Efficiency Measurement, and Part 3: Traditional Differences in Power Consumption: Hard Disk Drives vs Solid State Drives.  Bookmark this blog series and explore the topic further in the SNIA Green Storage Knowledge Center. Impact of Storage Architectures on Power Efficiency Measurement Ultimately, the interplay between hardware and software storage architectures can have a substantial impact on power consumption. Optimizing these architectures based on workload characteristics and performance requirements can lead to better power efficiency and overall system performance. Different hardware and software storage architectures can lead to varying levels of power efficiency. Here's how they impact power consumption. Hardware Storage Architectures

1. HDDs v SSDs: Solid State Drives (SSDs) are generally more power-efficient than Hard Disk Drives (HDDs) due to their lack of moving parts and faster access times. SSDs consume less power during both idle and active states.
2. NVMe® v SATA SSDs: NVMe (Non-Volatile Memory Express) SSDs often have better power efficiency compared to SATA SSDs. NVMe's direct connection to the PCIe bus allows for faster data transfers, reducing the time components need to be active and consuming power. NVMe SSDs are also performance optimized for different power states.
3. Tiered Storage: Systems that incorporate tiered storage with a combination of SSDs and HDDs optimize power consumption by placing frequently accessed data on SSDs for quicker retrieval and minimizing the power-hungry spinning of HDDs.
4. RAID Configurations: Redundant Array of Independent Disks (RAID) setups can affect power efficiency. RAID levels like 0 (striping) and 1 (mirroring) may have different power profiles due to how data is distributed and mirrored across drives.

Software Storage Architectures

1. Compression and Deduplication: Storage systems using compression and deduplication techniques can affect power consumption. Compressing data before storage can reduce the amount of data that needs to be read and written, potentially saving power.
2. Caching: Caching mechanisms store frequently accessed data in faster storage layers, such as SSDs. This reduces the need to access power-hungry HDDs or higher-latency storage devices, contributing to better power efficiency.
3. Data Tiering: Similar to caching, data tiering involves moving data between different storage tiers based on access patterns. Hot data (frequently accessed) is placed on more power-efficient storage layers.
4. Virtualization Virtualized environments can lead to resource contention and inefficiencies that impact power consumption. Proper resource allocation and management are crucial to optimizing power efficiency.
5. Load Balancing: In storage clusters, load balancing ensures even distribution of data and workloads. Efficient load balancing prevents overutilization of certain components, helping to distribute power consumption evenly
6. Thin Provisioning: Allocating storage on-demand rather than pre-allocating can lead to more efficient use of storage resources, which indirectly affects power efficiency

Measuring power efficiency in datacenter storage is a complex endeavor. A number of factors play a role in assessing individual storage devices or system-level logical storage for power efficiency. Luckily, our SNIA experts make the measuring easier! In this SNIA Experts on Data blog series, our experts in the SNIA Solid State Storage Technical Work Group and the SNIA Green Storage Initiative explore factors to consider in power efficiency measurement, including the nature of application workloads, IO streams, and access patterns; the choice of storage products (SSDs, HDDs, cloud storage, and more); the impact of hardware and software components (host bus adapters, drivers, OS layers); and access to read and write caches, CPU and GPU usage, and DRAM utilization. Join us on our final installment on the  journey to better power efficiency – Part 4: Impact of Storage Architectures on Power Efficiency Measurement. And if you missed our earlier segments, click on the titles to read them:  Part 1: Key Issues in Power Efficiency Measurement,  Part 2: Impact of Workloads on Power Efficiency Measurement, and Part 3: Traditional Differences in Power Consumption: Hard Disk Drives vs Solid State Drives.  Bookmark this blog series and explore the topic further in the SNIA Green Storage Knowledge Center. Impact of Storage Architectures on Power Efficiency Measurement Ultimately, the interplay between hardware and software storage architectures can have a substantial impact on power consumption. Optimizing these architectures based on workload characteristics and performance requirements can lead to better power efficiency and overall system performance. Different hardware and software storage architectures can lead to varying levels of power efficiency. Here’s how they impact power consumption. Hardware Storage Architectures

1. HDDs v SSDs: Solid State Drives (SSDs) are generally more power-efficient than Hard Disk Drives (HDDs) due to their lack of moving parts and faster access times. SSDs consume less power during both idle and active states.
2. NVMe® v SATA SSDs: NVMe (Non-Volatile Memory Express) SSDs often have better power efficiency compared to SATA SSDs. NVMe’s direct connection to the PCIe bus allows for faster data transfers, reducing the time components need to be active and consuming power. NVMe SSDs are also performance optimized for different power states.
3. Tiered Storage: Systems that incorporate tiered storage with a combination of SSDs and HDDs optimize power consumption by placing frequently accessed data on SSDs for quicker retrieval and minimizing the power-hungry spinning of HDDs.
4. RAID Configurations: Redundant Array of Independent Disks (RAID) setups can affect power efficiency. RAID levels like 0 (striping) and 1 (mirroring) may have different power profiles due to how data is distributed and mirrored across drives.

Software Storage Architectures

1. Compression and Deduplication: Storage systems using compression and deduplication techniques can affect power consumption. Compressing data before storage can reduce the amount of data that needs to be read and written, potentially saving power.
2. Caching: Caching mechanisms store frequently accessed data in faster storage layers, such as SSDs. This reduces the need to access power-hungry HDDs or higher-latency storage devices, contributing to better power efficiency.
3. Data Tiering: Similar to caching, data tiering involves moving data between different storage tiers based on access patterns. Hot data (frequently accessed) is placed on more power-efficient storage layers.
4. Virtualization Virtualized environments can lead to resource contention and inefficiencies that impact power consumption. Proper resource allocation and management are crucial to optimizing power efficiency.
5. Load Balancing: In storage clusters, load balancing ensures even distribution of data and workloads. Efficient load balancing prevents overutilization of certain components, helping to distribute power consumption evenly
6. Thin Provisioning: Allocating storage on-demand rather than pre-allocating can lead to more efficient use of storage resources, which indirectly affects power efficiency

The post Power Efficiency Measurement – Our Experts Make It Clear – Part 4 first appeared on SNIA Compute, Memory and Storage Blog.

At our recent SNIA SCSI Trade Association Forum webinar, “Storage Trends 2024” our industry experts discussed new storage trends developing in the coming year, the applications and other factors driving these trends, and shared market data that illustrated the assertions. If you missed the live event, you can watch it on-demand in the SNIA Educational Library. Questions from the audience ranged from projections about the split between on-prem vs public cloud to queries about different technologies and terms such as NVMe, LTO tape, EDSFF and Cyber Storage. Here are answers to the audience’s questions. 

Q1: What is the future of HDDs? 

A1: HDDs are not in any hurry to depart the market. Despite high-capacity flash being out there, the cost per terabyte for HDDs are still very much in the platters’ favor. Not all jobs need to be done on flash. 

Q2: Are hard drives out in 2024? Is the world ready for full flash? 

A2. We are seeing a lot of flash for back up. Not necessarily to make your back up faster, but for something like instant restores or vehement instant recovery, to be able to run those on robust infrastructure is interesting. Also from the AI perspective, to be able to run AI jobs against backup data that doesn’t hit production or doesn’t require a big batch of copy data to be stood up so that we now have multiple copies of data, is an interesting use case. This is very much in the minority, but something we’re seeing. 

Q3 What will be the role of LTO tape in this future? 

A3: We’re seeing object deployed on tape. Quantum is doing this, a lot of clouds are doing this, despite how they may market it. Tape is still very much a thing, outside of today’s scope, but it is storage. When you look at some of these big libraries like DiamondBack from IBM, SpectraLogic has got several – both were very popular booths & technologies at the SC'23 industry event. The HPC installments are using tape, and I would imagine a massive chunk of the Fortune 500 are as well. 

Q4: What about NVMe HDDs? 

A4: The IDC chart about NVMe HDD penetration in the market didn’t even have NVMe HDDs on the line, but they show up at OCP at the storage sessions there. NVMe HDDs have been shown at industry events before, and StorageReview.com has written about them a couple of times. While they are not really a large presence in the market yet, they are certainly interesting and fun to think about. Like with EDSFF, where we are unifying on a connector, and it is interesting to think about a universal connector for drives. There’s certainly some interest. When you really step back, it is an interface change, but one of the reasons that’s a little slow to adopt, if you look at the earlier IDC chart about amount of capacity and the amount of infrastructure that’s currently in place, that is SAS-leveraged, it’s hard to displace that. While obviously there are some benefits around trying to unify around NVMe, over the long term there could be some TCO benefits. There is a very large installed SAS base, and the market continues to see value in that. That said, is it interesting, is it exciting – yes, you can see how some of these technologies may continue to evolve, and as flash does continue to become a bigger percentage of the ecosystem, it may help to further that along. There’s a lot of ecosystem that’s still built around SAS. 

Q5. Does it make sense for HDDs to stay at a SAS interface when SSDs are moving to NVMe? Wouldn't it make more sense to have HDDs with NVMe interface to leverage a single interface. 

A5. This is an active project within OCP. It simplifies design but also introduces new challenges, as in PCIe lane allocation in system designs. 

Q6: This question is for Jeff Janukowicz (IDC), any projections on the split between public cloud and on-prem AI storage? Can you share any trends with regards to on prem vs cloud-based data archival? 

A6: IDC has done some work, but hasn’t published anything around that quite yet. Obviously, the initial wave around AI is being driven by the public cloud vendors. IDC survey work suggests that the way this will ultimately start to play out will be that a lot of folks will then look to customize or build upon some of those publicly available models that are out there. Those are likely to move back on-prem, for compliance reasons or security reasons, or simply that people want to have that data in-house. When we say AI, it doesn’t mean that everything is going to the cloud. In this AI evolution, there will be a place for it in the cloud, on-prem and at the edge as well. The edge is where we will be collecting a lot of the data, when we think about inferencing and more, that will be done better at the edge. For client devices such as the PC, where Apple and Microsoft are pushing to integrate those AI features directly in the device, and next is mobile. The idea of “AI everywhere” will proliferate, and we are confident in saying yes, there will be a strong AI presence in on-prem data center as well. 

Q7: It seems "Cyber Storage" is a trend, is this just a feature of storage, or an entirely new product category? 

A7: It’s a term coined by Gartner, which is defined as doing threat detection and response in storage software or hardware. The SNIA Cloud Storage Technologies Initiative did a webinar on it. 

Q8: Is the storage demands trend mainly affected by Hyperscalers? If yes, what do you expect in enterprise on-premise infrastructure? 

A8: Demand from hyperscalers continues to be very large and represents most of the demand. Even in the IDC chart shown earlier around capacity optimization, that is from hyperscalers. This doesn’t mean that on-prem data centers are going away. People tend to leverage it either for security, compliance, or legacy reasons, so on-prem doesn’t go away. We see both continuing to co-exist and continuing to have value for different customers with different needs. Additionally, especially from the SSD development process, there is a strong desire by SSD manufacturers to make less variants of their drives. We’re seeing a desire to manufacture drives that go to hyperscalers, who typically buy in volume and dictate what gets made for the enterprise as a by-product. We are seeing more interest in having one skew or possibly a skew with a different firmware for hyperscalers and enterprise; if we get there, that efficiency should be positive for the overall market in terms of enabling SSD vendors to make that one product in scale, and then tune it a little bit for an enterprise vs a hyperscale use. There are potential efficiencies coming. 

Q9: There are so many form factors as part of "EDSFF", how can we really call it standard, when it seems like there are 10 to choose from, plus more to come? How are drive suppliers going to focus efforts on commonality with so many choices? 

A9: The standard wants to be able to accommodate all the use cases as possible, but there are many aspects that get defined with a lot of those aspects being optional. Then, just a handful of the those are defined as required. A couple more aspects become de facto standards, but there are several optional outliers that are choices for companies to develop around for more specific use cases. Initially what we are seeing is E.1S and E3.S are most likely going to be the most prevalent form factor versions, with E1.L and E3.S2T as maybe the next more common ones. There are a lot of variants that are defined just so companies can have options for specific use cases. 

Q10. QLC has been asserted by some as the end of HDD. Given the projections shown today, is the death of HDDs realistic? 

A10. QLC SSDs offer tremendous capacity gains over TLC SSDs and HDDs. They’re not less expensive per TB though, and that’s where HDDs will continue to hold a very critical spot. And while QLC SSDs are of course faster than HDDs, there are plenty of workloads where that speed simply isn’t needed. 

Q11. While QLC has been around for years, QLC chatter and activity seems to have really picked up over the last 6 months or so. Is QLC on the precipice of having meaningful shipping volume compared to TLC? What are the drivers? 

A11. For workloads that are heavily read-dependent (arguably, most workloads) QLC performance is on par with TLC. Even the endurance of QLC is more robust than most think, we’ve proven that out with our various Pi world record calculations. The density of course is another major benefit for QLC, 61.44TB in a U.2 form factor, even more in E1.L form factors. TLC will remain the go to for mixed or heavy write workloads, however. 

Q12. How do you think composable memory system? is it going be a trend? 

A12. Future versions of CXL promise composability and sharing of certain resources like DRAM, but this is still very fluid. 

Q13. It’s not just capacity. Performance is important. Why purchase a bunch of expensive GPUs just to have them idle while waiting for data? Speed is about keeping these assts highly utilized. 

A13. In our experience exploring and utilizing AI, the bottleneck in keeping GPUs busy is fabric, not storage performance. Once 800GbE gain proliferation, that math may change some and force Gen5/6 SSDs to be the default choice, but for now, it’s networking that’s limiting GPU utilization. Also remember, all AI is not created equal, and different use cases will have different storage performance needs, look at edge inferencing for a different model than data center training for instance. 

Q14. We are seeing a NAND tightening from our suppliers on availability of the larger capacity SSD drives on SAS and NVMe, what is worse is non=fips vs fips, what's up from your perspective. Thanks 

A14. The NAND flash industry continues to recover from the recent memory downturn and ramping up production.  Until the industry fully recovers, NAND flash supply (and SSDs) will remain tight. FIPS vs Non-FIPS, such a long lead time for getting FIPS and that’s affecting all of the industry. Getting certification is taking a long time, with 140-3. The storage industry generally values FIPS certification highly and are following the process closely. NIST is the organization managing. 

Q15. What power limits trends are seeing from devices being deployed in E3 form factor? The trend I see is the drive companies are continuing to consume more and more power. 

A15. E3 offers a variety of power envelopes based on each form factor, ranging up to 70 watts. As you go up in wattage, you are gaining in capacity and performance, but this needs to be managed carefully to create a sustainable balance with data center efficiency. Efficient performance per watt is the goal. 

Q16. Is AI accelerating the need for SSDs or is there something new needed? What's the biggest unique SSD requirement that isn't in all the other existing applications? Seems higher density, lower power, performance improvements are not new? 

A16. There is not much increased demand we can see directly tied to AI, but as AI becomes more deployed, we will likely see an increase in overall SSD demand, as well as high capacity SSDs. 

Q17. I have heard that storage as percentage of total IT spend has dropped significantly. As a lot of dollars are now going toward GPUs how will this trend respond? 

A17. There is an evolution from GPU spend supporting increased storage spend. 

About the Authors By Jeff Janukowicz, Research Vice President at IDC; Brian Beeler, Owner and Editor In Chief, StorageReview.com; and Cameron T. Brett, SNIA STA Forum Chair. Note to our readers: We had quite a few questions regarding SSDs, forecasting for SSDs, and comparing that with the future of HDDs. As a result, we are preparing a blog addressing these questions which will go into more detail.

Moving well beyond “fix it when it breaks,” AIOps introduces intelligence into the fabric of IT thinking and processes. The impact of AIOps and the shift in IT practices were the focus of a recent SNIA Cloud Storage Technologies Initiative (CSTI) webinar, “AIOps: Reactive to Proactive – Revolutionizing the IT Mindset.” If you missed the live session, it’s available on-demand together with the presentation slides, at the SNIA Educational Library. The audience asked several intriguing questions. Here are answers to them all: Q. How do you align your AIOps objectives with your company’s overall AI usage policy when it is still fairly restrictive in terms of AI use and acceptance? A.There are a lot of misconceptions on company policies and also what constitutes AI and the actual risk. So, there are several steps you can take:

• Understand the policy and intent
• Focus on low risk and high value use cases, for example, data used in IT management is often low risk and high value – e.g. metrics, or number of incidents or events
• Start with a well-controlled and small environment and show value
• Be transparent and demonstrate transparency. Even put human in the loop for a while.
• Maintain data governance – responsible data handling.
• Use industry’s best practices.

Q. What are the best AIOps tools in the market? A. There are many tools that claim to be an AIOps tool. But as the webinar shows, there is no single good tool and there will never be one best tool. It depends on what problem you are trying to solve.

• Step 1: Identify the areas of the software development life cycle (SDLC) that you are focused on
• Step 2: Identify the problem areas
• Step 3: identify the tools that can help catch the problems earlier and solve them

 Q. What kind of coding and tool experience is needed for AIOps? A. Different parts of the lifecycle require different levels of experience with coding or tools. Many don’t need any coding experience. However, a number of them require a thorough understanding of processes and best practices in software development or IT management to use them effectively. Q. How can a DevOps engineer upskill to AIOps? A. It is very easy for a DevOps engineer to upskill to use AIOps tools. A lot of these capabilities are available as open source. It is best to start experimenting with open-source tools and see their value. Second, focus on a smaller section of the problem (looking at the lifecycle) and then identify the tools that solve that problem. Free tiers, open-source tools, and even manual scripts help upskill without buying these tools. A lot of on-line course sites like Udemy are now offering AIOps classes as well. Q. What are examples of existing AI cloud cost optimization tools? There are 2 types of cloud cost optimization tools

• ITOps tools – automate actions to optimize cost
• FinOps tools – analyze and recommend actions to optimize cost.

The analysis tools are good at identifying issues but fall short of actually providing value unless you manually take action. The tools that automate provide value immediately but need greater buy in from the organization to allow a tool to take action. Some optimization tools available: Turbonomic from IBM, others are from Flexera, Apptio, Densify, AWS cost explorer, Azure Cost Management + Billing, some are built into the cloud providers. Q. Can you please explain runbooks further? A.Runbooks are a sequence of actions often coded as scripts that are used to automate the action or remediation in response to a problem or incident. These are pre-defined procedures. Usually, they are built out of a set of manual actions an operator takes and then codifies in the form of a procedure and then code.    

SNIA’s twelve Technical Work Groups collaborate to develop and promote vendor-neutral architectures, standards, and education for management, movement, and security for technologies related to handling and optimizing data. One of the more unique work groups is the  SNIA Input/Output Traces, Tools, and Analysis Technical Work Group (IOTTA TWG). SNIA Compute, Memory, and Storage Initiative recently sat down with IOTTA TWG Chairs Geoff Kuenning of Harvey Mudd College and Tom West of hyperI/O LLC to learn about some exciting new developments in their work activities and how SNIA members and colleagues can get involved. Q: What does the IOTTA TWG do? A: The IOTTA TWG is for those interested in the use of empirical data/metrics to better understand the actual operation and performance characteristics of storage I/O, especially as they pertain to application workloads. We summarize our work in this SNIA video https://www.youtube.com/watch?v=4EVW5IHHhEk One of our most important activities is to sponsor a collaborative worldwide repository for storage-related I/O trace collection and analysis tools, application workloads, I/O traces, and best practices around such topics. Q: What are the goals of the IOTTA Repository collaboration? A: The primary goal of the IOTTA Repository collaboration is to create a worldwide repository for storage related I/O trace files, associated tools, and other related information, all of which are made available free of charge to the storage research and development communities in both academia and industry. Repository data is often cited in research publications, with 627 citations to date listed on the IOTTA Repository website. Q: Why is keeping and sharing information by way of a Repository important? A: The IOTTA Repository provides a common facility through which a broad community (including storage vendors, storage users, and the academic community) can avail themselves of a variety of storage related I/O traces (especially contemporary I/O traces). We like to think of it as a “One-Stop-Shop”. Q: What kind of information are you gathering for the Repository?  Is some information more important than other(s)? A: The Repository contains a wide variety of storage related I/O trace types, including Block I/O, HPC Summaries, Key-Value Traces, NFS Traces, Parallel Traces, Static Snapshots, System Call Traces, and Workload Summaries. Reliability Traces are the latest category of traces added to the IOTTA Repository. Generally, the Reliability Traces category includes records of storage system reliability, for example, long-term records of hard-drive failures. The IOTTA Repository additionally provides an off-site link to traces that cannot be included directly within the repository (e.g., unable to obtain permission to host a particular trace within the repository). Q: Who downloads this information? What groups can make use of this information? A: Academic institutions are among the most frequent downloaders of Repository information, along with storage companies. Practitioners can make use of various IOTTA Repository traces to gain a better understanding of actual I/O storage operation activity within various environments and scenarios.  Traces can also be used as a basis for benchmarking and testing proposed solutions. SNIA IOTTA TWG members receive a monthly report that shows the number and types (i.e., trace names) of the traces downloaded during the month, including the downloader region (e.g., Asia, Europe, North America). The report also includes company/institution names associated with the downloaders. More information on joining the IOTTA TWG is at http://iotta.snia.org/faqs/joinIOTTA. Q: What is some of the latest information in the Repository? A: In February 2024, we posted NVMe drive reliability traces collected by Alibaba. The collection includes both fail-stop and fail-slow data for a large drive population in Alibaba’s servers. Q: What is the importance of these traces? A: The authors of the associated USENIX ATC 2022 paper indicate that the Alibaba Fail-Stop dataset is the first large-scale public dataset on real-world operational data of NVMe SSD.  From their analysis of the dataset, they identified a series of major reliability changes in NVMe SSD. In addition, the authors of the associated USENIX FAST 2023 paper indicate that the Alibaba Fail-Slow dataset is the first large-scale, clear-labeled public dataset on real-world operational traces aiming at fail-slow detection (i.e., where the drive continues to run but with poor performance). Based upon the dataset, the authors have provided a root cause analysis on fail-slow drives. With the growing importance of NVMe SSDs in the data center, it is critical to understand the reliability of hardware in the cloud.  The Repository provides the traces download and also links to the papers and presentation videos that discuss these large-scale SSD reliability studies. Q: What new activity would you like to see in the Repository? A: We’d like to see more trace downloads for analysis.  Most downloads today are related to benchmarking and replay.  Trace activity could feed into a simulated computer system to test activities like failures. We would also like to see more input of data related to tape storage. The Repository does not have much information on cold storage and multilevel storage between hot and cold storage. Finally, we would like feedback on how people are using what they download – for analysis, reliability, benchmarks and other areas they have found the downloads useful. We also want to know what else you would like to be able to download.  You can contact us directly at iottachairs@snia.org. Thanks for your time and the great information about the IOTTA Repository.  Learn more about the IOTTA Repository on their FAQ page. The post Just What is an IOTTA?  Inquiring Minds Learn Now! first appeared on SNIA Compute, Memory and Storage Blog.

Anyone who follows technology knows that it is a fast-paced world with rapid changes and constant innovations. SNIA, together with its members, technical work groups, Forums, and Initiatives, continues to embrace, educate, and develop standards to make technology more available and better understood. At the SNIA Networking Storage Forum, we’ve been at the forefront of diving into technology topics that extend beyond traditional networked storage, providing education on AI, edge, acceleration and offloads, hyperconverged infrastructure, programming frameworks, and more. We still care about and spend a lot of time on networked storage and storage protocols, but we felt it was time that the name of the group better reflected the broad range of timely topics we’re covering. We are excited to announce our new name: SNIA Data, Networking & Storage Forum (DNSF). This group name aligns with SNIA’s data-centric focus and summarizes our belief that data is the center of networking and storage. In the same way that storage solutions have moved beyond silos to deliver an array of data services, so are we embracing a range of extended, but interrelated data technologies. We've also updated our charter to include the breadth of topics we cover that extend beyond traditional networked storage. Our Charter: The SNIA Data, Networking & Storage Forum (DNSF) educates and provides insights and leadership for applying technologies to a broad spectrum of end-to-end solutions. The DNSF mission is to:

• Educate the industry and end users about technologies, standards and implementations in storage, networking, and data
• Promote a broad range of solutions, including storage area networks (SAN), networked attached storage (NAS), software-defined storage (SDS), disaggregated and hyperconverged infrastructure (HCI)
• Improve understanding of the impacts and opportunities of a wide variety of emerging technologies and use cases by leveraging cross industry expertise and collaboration

We hope you’ve had an opportunity to attend some of the educational webinars we’ve produced. We’re proud of the vast webinar library we’ve built and the positive feedback we get from our attendees. They know they can count on us to tackle technologies in a vendor-neutral way. It can be challenging to do that, but it really is a key tenet that sets SNIA apart. Our webinars in the past few years have ranged from storage networking security, storage performance metrics, and SAN basics to accelerating generative AI, NVMe/TCP and data center sustainability. In addition to our robust and highly-rated webinar program, our members also author and publish white papers, contributed articles, and blogs. We are excited about our new name and expanded charter! We have many great initiatives planned for 2024. Want to join us? Our DNSF members are highly committed and active at our weekly meetings, and we welcome new insights and expertise. Join one of our meetings to see what it’s all about!  Learn more about joining us here. Or email us if you have questions or would like an invite to a meeting. We hope you’ll give consideration to joining our team!

Anyone who follows technology knows that it is a fast-paced world with rapid changes and constant innovations. SNIA, together with its members, technical work groups, Forums, and Initiatives, continues to embrace, educate, and develop standards to make technology more available and better understood. At the SNIA Networking Storage Forum, we’ve been at the forefront of diving into technology topics that extend beyond traditional networked storage, providing education on AI, edge, acceleration and offloads, hyperconverged infrastructure, programming frameworks, and more. We still care about and spend a lot of time on networked storage and storage protocols, but we felt it was time that the name of the group better reflected the broad range of timely topics we’re covering. We are excited to announce our new name: SNIA Data, Networking & Storage Forum (DNSF). This group name aligns with SNIA’s data-centric focus and summarizes our belief that data is the center of networking and storage. In the same way that storage solutions have moved beyond silos to deliver an array of data services, so are we embracing a range of extended, but interrelated data technologies. We’ve also updated our charter to include the breadth of topics we cover that extend beyond traditional networked storage. Our Charter: The SNIA Data, Networking & Storage Forum (DNSF) educates, provides insights, and leadership for applying technologies to a broad spectrum of end-to-end solutions. The DNSF mission is to:

• Educate the industry and end users about technologies, standards and implementations in storage, networking, and data
• Promote a broad range of solutions, including storage area networks (SAN), networked attached storage (NAS), software-defined storage (SDS), disaggregated and hyperconverged infrastructure (HCI)
• Improve understanding of the impacts and opportunities of a wide variety of emerging technologies and use cases by leveraging cross industry expertise and collaboration

We hope you’ve had an opportunity to attend some of the educational webinars we’ve produced. We’re proud of the vast webinar library we’ve built and the positive feedback we get from our attendees. They know they can count on us to tackle technologies in a vendor-neutral way. It can be challenging to do that, but it really is a key tenet that sets SNIA apart. Our webinars in the past few years have ranged from storage networking security, storage performance metrics, and SAN basics to accelerating generative AI, NVMe/TCP and data center sustainability. In addition to our robust and highly-rated webinar program, our members also author and publish white papers, contributed articles, and blogs. We are excited about our new name and expanded charter! We have many great initiatives planned for 2024. Want to join us? Our DNSF members are highly committed and active at our weekly meetings, and we welcome new insights and expertise. Join one of our meetings to see what it’s all about!  Learn more about joining us here. Or email us if you have questions or would like an invite to a meeting. We hope you’ll give consideration to joining our team! The post SNIA Networking Storage Forum – New Name, Expanded Charter first appeared on SNIA on Data, Networking & Storage.

Generative AI is front page news everywhere you look. With advancements happening so quickly, it is hard to keep up. The SNIA Networking Storage Forum recently convened a panel of experts from a wide range of backgrounds to talk about Gen AI in general and specifically discuss how dataflow bottlenecks can constrain Gen AI application performance well below optimal levels. If you missed this session, “Accelerating Generative AI: Options for Conquering the Dataflow Bottlenecks,” it’s available on-demand at the SNIA Educational Library. We promised to provide answers to our audience questions, and here they are. Q: If ResNet-50 is a dinosaur from 2015, which model would you recommend using instead for benchmarking? A: Setting aside the unfair aspersions being cast on the venerable ResNet-50, which is still used for inferencing benchmarks 😊, we suggest checking out the MLCommons website. In the benchmarks section you’ll see multiple use cases on Training and Inference. There are multiple benchmarks available that can provide more information about the ability of your infrastructure to effectively handle your intended workload. Q: Even if/when we use optics to connect clusters, there is a roughly 5ns/meter delay for the fiber between clusters. Seems like that physical distance limit almost mandates alternate ways of programming optimization to ‘stitch’ the interplay between data and compute? A: With regards to the use of optics versus copper to connect clusters, signals propagate through fiber and copper at about the same speed, so moving to an all-optical cabling infrastructure for latency reduction reasons is probably not the best use of capital. Also, even if there were a slight difference in the signal propagation speed through a particular optical or copper based medium, 5ns/m is small compared to switch and NIC packet processing latencies (e.g., 200-800 ns per hop) until you get to full metro distances. In addition, the software latencies are 2-6 us on top of the physical latencies for the most optimized systems. For AI fabrics data/messages are pipelined, so the raw latency does not have much effect. Interestingly, the time for data to travel between nodes is only one of the limiting factors when it comes to AI performance limitations and it’s not the biggest limitation either. Along these lines, there’s a phenomenal talk by Stephen Jones (NVIDIA) “How GPU computing works” that explains how latency between GPU and Memory impacts the overall system efficiency much more than anything else. That said, the various collective communication libraries (NCCL, RCCL, etc) and in network compute (e.g., SHARP) can have a big impact on the overall system efficiency by helping to avoid network contention. Q: Does this mean that GPUs are more efficient to use than CPUs and DPUs? A: GPUs, CPUs, AI accelerators, and DPUs all provide different functions and have different tradeoffs. While a CPU is good at executing arbitrary streams of instructions through applications/programs, embarrassingly parallelizable workloads (e.g., matrix multiplications which are common in deep learning) can be much more efficient when performed by GPUs or AI accelerators due to the GPUs’ and accelerators’ ability to execute linear algebra operations in parallel. Similarly, I wouldn’t use a GPU or AI accelerator as a general-purpose data mover, I’d use a CPU or an IPU/DPU for that. Q: With regards to vector engines, are there DPUs, switches (IB or Ethernet) that contain vector engines? A: There are commercially available vector engine accelerators but currently there are no IPUs/DPUs or switches that provide this functionality natively. Q: One of the major bottlenecks in modern AI is GPU to GPU connectivity. Ex. NVIDIA uses a proprietary GPU-GPU interconnect, At DGX-2 the focus was on 16 GPUs within a single box with NVSwitch, but then with A100 NVIDIA pulled this back to 8GPUs. But then expanded on that to a super-pod and a second level of switching to get to 256GPUS. How does NVlink, or other proprietary GPU to GPU interconnects address bottlenecks? And why has industry focused on an 8 GPU deployment vs a 16 GPU deployment resolution, given that LLMs are not training on 10's of thousands of GPUs? A: GPU-GPU interconnects all addresses bottlenecks in the same way that other high-speed fabrics do. GPU-GPU have direct connections featuring large bandwidth, optimized interconnect (point to point or parallel paths), and lightweight protocols. These interconnects have so far been proprietary and not interoperable across GPU vendors. The number of GPUs in a server chassis is dependent on many practical factors, e.g., 8 Gaudis per server leveraging standard RoCE ports provides a good balance to support training and inference. Q: How do you see the future of blending of memory and storage being enabled for generative AI workloads and the direction of "unified" memory between accelerators, GPUs, DPUs and CPUs? A: If by unified memory, you mean centralized memory that can be treated like a resource pool and be consumed by GPUs in place of HBM or by CPUs/DPUs in place of DRAM, then we do not believe we will see unified memory in the foreseeable future. The primary reason is latency. To have a unified memory would require centralization. Even if you were to constrain the distance (i.e., between the end-devices and the centralized memory) to be a single rack, the latency increase caused by the extra circuitry and physical length of the transport media (at 5ns per meter) could be detrimental to performance. However, the big problem with resource sharing is contention. Whether it be congestion in the network or contention at the centralized resource access point (interface), sharing resources requires special handling that will be challenging in the general case. For example, with 10 “compute” nodes attempting to access a pool of memory on a CXL Type 3 device, many of the nodes will end up waiting an unacceptably long period of time for a response. If by unified memory, you mean creating a new “capacity” tier of memory that is more performant than SSD and less performant than DRAM, then CXL Type 3 devices appear to be the way the industry will address that use case, but it may be a while before we see mass adoption. Q: Do you see the hardware design to more specialized into the AI/ML phases (training, inference, etc.)? But today's enterprise deployments you can have the same hardware performing several tasks in parallel. A: Yes, not only have specialized HW offerings (e.g., accelerators) already been introduced (such as in consumer laptops combining CPUs with inference engines), but also specialized configurations that have been optimized for specific use cases (e.g., inferencing) to be introduced as well. The reason is related to the diverse requirements for each use case. For more information, see the OCP Global Summit 23 presentation “Meta’s evolution of network AI” (specifically starting at time stamp 4:30). They describe how different use cases stress the infrastructure in different ways. That said, there is value in accelerators and hardware being able to address any of the work types for AI so that a given cluster can run whichever mix of jobs is required at a given time. Q: Google leaders like Amin Vahdat have been casting doubts on the possibility of significant acceleration far from the CPU. Can you elaborate further on positioning data-centric compute in the face of that challenge? A: This is a multi-billion-dollar question! There isn’t an obvious answer today. You could imagine building a data processing pipeline with data transform accelerators ‘far’ from where the training and inferencing CPU/accelerators are located. You could build a full “accelerator only” training pipeline if you consider a GPU to be an accelerator not a CPU. The better way to think about this problem is to consider that there is no single answer for how to build ML infrastructure. There is also no single definition of CPU vs accelerator that matters in constructing useful AI infrastructure solutions. The distinction comes down to the role of the device within the infrastructure. With emerging ‘chiplet’ and similar approaches we will see the lines and distinctions blur further. What is significant in what Vahdat and others have been discussing: fabric/network/memory construction plus protocols to improve bandwidth, limit congestion, and reduce tail latency when connecting the data to computational elements (CPU, GPU, AI accelerators, hybrids) will see significant evolution and development over the next few years.