This talk describes a set of desired distributed location and data motion patterns that optimize storage/data placement for hybrid/multicloud and/or edge-to-core-to-cloud outcomes. What capabilities do you need to expect from your storage platforms to execute on a strategy that mitigates or completely eliminates cloud egress costs, yet also maximizes performance for workloads wherever they need to reside? How do you include sovereignty and governance into your foundational architecture?

According to IDC forecast, global data will almost double in 3 years and 90% of the data generated is unstructured data which comes in many forms such as documents, videos, images, audio, IOT data, etc. The difficulty of managing the different forms of unstructured data exacerbated by myriads of interfaces and technologies in hybrid multi-cloud environments lead to data sprawl which is one of the major pain points for any organization.

Hyperscale deployments are using Serial Attached SCSI (SAS) today, to solve capacity and cost challenges within their large-scale storage infrastructure. Specific features within SAS address these problems and offer developers real solutions. This presentation will discuss how SAS addresses the scaling, reliability, cost and flexibility requirements of modern hyperscale architectures and show how this maps to Meta's Grand Canyon architecture.

In our current data driven world, the ability to store massive amounts of data is inevitable for not just large enterprises. As the legislation evolves, there are more and more requirements against distributed storage systems, but security has been there since the beginning, just as in Apache Ozone. We have different layers where security considerations have to play a central role. Data storage, data access, data transfer, or encryption just to name a few. All layers have their own specialities, and an already established frame for solutions defined mainly by already mature systems.

At SNIA SDC 2022, we introduced Cloud Storage Acceleration Layer (CSAL), a host based FTL deployed at scale at Alibaba, which doubled VM density and cut time to insight in half in their cloud services. To deliver these technology benefits to more customers, we released CSAL to open source in SPDK v22.09 and at SNIA SDC 2023 we are announcing a reference design published on the SPDK website. With Intel winding down Optane and the increased availability of dense QLC SSDs, the need for CSAL has increased. Accordingly, Solidigm has acquired the CSAL technology and development team.

/*-->*/

Software Defined Storage (SDS) frameworks offer storage services that provide high availability, durability and reliability guarantees. However, these guarantees come at a performance cost. While drives can offer microsecond latency and throughput of millions of IOPs, SDS services typically offer millisecond access latencies with 10-100K IOPs throughput.

/*-->*/

Cloud customers need a range of storage types to use as the durable layer of their systems. For example, some customers want a Blob interface, some want to use files, and others want disks. Cloud providers must offer all of these and more options, while continuing to innovate and reduce costs. Creating separate systems for each product offering is inefficient. Trying to build all product offerings on one system is ineffective.

/*-->*/