Physical Storage

A growing trend in the market is capacity of data. This data growth is creating challenges within modern storage infrastructures and a new way to think of data is needed. The SNIA Computational Storage TWG was formed in October of 2018 to address this opportunity for the Storage industry to use innovative technologies that bring computational capabilities closer to or within the storage device.

The applications using NVMe, SAS, SATA, USB based storage devices find a new use and one of them is mining for open source cryptocurrency such as Burst Coin. Using low power or solar power HDD’s, SSD and most importantly NVMe technology can improve turnaround latency and build blocks on a faster scale. Utilization of security protocols allows anonymization as well as protection of the users and vendors. Burst coin has an extensive developer’s community and can run on the cloud, has dApps, its own ATM and more.

One of the most common benchmarks in the storage industry is 4KiB random read I/O per second. Over the years, the industry first saw the publication of 1M I/Ops on a single box, then 1M I/Ops on a single thread (by SPDK). More recently, there have been publications outlining 10M I/Ops on a single box using high performance NVMe devices and more than 100 CPU cores. This talk will present a benchmark of SPDK performing more than 10 million random 4KiB read operations per second from a single thread to 20 NVMe devices, a large advance compared to the state of the art of the industry.

NVMe KV (Key-Value) is an industry-wide proposal for a new command structure that allows access to data on an NVMe SSD controller using a "key" rather than a block address. Developed within the NVM Express technical working group, this Key Value command set provides a "key" to store a corresponding "value" on non-volatile media, then retrieves that "value" from the media by specifying the corresponding "key." In addition to extensive work being undertaken by the NVM Express working group, the SNIA has completed an overarching KeyValue API released for a membership vote in January 2019.

This talks explains the exciting new features in the Linux NVMe driver and software target in the last two years, as well as the relevant block layer changes to support these features.

As a unified data processing engine, Spark is expected to achieve high throughput and ultralow latency for different workloads like ad hoc queries, real-time streaming, and machine learning. However, under certain workloads (large join/aggregation), its performance is limited by the overhead from the persistence on local shuffle drives and transferring with TCP/IP networking.

Often one needs to trade some performance for security, but not today. We'll be talking about kernel bypass based storage acceleration along with securing that storage via hardware within the server's own network interface card. We'll discuss using NVMe/TCP with kernel bypass, and how it can yield storage performance measured in 10s of microseconds as opposed to locally attached spinning disks at 10s of milliseconds. Also, we'll review how this storage can be secured by adding a mere 250 to 750 nanoseconds of latency required to adequately filter network accesses to this storage.

Since its ratification in 2016, NVMe over Fabrics has become the accepted protocol for disaggregating NVMe media. Even as more features get added, such as support for TCP as a transport, universal adoption of NVMe over Fabrics has been slow due to the challenges of deploying across any OS and hypervisor platform as well as supporting the Storage Services required in a production environment. SmartNICs are a new platform with the necessary features and capabilities to enable this.

Zoned Namespaces is an exciting new interface in the NVMe specification. It expose a new storage interface between the host and SSD, in which the SSD exposes a set of sequential write only zones, that the SSD internally can align perfectly to its physical media characteristics. The new interface allows one to minimize SSD write amplification, improve throughput and latency, and extend the life of the SSD. However, to take advantage of it, one much have host support (file-systems and/or applications) that understands the interface, and can make efficient use of it.

Customers of SSDs have very different requirements. While a single uniform space of non-volatile storage media may satisfy some, others require independent areas with performance isolation. Still others may require a small amount of low-latency storage and a much larger amount of higher-latency storage. To enable a single SSD type to be configured by the customer to satisfy their particular requirements, NVM express has defined a mechanism to configure non-volatile media in not just SSDs but also in storage arrays.


Subscribe to RSS - Physical Storage