On February 28th, the SNIA Networking Storage Forum (NSF) took at look at what’s happening in Scale-Out File Systems. We discussed general principles, design considerations, challenges, benchmarks and more. If you missed the live webcast, it’s now available on-demand. We did not have time to answer all the questions we received at the live event, so here are answers to them all.
Q. Can scale-out file systems do Erasure coding?
A. Indeed, Erasure coding is a common method to improve resilience.
Q. How does one address the problem of a specific disk going down? Where does scale-out architecture provide redundancy?
A. Disk failures typically are covered by RAID software. Some of scale-out software also use multiple replicators to mitigate the impact of disk failures.
Q. Are there use cases where a hybrid of these two styles is needed?
A. Yes, for example, in some environments, the foundation layer might be using the dedicated storage server to form the large storage pool, which is the 1st style, and then export LUNs or virtual disks to the compute nodes (either physical or virtual) to run the applications, which is the 2nd style.
Q. Which scale-out file systems present on windows, Linux platforms?
A. Some of the scale-out file systems provide native client software across multiple platforms. Another approach is to use Samba to build SMB gateways to make the scale-out file system available to Windows computers.
Q. Is Amazon elastic file system (EFS) on AWS scale-out file systems?
A. Please see:
https://docs.aws.amazon.com/efs/latest/ug/performance.html
“Amazon EFS file systems are distributed across an unconstrained number of storage servers, enabling file systems to grow elastically to petabyte scale and allowing massively parallel access from Amazon EC2 instances to your data. The distributed design of Amazon EFS avoids the bottlenecks and constraints inherent to traditional file servers.”
Q. Where are the most cost effective price/performance uses of NVMe?
A. NVMe can support very high IOPS and very high throughput as well. The best use case would be to couple NVMe with high performance storage software that would not limit the NVMe.
On February 13, 2019, the SNIA Cloud Storage Technologies Initiative (CSTI) presented a live webcast, Why Composable Infrastructure? Our goal was to clearly explain the reasoning behind, and the benefits of, composable infrastructure in an educational, vendor-neutral way. We believe our speakers, Philip Kufeldt and Mike Jochimsen, did just that. Now, as promised, Philip and Mike have answered the many interesting questions we received during the live event.
Q. Are composable infrastructure solutions incompatible with virtualized or containerized environments? Will these solutions only serve bare metal environments?
A. Composable infrastructure solutions will eventually work across any environment that supports the orchestration toolsets. There are no compatibility issues between virtualization/containerization and composable infrastructure, even if they fundamentally look at allocation of resources within a defined resource differently. For example, in a virtualized environment if the need for network bandwidth or storage capacity exceeds the capability of a given resource, a "larger" resource could be composed using the orchestration tools. It would then be managed within the virtualization layer like any other resource.
Q. Typically new technology adoption is slowed due to support within commercial operating systems. Are there changes needed in the major OS environments (Linux, Windows, VMware, etc.), that will need to be released before composable infrastructure solutions will be supported?
A. So the PCIe and Ethernet based fabrics are already well established and have great OS support. The storage world and networking worlds already deploy composable infrastructure. However, the newer standards such as Gen-Z, OpenCAPI, and CCIX will need both hardware and software support. ARM SoCs are showing up with CCIX HW and OpenCAPI is in the Power architecture. But this is just the early stage; switches, enclosures and components that support these standards are still in the offing. Furthermore the OS support for these standards is also unavailable. And finally the management mechanisms and composability software is also undefined. So we still are a good distance from the larger composable infrastructure being available.
Q. Are the data center orchestration tools currently on the market capable of building a composable infrastructure solution like you described?
A. The tools on the market are still in the early stages of providing this capability. Some are purpose built for specific environments while others encompass a wider set of environments, but lack some of the dynamic/automation capabilities. Also, there is work going on, or starting up, in standards bodies to define the APIs needed for orchestration to work in truly heterogeneous application, operating system and hardware environments.
Q. In composable environments, how does security scale with it, specifically, encryption? Encrypt everything? Or some subset of jobs that truly are only jobs requiring encryption?
A. Fabrics can be configured to be local and private relieving the need for encrypted transfers. However there will be new issues to contend with. For example, consider memory that was previously used in one configuration that was disassembled and then reused in another. Ensuring that memory is cleaned before reuse will become required to prevent information leakage.
Q. For Gen-Z, Pooled Memory or Memory from different Racks, what about the Latency issues? Local memories don’t have issues with latency?
A. Although Gen-Z supports longer distance interconnects, it does not mean that only long distance configurations will be utilized. Think of it as a set of tools in a toolbox. Some memories will be close for lower latencies and others will be farther to provide for 4th or 5th level caching.
Q. Is it more about declarative mapping of the components? At this point software and hardware are decoupled, so the messaging and logic are really the requirement for orchestration.
A. The orchestration layer provides a translation between a declarative and imperative state in composable infrastructure. It is responsible for gathering the requirements from the application (declarative - "this is what I want"), then identifying the capabilities of the components on the network and logically mapping them to create a virtual infrastructure (imperative - "this is how to do it").
Q. As apps start to be built from microservices, which may run across different physical nodes, I would think this further raises performance challenges on disaggregated infrastructure. How this will impact things would be an interesting next topic.
A. Agreed. Although I believe microservices will actually be enhanced by composable infrastructure. Composable infrastructure in general will create smaller systems that more closely fit the needs of the service or classes of services that will run on them. Just as in a bin packing problem having smaller units tends to provide better utilization of the container.
Got more questions? Feel free to comment on this blog and we’ll answer.
On February 13, 2019, the SNIA Cloud Storage Technologies Initiative (CSTI) presented a live webcast, 
Mark Carlson (MC): SNIA does not just sit back and dream up work to do, but rather relies on industry input and requirements. A great example of how SNIA work is created is the 2018 launch of the Computational Storage Technical Work Group (TWG). It all started at the Flash Memory Summit industry event with a simple Birds-of-a-Feather on a thing called “computational storage”. Interest definitely was high – the room was packed - and the individuals assembled decided to use SNIA as the vehicle to do definition and standardization of this new concept.
SOS: What made them choose SNIA?
MC: The industry views SNIA as a place where they can come to get agreement on technical issues. SNIA is committed to developing and promoting standards in the broad and complex fields of digital storage and information management to allow solutions to be more easily produced. Technical Work Groups (TWGs) are collaborative technical groups within the SNIA that work on specific areas of technology development.
In the case of computational storage, it was clear that the industry saw SNIA as a catalyst to get things off and running with computational storage in the standards world because…six months after FMS we have a new Technical Work Group with 90 individuals from 30 member companies starting their work and evangelizing computational storage at events like March’s OCP Summit.
SOS: What are some of the changing storage industry trends that might create a need for new SNIA efforts?
MC: We see some exciting activity in the areas of composable infrastructure, next generation fabrics, and new data center innovations. Standard computer architectures in data centers today are essentially servers acting as the building blocks for assembling data center sized solutions. But imagine if CPUs, memory, storage, and networks were individual components on next generation fabrics, making more granular building blocks. The definition then of what a server is would be an assemblage on the fly of these components. Scalability of any individual platform instance would become a matter of combining sufficient components for the task.
SOS: What role would SNIA play?
MC: Composable infrastructure will now need to be managed in a different way than when using servers as a building block. Rather than having a single point of management for multiple components in a box, management now needs to be component-based and handled by the components themselves. Redfish from the DMTF is already architected for this very issue of scale out and SNIA’s Swordfish is continuing to extend this into storage architectures.
SNIA is already a pioneer in the area of storage management, and anticipating this coming possible disruption by moving its standards to modern protocols with alliance partners like DMTF and NVM Express. DMTF Redfish and SNIA Swordfish are examples of this work where Object Drive and other work items create a Redfish profile for NVMe drives. NVMe drives will be a storage component and a peer of the CPU either attached to the host or not attached and outside the box.
Next generation fabrics are enabling these composable infrastructures, making new data center innovation possible. But the new fabrics themselves need to be managed as well.
SOS: Where can folks learn about these activities in person?
MC: Check out the Computational Storage Birds-of-a-Feather at 
Hyperconverged infrastructures (also known as “HCI”) are designed to be easy to set up and manage. All you need to do is add networking. In practice, the “add networking” part has been more difficult than most anticipated. That’s why the SNIA Networking Storage Forum (NSF) hosted a live webcast “
/TCP Means to Networked Storage
Leave a Reply