Leave it to Rob Peglar, SNIA Board Member and the MC of SNIA’s 7^th annual Persistent Memory Summit to capture the Summit day as persistently fun with a metric boatload of great presentations and speakers! And indeed it was a great day, with fourteen sessions presented by 23 speakers covering the breadth of where PM is in 2019 – real world, application-focused, and supported by multiple operating systems. Find a great recap on the Forbes blog by Tom Coughlin of Coughlin Associates. Attendees enjoyed live demos of Persistent Memory technologies from AgigA Tech, Intel, SMART Modular, the SNIA Solid State Storage Initiative, and Xilinx.  Learn more about what they presented here. And for the first time as a part of the Persistent Memory Summit, SNIA hosted a Persistent Memory Programming Hackathon sponsored by Google Cloud, where SNIA PM experts mentored software developers to do live coding to understand the various tiers and modes of PM and what existing methods are available to access them.  Upcoming SNIA SSSI on Solid State Storage blogs will give details and insights into “PM Hacking”.  Also sign up for the SNIAMatters monthly newsletter to learn more, and stay tuned for upcoming Hackathons – next one is March 10-11 in San Diego. Missed out on the live sessions?  Not to worry, each session was videotaped and can be found on the SNIA Youtube Channel.  Download the slides for each session on the PM Summit agenda at www.snia.org/pm-summit.  Thanks to our presenters from Advanced Computation and Storage, Arm, Avalanche Technology, Calypso Systems, Coughlin Associates, Dell, Everspin Technologies, In-Cog Solutions, Intel, Mellanox Technologies, MemVerge, Microsoft, Objective Analysis, Sony Semiconductor Solutions Corporation, Tencent Cloud, Western Digital, and Xilinx.   And thanks also to our great audience and their questions – your enthusiasm and support will keep us persistently having even more fun!

The SNIA Networking Storage Forum (NSF) kicked off the New Year with a live webcast "Virtualization and Storage Networking Best Practices." We guessed it would be popular and boy, were we right! Nearly 1,000 people registered to attend. If you missed out, it's available on-demand. You can also download a copy of the webcast slides. Our experts, Jason Massae from VMware and Cody Hosterman from Pure Storage, did a great job sharing insights and lessons learned on best practices on configuration, troubleshooting and optimization. Here's the blog we promised at the live event with answers to all the questions we received. Q. What is a fan-in ratio? A. fan-in ratio (sometimes also called an "oversubscription ratio") refers to the number of hosts links related to the links to a storage device. Using a very simple example can help understand the principle: Say you have a Fibre Channel network (the actual speed or protocol does not matter for our purposes here). You have 60 hosts, each with a 4GFC link, going through a series of switches, connected to a storage device, just like in the diagram below: This is a 10:1 oversubscription ratio; the "fan-in" part refers to the number of host bandwidth in comparison to the target bandwidth. Block storage protocols like Fibre Channel, FCoE, and iSCSI have much lower fan-in ratios than file storage protocols such as NFS, and deterministic storage protocols (like FC) have lower than non-deterministic (like iSCSI). The true arbiter of what the appropriate fan-in ratio is determined by the application. Highly transactional applications, such as databases, often require very low ratios. Q. If there's a mismatch in the MTU between server and switch will the highest MTU between the two get negotiated or else will a mismatch persist? A. No, the lowest value will be used, but there's a caveat to this. The switch and the network in the path(s) can have MTU set higher than the hosts, but the hosts cannot have a higher MTU than the network. For example, if your hosts are set to 1500 and all the network switches in the path are set to 9k, then the hosts will communicate over 1500. However, what can, and usually does, happen is someone sets the host(s) or target(s) to 9k but never changes the rest of the network. When this happens, you end up with unreliable or even loss of connectivity. Take a look at the graphic below: A large ball can't fit through a hole smaller than itself. Consequently, a 9k frame cannot pass through a 1500 port. Unless you and your network admin both understand and use jumbo frames, there's no reason to implement in your environment. Q. Can you implement port binding when using two NICs for all traffic including iSCSI? A. Yes you can use two NICs for all traffic including iSCSI, many organizations use this configuration. The key to this is making sure you have enough bandwidth to support  all  the traffic/ IO that will use those NICs. You should, at the very least, use 10Gb NICs faster if possible. Remember, now all your management, VM and storage traffic are using the same network devices. If you don't plan accordingly, everything can be impacted in your virtual environment. There are some hypervisors capable of granular network controls to manage which type of traffic uses which NIC, certain failover details and allow setting QoS limits on the different traffic types. Subsequently, you can ensure storage traffic gets the required bandwidth or priority in a dual NIC configuration. Q. I've seen HBA drivers that by default set their queue depth to 128 but the target port only handles 512. So two HBAs would saturate one target port which is undesirable. Why don't the HBA drivers ask what the depth should be at installation? A. There are a couple of possible reasons for this. One is that many do not know what it even means, and are likely to make a poor decision (higher is better, right?!). So vendors tend to set these things at defaults and let people change them if needed—and usually that means they have purpose to change them. Furthermore, every storage array handles these things differently, and that can make it more difficult to size these things. It is usually better to provide consistency—having things set uniformly makes it easier to support and will give more consistent expectations even across storage platforms. Second, many environments are large—which means people usually are not clicking and type through installation. Things are templatized, or sysprepped, or automated, etc. During or after the deployment their automation tools can configure things uniformly in accordance with their needs. In short, it is like most things: give defaults to keep one-off installations simple (and decrease the risks from people who may not know exactly what they are doing), complete the installations without having to research a ton of settings that may not ultimately matter, and yet still provide experienced/advanced users, or automaters, ways to make changes. Q. A number of white papers show the storage uplinks on different subnets. Is there a reason to have each link on its own subnet/VLAN or can they share a common segment?   A. One reason is to reduce the number of logical paths. Especially in iSCSI, the number of paths can easily exceed supported limits if every port can talk to every target. Using multiple subnets or VLANs can drop this in half—and all you really use is logical redundancy, which doesn't really matter. Also, if everything is in the same subnet or VLAN and someone make some kind of catastrophic change to that subnet or VLAN (or some device in it causes other issues), it is less likely to affect both subnets/VLANs. This gives some management "oops" protection. One change will bring all storage connectivity down.      

The SNIA Networking Storage Forum (NSF) kicked off the New Year with a live webcast “Virtualization and Storage Networking Best Practices.” We guessed it would be popular and boy, were we right! Nearly 1,000 people registered to attend. If you missed out, it’s available on-demand. You can also download a copy of the webcast slides. Our experts, Jason Massae from VMware and Cody Hosterman from Pure Storage, did a great job sharing insights and lessons learned on best practices on configuration, troubleshooting and optimization. Here’s the blog we promised at the live event with answers to all the questions we received. Q. What is a fan-in ratio? A. fan-in ratio (sometimes also called an “oversubscription ratio”) refers to the number of hosts links related to the links to a storage device. Using a very simple example can help understand the principle: Say you have a Fibre Channel network (the actual speed or protocol does not matter for our purposes here). You have 60 hosts, each with a 4GFC link, going through a series of switches, connected to a storage device, just like in the diagram below: This is a 10:1 oversubscription ratio; the “fan-in” part refers to the number of host bandwidth in comparison to the target bandwidth. Block storage protocols like Fibre Channel, FCoE, and iSCSI have much lower fan-in ratios than file storage protocols such as NFS, and deterministic storage protocols (like FC) have lower than non-deterministic (like iSCSI). The true arbiter of what the appropriate fan-in ratio is determined by the application. Highly transactional applications, such as databases, often require very low ratios. Q. If there’s a mismatch in the MTU between server and switch will the highest MTU between the two get negotiated or else will a mismatch persist? A. No, the lowest value will be used, but there’s a caveat to this. The switch and the network in the path(s) can have MTU set higher than the hosts, but the hosts cannot have a higher MTU than the network. For example, if your hosts are set to 1500 and all the network switches in the path are set to 9k, then the hosts will communicate over 1500. However, what can, and usually does, happen is someone sets the host(s) or target(s) to 9k but never changes the rest of the network. When this happens, you end up with unreliable or even loss of connectivity. Take a look at the graphic below: A large ball can’t fit through a hole smaller than itself. Consequently, a 9k frame cannot pass through a 1500 port. Unless you and your network admin both understand and use jumbo frames, there’s no reason to implement in your environment. Q. Can you implement port binding when using two NICs for all traffic including iSCSI? A. Yes you can use two NICs for all traffic including iSCSI, many organizations use this configuration. The key to this is making sure you have enough bandwidth to support all the traffic/ IO that will use those NICs. You should, at the very least, use 10Gb NICs faster if possible. Remember, now all your management, VM and storage traffic are using the same network devices. If you don’t plan accordingly, everything can be impacted in your virtual environment. There are some hypervisors capable of granular network controls to manage which type of traffic uses which NIC, certain failover details and allow setting QoS limits on the different traffic types. Subsequently, you can ensure storage traffic gets the required bandwidth or priority in a dual NIC configuration. Q. I’ve seen HBA drivers that by default set their queue depth to 128 but the target port only handles 512. So two HBAs would saturate one target port which is undesirable. Why don’t the HBA drivers ask what the depth should be at installation? A. There are a couple of possible reasons for this. One is that many do not know what it even means, and are likely to make a poor decision (higher is better, right?!). So vendors tend to set these things at defaults and let people change them if needed—and usually that means they have purpose to change them. Furthermore, every storage array handles these things differently, and that can make it more difficult to size these things. It is usually better to provide consistency—having things set uniformly makes it easier to support and will give more consistent expectations even across storage platforms. Second, many environments are large—which means people usually are not clicking and type through installation. Things are templatized, or sysprepped, or automated, etc. During or after the deployment their automation tools can configure things uniformly in accordance with their needs. In short, it is like most things: give defaults to keep one-off installations simple (and decrease the risks from people who may not know exactly what they are doing), complete the installations without having to research a ton of settings that may not ultimately matter, and yet still provide experienced/advanced users, or automaters, ways to make changes. Q. A number of white papers show the storage uplinks on different subnets. Is there a reason to have each link on its own subnet/VLAN or can they share a common segment?  A. One reason is to reduce the number of logical paths. Especially in iSCSI, the number of paths can easily exceed supported limits if every port can talk to every target. Using multiple subnets or VLANs can drop this in half—and all you really use is logical redundancy, which doesn’t really matter. Also, if everything is in the same subnet or VLAN and someone make some kind of catastrophic change to that subnet or VLAN (or some device in it causes other issues), it is less likely to affect both subnets/VLANs. This gives some management “oops” protection. One change will bring all storage connectivity down.      

To meet the increasingly higher demand on both capacity and performance in large cluster computing environments, the storage subsystem has evolved toward a modular and scalable design. The scale-out file system has emerged as one implementation of the trend, in addition to scale-out object and block storage solutions. What are the key principles when architecting a scale-out file system? Find out on February 28^th when the SNIA Networking Storage Forum (NSF) hosts The Scale-Out File System Architecture Overview, a live webcast where we will present an overview of scale-out file system architectures. This presentation will provide an introduction to scale-out-file systems and cover:

• General principles when architecting a scale-out file system storage solution
• Hardware and software design considerations for different workloads
• Storage challenges when serving a large number of compute nodes, e.g. name space consistency, distributed locking, data replication, etc.
• Use cases for scale-out file systems
• Common benchmark and performance analysis approaches

Register today to save your spot. We hope you will join us.

To meet the increasingly higher demand on both capacity and performance in large cluster computing environments, the storage subsystem has evolved toward a modular and scalable design. The scale-out file system has emerged as one implementation of the trend, in addition to scale-out object and block storage solutions. What are the key principles when architecting a scale-out file system? Find out on February 28^th when the SNIA Networking Storage Forum (NSF) hosts The Scale-Out File System Architecture Overview, a live webcast where we will present an overview of scale-out file system architectures. This presentation will provide an introduction to scale-out-file systems and cover:

• General principles when architecting a scale-out file system storage solution
• Hardware and software design considerations for different workloads
• Storage challenges when serving a large number of compute nodes, e.g. name space consistency, distributed locking, data replication, etc.
• Use cases for scale-out file systems
• Common benchmark and performance analysis approaches

Register today to save your spot. We hope you will join us.

The landscape of containers is moving fast and constantly changing, with new standards emerging every few months. If you wondering what’s new in container storage, you are not alone. That’s why the SNIA Cloud Storage Technologies Initiative is hosting a live webcast on February 26, 2019, “What’s New in Container Storage.” In this webcast, Keith Hudgins of Docker joins us as a follow up to his earlier container webcast “Intro to Containers, Container Storage and Docker.” It’s our most popular webcast to date with thousands of views. If you missed it, it’s available on demand and will provide you with some great background information before our February 26^th webcast. I encourage you to register today for the February 26^th session where you’ll learn:

• What’s new, what to pay attention to, and how to make sense of the ever-shifting container landscape.
• Container storage types and Container Frameworks
• An overview of the various storage APIs for the container landscape
• How to identify the most important projects to follow in the container world
• The Container Storage Interface spec and Kubernetes 1.13
• How to get involved in the container community

It will be live, so bring your questions!

When it comes to persistent memory, many application developers initially think of change as hard work that likely yields incremental result.  It’s perhaps a better idea to look at the capability that’s new, but that’s already easily accessible using the methods that are in place today.  It’s not that enabling persistent memory is effortless, it’s more that normal code improvement can take advantage of the new features in the standard course of development. The concept of multiple memory tiers is ingrained in nearly every programming model.  While the matrix of possibility can get fairly complex, it’s worth looking at three variables of the memory model.  The first is the access type, either via load/store or block operation. The second is the latency or distance from the processing units; in this case the focus would be on the DIMM.  The last would be memory persistence. Adding persistence to the DIMM tier of memory provides opportunity to programmers in a variety of ways.  Typically, this latency is used for most of the program flow, while data eventually is moved to a farther tier such as disk or network for persistence.  Allocating the majority of data to a low-latency tier like a DIMM has significant potential. An example of this in the marketplace would be SAP’s HANA in-memory database.  However, it’s less well-known that more traditional database products in the same category have built-in methodologies for moving data that is repeatedly accessed into the DIMM tier, later committing changes to storage via background processes.  It’s likely that adding persistence to DIMMs in volume would be both valuable and also architecturally possible in a short period of development time. One way that this process is simplified for developers is the fact that the SNIA NVM Programming Model for DIMM-based persistence incorporates both load/store and block access modes.   Developers already familiar with using SSD over rotating media -- that would be a fourth memory vector, deal with the ambiguity -- would be able to see some incremental performance and potentially some system design simplification.  Those already using memory for data storage could utilize better recovery options as well as explore changes that high-performance storage could bring. Join other developers on Wednesday, January 23rd at the SNIA Persistent Memory Programming Hackathon to explore options for how your software can take advantage of this new opportunity. Complimentary registration is available at this link.

When it comes to persistent memory, many application developers initially think of change as hard work that likely yields incremental result.  It’s perhaps a better idea to look at the capability that’s new, but that’s already easily accessible using the methods that are in place today.  It’s not that enabling persistent memory is effortless, it’s more that normal code improvement can take advantage of the new features in the standard course of development. The concept of multiple memory tiers is ingrained in nearly every programming model.  While the matrix of possibility can get fairly complex, it’s worth looking at three variables of the memory model.  The first is the access type, either via load/store or block operation. The second is the latency or distance from the processing units; in this case the focus would be on the DIMM.  The last would be memory persistence. Adding persistence to the DIMM tier of memory provides opportunity to programmers in a variety of ways.  Typically, this latency is used for most of the program flow, while data eventually is moved to a farther tier such as disk or network for persistence.  Allocating the majority of data to a low-latency tier like a DIMM has significant potential. An example of this in the marketplace would be SAP’s HANA in-memory database.  However, it’s less well-known that more traditional database products in the same category have built-in methodologies for moving data that is repeatedly accessed into the DIMM tier, later committing changes to storage via background processes.  It’s likely that adding persistence to DIMMs in volume would be both valuable and also architecturally possible in a short period of development time. One way that this process is simplified for developers is the fact that the SNIA NVM Programming Model for DIMM-based persistence incorporates both load/store and block access modes.   Developers already familiar with using SSD over rotating media — that would be a fourth memory vector, deal with the ambiguity — would be able to see some incremental performance and potentially some system design simplification.  Those already using memory for data storage could utilize better recovery options as well as explore changes that high-performance storage could bring. Join other developers on Wednesday, January 23rd at the SNIA Persistent Memory Programming Hackathon to explore options for how your software can take advantage of this new opportunity. Complimentary registration is available at this link.

With another successful year of SNIA Storage Developer Conferences (SDC) completed, SNIA on Storage spoke with Mark Carlson, SNIA Technical Council Co-Chair, on 2018 highlights and 2019 plans to educate and support this important technical community. SNIA On Storage (SOS):  In 2018, SNIA volunteers provided key resources and time supporting our storage developer community, a central focus for the Technical Council.  What activities were the TC most pleased with in 2018? Mark Carlson (MC):  For the past several years we have been pushing SDC to expand globally. Starting with SDC India and this year adding SDC EMEA, we are reaching developers around the world now with educational content. Each year we get better and better talk proposals and the hard part is not being able to present more content. I always encourage developers to submit talks on their latest discoveries and architectures. Other developers want to hear about it. SOS:  There was a lot of good feedback on the content from SDC North America in 2018.  What was your overall impression of this 21^st SDC event? MC:  We had more folks than ever attending SDC 2018 and there was great energy in the sessions – particularly those hallway track ones.  We had strong topics and speakers with 16 keynotes as well as the breakout sessions and BoFs. I enjoyed hearing about new architectures such as Computational Storage and composable infrastructure. And for those who missed attending in person, all 100 session presentations are available for download from the SNIA web site. SOS:  What are some of the highlights for those who could not attend? MC:  SNIA’s new activities on computational storage were well received.  The keynote from Andy Walker on the Evolution of Solid State Memory was of high interest. Also Storage Lessons from High Performance Computing by Gary Grider of Los Alamos Labs generated lots of good questions. SNIA recorded all the keynotes and general sessions, which are available on the SNIA Video channel.  The Technical Council has also chosen the audio recordings of the best breakout sessions to be available as podcasts, with a new one released every week.  They can be found on the SNIA podcast site and are available on Apple iTunes.  The 2018 podcasts begin with #76.  You can also find podcasts from previous years on that page. SOS:  Now that SDC has expanded beyond North America, what’s on tap for 2019? MC:  SNIA returns to Tel Aviv, Israel with the second annual SDC EMEA on January 30, 2019. Delegate registration is now open and is rapidly filling up. SDC EMEA 2019 is also running an SMB Plugfest all week and has also scheduled an afternoon Technical Seminar on SMB.  You can register and learn more at https://www.snia.org/events/sdcemea SDC will also return to Bangalore, India on May 23-24, 2019.  The call for presentations is open at https://www.snia.org/events/sdcindia.  The SNIA India developer community is very strong and has an important role in establishing this agenda. We welcome all companies with developer activities in India to participate in SDC and also in the SNIA India T/E/N meetups which happen throughout the year. SOS:  Great talking with you, Mark, and we look forward to an excellent year of storage developer education in 2019.

It’s very rare that there is a significant change in computer architecture, especially one that is nearly immediately pervasive across the breadth of a market segment.  It’s even more rare when a fundamental change such as this is supported in a way that software developers can quickly adapt to existing software architecture. Most significant transitions require a ground-up rethink to achieve performance or reliability gains, and the cost-benefit analysis generally pushes a transition to the new thing be measured in multiple revisions as opposed to one, big jump. In the last decade the growth of persistent memory has bucked this trend.  The introduction of the solid-state disk made an immediate impact on existing software, especially in the server market.  Any program that relied on multiple, small-data, read/write cycles to disk recognized significant performance increases. In cases such as multi-tiered databases, the software found a, “new tier,” of storage nearly automatically and started partitioning data to it.  In an industry where innovation takes years, improvement took a matter of months to proliferate across new deployments. While the SSD is now a standard consideration there is unexplored opportunity in solid-state storage.  The NVDIMM form factor has been in existence for quite some time, providing data persistence significantly closer to processing units in the modern server and workstation.  Many developers, however, are not aware that programming models already exist to easily incorporate some simple performance and reliability, both for byte and block access in programs.  Moreover, new innovations of persistent memory are on the horizon that will increase the density and performance of DIMM form factors. Perhaps it’s time that more software architecture should be working on adapting this exciting technology.  The barriers to innovation are very low, and opportunity is significant. Over the year 2019, SNIA will be sponsoring the delivery of several workshops dedicated to opening up persistent memory programming to the developer community.  The first of these will be a Persistent Memory Programming Hackathon at the Hyatt Regency Santa Clara CA on January 23, 2019, the day before the SNIA Persistent Memory Summit.   Developers will have the opportunity to work with experienced software architects to understand how to quickly adapt code to use new persistent memory modes in a hackathon format.  Learn more and register at this link. Don’t miss the opportunity to move on a strategic software inflection point ahead of the competition.  Consider attending the 2019 SNIA Persistent Memory Summit and exploring the opportunity with persistent memory.