Many times we know things without even realizing it, or remembering how we came to know them. In technology, this often comes from direct, personal experience rather than some systematic process. In turn, this leads to "best practices" that come from tribal knowledge, rather than any inherent codified set of rules to follow. In the world of storage, for example, it's very tempting to simply think of component parts that can be swapped out interchangeably. Change out your spinning hard drives for solid state, for example, you can generally expect better performance. Change the way you connect to the storage device, get better performance... or do you? Storage is more holistic than many people realize, and as a result there are often unintended consequences for even the simplest of modifications. With the ‘hockey stick-like' growth in innovation over the past couple of years, many people have found themselves facing terms and concepts in storage that they feel they should have understood, but don't. These series of webcasts are designed to help you with those troublesome spots: everything you thought you should know about storage but were afraid to ask. Here, we're going to go all the way back to basics and define the terms so that people can understand what people are talking about in those discussions. Not only are we going to define the terms, but we're going to talk about terms that are impacted by those concepts once you start mixing and matching. For example, when we say that we have a "memory mapped" storage architecture, what does that mean? Can we have a memory mapped storage system at the other end of a network? If so, what protocol should we use – iSCSI? POSIX? NVMe over Fabrics? Would this be an idempotent system or an object-based storage system? Now, if that above paragraph doesn't send you into fits of laughter, then this series of webcasts is for you (hint: most of it was complete nonsense... but which part? Come watch to find out!). On September 7^th, we will start with the very basics – The Naming of the Parts. We'll break down the entire storage picture and identify the places where most of the confusion falls. Join us in this first webcast – Part Chartreuse – where we'll learn:

• What an initiator is
• What a target is
• What a storage controller is
• What a RAID is, and what a RAID controller is
• What a Volume Manager is
• What a Storage Stack is

  With these fundamental parts, we'll be able to place them into a context so that you can understand how all these pieces fit together to form a Data Center storage environment. Future webcasts will discuss: Part Mauve - Architecture Pod:

• Channel v. bus
• Control plane v. data plane
• Fabric v. network

Part Teal - Buffering Pod:

• Buffering v. Queueing (with Queue Depth)
• Flow Control
• Ring Buffers

Part Rosé - iSCSI Pod:

• iSCSI offload
• TCP offload
• Host-based iSCSI

Part Sepia - Getting-From-Here-To-There Pod:

• Encapsulation v. Tuning
• IOPS v. Latency v. Jitter

Part Vermillion - The What-if-Programming-and-Networking-Had-A-Baby Pod:

• Storage APIs v. POSIX
• Block v. File v. Object
• Idempotent
• Coherence v. Cache Coherence
• Byte Addressable v. Logical Block Addressing

Part Turquoise - Where-Does-My-Data-Go Pod:

• Volatile v. Non-Volatile v Persistent Memory
• NVDIMM v. RAM v. DRAM v. SLC v. MLC v. TLC v. NAND v. 3D NAND v. Flash v SSDs v. NVMe
• NVMe (the protocol)

Part Cyan - Storage Management:

• Management Processes: Discovery, Provisioning and Configuration
• Software-Defined Storage
• Storage Management Standards: SMI-S, SNIA Swordfish

Part Aqua - Storage Controllers:

• Storage Controllers 101
• SCSI Controllers
• Fibre Channel Controllers
• NVMe Controllers
• Networking SDN and Storage SDN Controllers

Part Taupe - Memory Pod:

• Memory Mapping
• Physical Region Page (PRP)
• Scatter Gather Lists
• Offset

Part Burgundy - Orphans Pod

• Doorbells
• Controller Memory Buffers

Of course, you may already be familiar with some, or all, of these concepts. If you are, then these webcasts aren't for you. However, if you're a seasoned professional in technology in another area (compute, networking, programming, etc.) and you want to brush up on some of the basics without judgment or expectations, this is the place for you. Oh, and why are the parts named after colors, instead of numbered? Because there is no order to these webcasts. Each is a standalone seminar on understanding some of the elements of storage systems that can help you learn about technology without admitting that you were faking it the whole time! If you are looking for a starting point – the absolute beginning place – please start with Part Chartreuse, "The Naming of the Parts." Update: This series is now available on-demand. You can also download the webcast slides. Happy viewing! Watch: Chartreuse - The Basics  Download: Webcast slides Watch: Mauve - The Architecture Pod Download: Webcast slides Watch: Teal - The Buffering Pod  Download: Webcast slides Watch: Rosé - This iSCSI Pod  Download: Webcast slides Watch: Sepia - Getting from Here to There Download: Webcast slides Watch: Vermillion - What if Programming and Networking Had a Storage Baby  Download: Webcast slides Watch: Turquoise - Where Does My Data Go?  Download: Webcast slides Watch: Cyan - Storage Management Download: Webcast slides Watch: Aqua - Storage Controllers Download: Webcast slides   Watch: Taupe - Memory    Download: Webcast slides  

Many times we know things without even realizing it, or remembering how we came to know them. In technology, this often comes from direct, personal experience rather than some systematic process. In turn, this leads to “best practices” that come from tribal knowledge, rather than any inherent codified set of rules to follow.

In the world of storage, for example, it’s very tempting to simply think of component parts that can be swapped out interchangeably. Change out your spinning hard drives for solid state, for example, you can generally expect better performance. Change the way you connect to the storage device, get better performance… or do you?

Storage is more holistic than many people realize, and as a result there are often unintended consequences for even the simplest of modifications. With the ‘hockey stick-like’ growth in innovation over the past couple of years, many people have found themselves facing terms and concepts in storage that they feel they should have understood, but don’t.

These series of webcasts are designed to help you with those troublesome spots: everything you thought you should know about storage but were afraid to ask.

Here, we’re going to go all the way back to basics and define the terms so that people can understand what people are talking about in those discussions. Not only are we going to define the terms, but we’re going to talk about terms that are impacted by those concepts once you start mixing and matching.

For example, when we say that we have a “memory mapped” storage architecture, what does that mean? Can we have a memory mapped storage system at the other end of a network? If so, what protocol should we use – iSCSI? POSIX? NVMe over Fabrics? Would this be an idempotent system or an object-based storage system?

Now, if that above paragraph doesn’t send you into fits of laughter, then this series of webcasts is for you (hint: most of it was complete nonsense… but which part? Come watch to find out!).

On September 7^th, we will start with the very basics – The Naming of the Parts. We’ll break down the entire storage picture and identify the places where most of the confusion falls. Join us in this first webcast – Part Chartreuse – where we’ll learn:

• What an initiator is
• What a target is
• What a storage controller is
• What a RAID is, and what a RAID controller is
• What a Volume Manager is
• What a Storage Stack is

 

With these fundamental parts, we’ll be able to place them into a context so that you can understand how all these pieces fit together to form a Data Center storage environment. Future webcasts will discuss:

Part Mauve – Architecture Pod:

• Channel v. bus
• Control plane v. data plane
• Fabric v. network

Part Teal – Buffering Pod:

• Buffering v. Queueing (with Queue Depth)
• Flow Control
• Ring Buffers

Part Rosé – iSCSI Pod:

• iSCSI offload
• TCP offload
• Host-based iSCSI

Part Sepia – Getting-From-Here-To-There Pod:

• Encapsulation v. Tuning
• IOPS v. Latency v. Jitter

Part Vermillion – The What-if-Programming-and-Networking-Had-A-Baby Pod:

• Storage APIs v. POSIX
• Block v. File v. Object
• Idempotent
• Coherence v. Cache Coherence
• Byte Addressable v. Logical Block Addressing

Part Taupe – Memory Pod:

• Memory Mapping
• Physical Region Page (PRP)
• Scatter Gather Lists
• Offset

Part Turquoise – Where-Does-My-Data-Go Pod:

• Volatile v. Non-Volatile v Persistent Memory
• NVDIMM v. RAM v. DRAM v. SLC v. MLC v. TLC v. NAND v. 3D NAND v. Flash v SSDs v. NVMe
• NVMe (the protocol)

Part Burgundy – Orphans Pod

• Doorbells
• Controller Memory Buffers

Of course, you may already be familiar with some, or all, of these concepts. If you are, then these webcasts aren’t for you. However, if you’re a seasoned professional in technology in another area (compute, networking, programming, etc.) and you want to brush up on some of the basics without judgment or expectations, this is the place for you.

Oh, and why are the parts named after colors, instead of numbered? Because there is no order to these webcasts. Each is a standalone seminar on understanding some of the elements of storage systems that can help you learn about technology without admitting that you were faking it the whole time! If you are looking for a starting point – the absolute beginning place – please start with Part Chartreuse, “The Naming of the Parts.” We look forward to seeing you on September 7^th at 10:00 a.m. PT. Register today.

Ethernet technology had been a proven standard for over 30 years and there are many networked storage solutions based on Ethernet. While storage devices are evolving rapidly with new standards and specifications, Ethernet is moving towards higher speeds as well: 10Gbps, 25Gbps, 50Gbps and 100Gbps….making it time to re-introduce Ethernet Networked Storage.

That’s exactly what Rob Davis and I plan to do on August 4^th in a live SNIA Ethernet Storage Forum Webcast, “Re-Introducing Ethernet Networked Storage.” We will start by providing a solid foundation on Ethernet networked storage and move to the latest advancements, challenges, use cases and benefits. You’ll hear:

• The evolution of storage devices – spinning media to NVM
• New standards: NVMe and NVMe over Fabric
• A retrospect of traditional networked storage including SAN and NAS
• How new storage devices and new standards would impact Ethernet networked storage
• Ethernet based software-defined storage and the hyper-converged model
• A look ahead at new Ethernet technologies optimized for networked storage in the future

I hope you will join us on August 4^th at 10:00 a.m. PT. We’re confident you will learn some new things about Ethernet networked storage. Register today!

The SNIA Data Protection and Capacity Optimization Committee (DPCO) would like to announce the creation of a new, Special Interest Group focusing on Data Replication for Disaster Recovery (DR) Standards. The mission of this SIG is focused on investigating existing ISO standards, carrying out surveys, and studying current guidance in order to identify if there is a need to improve the interoperability and resiliency, and/or education and best practices in the area of data replication for disaster recovery.

Why are we doing this? There have been a number of industry observations that customers either don’t know about standards that exist, cannot implement them or have other needs relating to DR that warrant exploration. The aim of this group is not to reinvent the wheel but examine what is out there, what can be used by customers and find out whether they are using appropriate standards, and if not why.

What are we doing? We are starting with a survey to be sent out to as many industry members as possible. The survey will examine replication DR needs that customers have, systems that they have implemented and questions about their knowledge regarding standards and other issues encountered in designing and operating DR, particularly in multi-site, multi-vendor environments.

What can you do? Get involved, of course! Contact the SNIA DPCO team to indicate your interest as we implement the organization structure for the Data Replication for DR Standards SIG.

John Olson and Gene Nagle

The SNIA Data Protection and Capacity Optimization Committee (DPCO) would like to announce the creation of a new, Special Interest Group focusing on Data Replication for Disaster Recovery (DR) Standards. The mission of this SIG is focused on investigating existing ISO standards, carrying out surveys, and studying current guidance in order to identify if there is a need to improve the interoperability and resiliency, and/or education and best practices in the area of data replication for disaster recovery.

Why are we doing this? There have been a number of industry observations that customers either don’t know about standards that exist, cannot implement them or have other needs relating to DR that warrant exploration. The aim of this group is not to reinvent the wheel but examine what is out there, what can be used by customers and find out whether they are using appropriate standards, and if not why.

What are we doing? We are starting with a survey to be sent out to as many industry members as possible. The survey will examine replication DR needs that customers have, systems that they have implemented and questions about their knowledge regarding standards and other issues encountered in designing and operating DR, particularly in multi-site, multi-vendor environments.

What can you do? Get involved, of course! Contact the SNIA DPCO team to indicate your interest as we implement the organization structure for the Data Replication for DR Standards SIG.

John Olson and Gene Nagle

Cloud storage has transformed the storage industry, however interoperability challenges that were overlooked during the initial stages of growth are now emerging as front and center issues. I hope you will join us on July 19^th for our live Webcast, “Cloud Storage: Solving Interoperability Challenges,” to learn the major challenges facing the use of businesses services from multiple cloud providers and moving data from one cloud provider to another. We’ll discuss how the SNIA Cloud Data Management Interface standard (CDMI) addresses these challenges by offering data and metadata portability between clouds and explain how the SNIA CDMI Conformance Test Program helps cloud storage providers achieve CDMI conformance. Join us on July 19^th to learn:

• Critical challenges that the cloud storage industry is facing
• Issues in a multi-cloud API environment
• Addressing cloud storage interoperability challenges
• How the CDMI standard works
• Benefits of CDMI conformance testing
• Benefits for end user companies

You can register today. We look forward to seeing you on July 19^th.

Cloud storage has transformed the storage industry, however interoperability challenges that were overlooked during the initial stages of growth are now emerging as front and center issues. I hope you will join us on July 19^th for our live Webcast, “Cloud Storage: Solving Interoperability Challenges,” to learn the major challenges facing the use of businesses services from multiple cloud providers and moving data from one cloud provider to another.

We’ll discuss how the SNIA Cloud Data Management Interface standard (CDMI) addresses these challenges by offering data and metadata portability between clouds and explain how the SNIA CDMI Conformance Test Program helps cloud storage providers achieve CDMI conformance.

Join us on July 19^th to learn:

• Critical challenges that the cloud storage industry is facing
• Issues in a multi-cloud API environment
• Addressing cloud storage interoperability challenges
• How the CDMI standard works
• Benefits of CDMI conformance testing
• Benefits for end user companies

You can register today. We look forward to seeing you on July 19^th.

At this month’s SNIA Ethernet Storage Forum Webcast, “Architectural Principles for Networked Solid State Storage Access,” Doug Voigt, Chair of the SNIA NVM Programming Technical Working Group, and a member of the SNIA Technical Council, outlined key architectural principles surrounding the application of networked solid state technologies. We had a flurry of questions near the end of the Webcast that we did not have enough time to answer. Here are Doug’s answers to all the questions we received during the event:

Q. Are there wait cycles in accessing persistent memory?

A. It depends entirely on which persistent memory (PM) technology is being accessed and how the memory interconnect is used.  Some technologies have write times that are quite different from read times.  When using tightly timed interconnects such as DDR with those technologies it may be difficult to avoid wait cycles.

Q. How do Pmalloc and malloc share the virtual address space of the application?

A. This is entirely up to the OS and other libraries operating within any constraints of the processor architecture-specific memory management units.  A good mental model would be fairly large regions of contiguous address space in both the physical and virtual domains, where each region will comprise a single type of memory. Capacity will be reserved for pmalloc and malloc in the appropriate regions.

Q. Always flush after doing your memory-mapped IO.  Is that simply good hygiene?

A. Not exactly. The term “Memory Mapped IO” is used to reference control plane (as opposed to data plane) access.  It is often reasonable to set up control plane memory as uncacheable. The need for strict order of access to physical control plane registers is so pervasive that caching is generally not useful. Uncacheable writes are always flushed by the processor, as opposed to the application.

Generally with memory mapped IO devices the data plane uses direct memory access (DMA).  With memory mapped files (as opposed to memory mapped IO) Load/Store (more commonly referred to as “Ld/St”), not DMA, is used in the data plane. Disabling caching in the data plane is generally a big performance sacrifice for small byte range access.

In the Ld/St datapath, strategically placed flushing is required to retain both performance and power failure recovery. The SNIA NVM Programming Model describes this type of functionality.

Q. Once NVDIMM support become pervasive with support from NVMe drives in the server box, should network storage be more focused on SAS Flash or just SAS HDDs?

A. Not necessarily.  NVMe over Fabric, Fibre Channel and iSCSI are also types of networked storage that will likely retain significant market share relative to SAS.

Q. Are the ‘Big Data’ Data Warehouse applications starting to use the persistence memory and domain technologies in their applications?

A. It is too early to see much of this yet. PM technologies might become a priority as a staging area for analytic applications with high ingest or checkpoint rates. NVDIMMs are likely to be too expensive to store anything “big” for quite a while.

Q. Also, is the persistence memory/domains being used in the Hyper-converged and Converged hardware infrastructures?

A. Persistent memory is quintessentially (Hyper-) converged.  It wouldn’t be unreasonable to expect some traction with hyper-converged solutions that experience high storage-performance demand.

Q. What distance would you associate with 10’s of microseconds?

A. In terms of transmission delay, 10’s of uS align with a campus or small city scale, but the distance itself is often not the primary factor.  Switching delays, transmission line properties and software overhead are generally bigger factors.

Q. So latency would be the binding factor for distances…not a question, an observation.

A. Yes, in effect, either through transmission or relay.  See above.

Q. Aren’t there multi-threaded SSDs?

A. Yes, but since the primary metric in this presentation is latency we ignore multi-threading.  It can enable more work to get done, but it generally increases latency rather than reducing it.

Q. Is Pmalloc universal usage?

A. The term is starting to be recognized among developers and has been used in research. Various similar names have been used in early research prototypessuch as pmalloc in Mnemosyne and nvmalloc in SCMFS.

Q. So how would PM help in a (stock broking) requirement, where we currently prophesize an RDMA or iWARP solution?

A. With PM the answer is always lower latency.  PM can be litegrated like memory or like flash. RDMA network paths for both of these options were discussed in the presentation. In either case, PM is low-latency enough that networking and software overheads will completely determine performance, even when using RDMA. The performance boost from PM is greatest when it is accessed locally.  If remote access is a requirement then the new work being done in the RDMA community should help.

Q. If data stored in memory requires to be copied to a different host, memory (for consistency) how does PM assist, or is there an extension to PM? Coherency between multiple hosts in a cluster, if you will?

A. PM technology does not help with this; the methods of managing consistency across hosts remain unchanged by PM.  All PM offers is low latency persistence.

Coordination across hosts or nodes in a cluster must use existing clustering techniques such as locking and quorums. In addition, the relative timescales of memory access and network communication suggest the application of asynchronous remote replication techniques used in today’s storage solutions.

Regarding coherency, PM brings nothing new to the known techniques for managing coherency.  Classical cluster architecture must be applied outside of symmetric multi-processing coherency domains. Within coherency domains, all of the logic is above the PM level in a processor side memory controller or a software emulation of the same algorithms.

 

 

 

 

There has been a lot of buzz around cloud object storage recently. But before you get deep into all that cloud object storage can do, it’s good to take a step back and make sure you understand the basics. That’s what the SNIA Cloud Storage Initiative is planning to do on July 14^th at our live Webcast “Cloud Object Storage 101.” Many organizations, like large service providers, have already begun to leverage software-defined object storage to support new application development and DevOps projects. Meanwhile, legacy enterprise companies are in the early stages of exploring the benefits of object storage for their particular business and are searching for how they can use cloud object storage to modernize their IT strategies, store and protect data, while dramatically reducing the costs associated with legacy storage sprawl. This Webcast will highlight the market trends towards the adoption of object storage, the definition and benefits of object storage, and the use cases that are best suited to leverage an underlying object storage infrastructure. Join us on July 14^th to learn:

• How to accelerate the transition from legacy storage to a cloud object architecture
• Understand the benefits of object storage
• Primary use cases
• How an object storage can enable your private, public or hybrid cloud strategy without compromising security, privacy or data governance

I hope you’ll register today to join my colleague, Nancy Bennis, Director of Alliances at Cleversafe (an IBM company), and me for this tutorial on cloud object storage.    

There has been a lot of buzz around cloud object storage recently. But before you get deep into all that cloud object storage can do, it’s good to take a step back and make sure you understand the basics. That’s what the SNIA Cloud Storage Initiative is planning to do on July 14^th at our live Webcast “Cloud Object Storage 101.”

Many organizations, like large service providers, have already begun to leverage software-defined object storage to support new application development and DevOps projects. Meanwhile, legacy enterprise companies are in the early stages of exploring the benefits of object storage for their particular business and are searching for how they can use cloud object storage to modernize their IT strategies, store and protect data, while dramatically reducing the costs associated with legacy storage sprawl.

This Webcast will highlight the market trends towards the adoption of object storage, the definition and benefits of object storage, and the use cases that are best suited to leverage an underlying object storage infrastructure.

Join us on July 14^th to learn:

• How to accelerate the transition from legacy storage to a cloud object architecture
• Understand the benefits of object storage
• Primary use cases
• How an object storage can enable your private, public or hybrid cloud strategy without compromising security, privacy or data governance

I hope you’ll register today to join my colleague, Nancy Bennis, Director of Alliances at Cleversafe (an IBM company), and me for this tutorial on cloud object storage.