Break Out Sessions and Agenda Tracks Include:

Note: This agenda is a work in progress. Check back for updates on additional sessions as well as the agenda schedule.

Using CDMI to Manage Swift, S3, and Ceph Object Repositories

David Slik, Technical Director, NetApp, Inc

Abstract

The Cloud Data Management Interface is designed to provide namespace-based management functionality for the superset of object, file and block protocols. This makes it ideally suited for use with common protocols such as NFS, CIFS, iSCSI, Swift and S3. This session provides an overview of how CDMI interoperates with these protocols, and how the use of CDMI as a management protocol adds value to multi-protocol systems. Concrete examples and use cases from end-users and vendors will be highlighted.

Learning Objectives

• Learn how to use CDMI to manage object repositories
• Learn how to use CDMI to manage file systems
• Learn how to use CDMI to manage block storage systems
• Learn how CDMI works with multi-protocol systems

Unistore: A Unified Storage Architecture for Cloud Computing

Yong Chen, Assistant Professor, Texas Tech University

Abstract

Emerging large-scale applications on Cloud computing platform, such as information retrieval, data mining, online business, and social network, are data- rather than computation-intensive. Storage system is one of the most critical components for Cloud computing. The traditional hard disk drives (HDD) are current dominant storage devices in Clouds, but are notorious for long access latency and failure prone. The recently emerged storage class memory (SCM) such as Solid State Drives provides a new promising storage solution of high bandwidth, low latency, and mechanical component free, but with inherent limitations of small capacity, short lifetime, and high cost. This talk will introduce an ongoing effort from Texas Tech University and Nimboxx Inc. of building an innovative unified storage architecture (Unistore) with the co-existence and efficient integration of heterogeneous HDD and SCM devices for Cloud storage systems. We will introduce the Unistore design principle and rationale. We will also discuss the prototyping implementation with newly designed data distribution and placement algorithm. This talk is intended for SNIA/SDC general attendees.

The Developer's Dilemma: Do-It-Yourself Storage or Surrender Your Data?

Luke Behnke, VP of Product, Bitcasa

Abstract

Creating an app isn’t simple. Early in the process of designing the app, decisions have to be made around how app data will be stored, and for most developers the cloud is an obvious choice. At this point, developers need to make an important choice: invest time, energy and resources in creating their own DIY file systems that sits on top of public cloud infrastructure; or take the shortcut and use a cloud storage API, and surrender their users’ data to popular cloud storage services. In this session, Bitcasa CEO, Brian Taptich will outline the impact of this dilemma on the future functionality and user experience of an app, and also discuss why the next generation of apps will require better file systems that offer broad capabilities, performance, security and scalability, and most importantly, developer control of user data and experience.

Learning Objectives

• To explain the benefits of utilizing the cloud for developers
• To dissect the various options developers have when utilizing cloud storage
• Why choosing the right platform is imperative for user experience and privacy
• Why owning user data is important if developers want to own the customer

What You Need to Know on Cloud Storage

David Slik, Technical Director, NetApp
Mark Carlson, Principal Engineer, Industry Standards, Toshiba

Abstract

This session assumes no prior knowledge on cloud storage and is intended to bring a storage developer up to speed on the concepts, conventions and standards in this space. The session will include a live demo of a storage cloud operating to reinforce the concepts presented.

Using REST API for Management Integration

Brian Mason, MTS-SW, NetApp

Abstract

Integration is key to managing storage systems today. Customers do not want a vendor lock-in or vendor specific management tools. They want to use their best in class management tools and have various storage systems integrate into their management tools. A REST API to your storage system is an absolute must in today's market. REST is the common denominator for management integration. Fortunately it is rather simple to create a REST API. It is a little harder to get one just right and to get the documentation done in a usable form.

Learning Objectives

• What is a REST API? How are they different from previous API protocols? Why are they so useful?
• Technology Primer for REST
• How to build a REST API
• Documentation Standards
• Using a REST API as a client

SNIA Tutorial:
Windows Interoperability Workshop

Christopher Hertel, SW Senior Program Engineer, Samba Team / Dell Compellent

Abstract

Windows and POSIX are different, and bridging the gap between the two—particularly with Network File Systems—can be a confusing and daunting endeavor ...and annoying, too.

This tutorial will provide an overview of the SMB3 network file protocol (the heart and soul of Windows Interoperability) and describe some of the unique and powerful features that SMB3 provides. We will also point out and discuss some of the other protocols and services that are integrated with SMB3 (such as PeerDist), and show how the different pieces are stapled together and made to fly. The tutorial will also cover the general structure of Microsoft's protocol documentation, the best available cartography for those lost in the Interoperability Jungle. Some simple code examples will be used sparingly as examples, wherever it may seem clever and useful to do so.

Learning Objectives

• Become familiar with the Windows Interoperability Ecosystem
• Better understand Microsoft's Specifications
• Identify Windows-specific semantic details

Next Generation Data Centers: Hyperconverged Architectures Impact On Storage

Mark OConnell, Distinguished Engineer, EMC

Abstract

A modern data center typically contains a number of specialized storage systems which provide centralized storage for a large collection of data center applications. These specialized systems were designed and implemented as a solution to the problems of scalable storage, 24x7 data access, centralized data protection, centralized disaster protection strategies, and more. While these issues remain in the data center environment, new applications, new workload profiles, and the changing economics of computing have introduced new demands on the storage system which drive towards new architectures, and ultimately towards a hyperconverged architecture. After reviewing what a hyperconverged architecture is and the building blocks in use in such architectures, there will be some predictions for the future of such architectures.

Learning Objectives

• What is a hyperconverged architecture
• How hyperconverged architectures differ from traditional architectures
• What technologies are being used to build hyperconverged architectures
• What workloads are appropriate for hyperconverged architectures

PCI Express: Driving the Future of Storage

Ramin Neshati, PCI-SIG Board Member and Marketing Chair, PCI-SIG

Abstract

The data explosion has led to a corresponding explosion in the demand for storage. At the same time, traditional storage interconnects such as SATA are being replaced with PCI Express (PCIe)-attached storage solutions. Leveraging PCIe technology removes the performance bottlenecks and provides long-term bandwidth and performance scalability as PCIe evolves from 8GT/s bit rate to 16GT/s and beyond. PCIe-attached storage delivers a robust solution that is supported natively in all Operating Systems and a wide array of form factors either chip-to-chip or through expansion modules and daughter cards.

Learning Objectives

• Gain insight into PCI Express technology and how it is used in storage solutions
• Learn how PCI Express technology advancements in lowering active and idle power can be used in your storage solution
• Learn how PCIe 3.0 and PCIe 4.0 provides a strategic solution for storage attachment
• Understand the work the PCI-SIG is doing in new form factors for driving broad adoption in storage applications

Next Generation Low Latency Storage Area Networks

Rupin Mohan, Chief Technologist - Storage Networking, HP
Craig Carlson, Senior Technologist, Q Logic

Abstract

In this session, we will present a current (FC, FCoE and iSCSI) and future state (iSER, RDMA, NVMe, and more) of the union of the next generation low latency Storage Area Networks (SAN's) and discuss how the future of SAN's protocols will look like for block, file and object storage.

The Pros and Cons of Developing Erasure Coding and Replication Instead of Traditional RAID in Next-Generation Storage Platforms

Abhijith Shenoy, Engineer, Hedvig

Abstract

Scale-out, hyperconverged, hyperscale, software-defined, hybrid arrays – the list of scalable and distributed storage systems is rapidly growing. But all of these innovations require tough choices on how best to protect data. Moreover, the abundance of 4- 8- and even 10-TB drives makes the traditional approach of RAID untenable because repairing drive failures can take days and even weeks depending on the architecture and drive capacity. New approaches that balance performance with availability are needed. Erasure coding and replication are emerging, rapidly maturing techniques that empower developers with new data protection methods.

This session will discuss the pros and cons of erasure coding and replication versus traditional RAID techniques. Specifically, this session will discuss the performance vs. availability tradeoffs with each technique as well as present and in-depth look at using tunable replication as the ideal data protection solution, as proven by large-scale distributed systems.

Learning Objectives

• Attendees will learn why RAID isn’t adequate in next-gen storage architectures.
• Attendees will learn the pros and cons of erasure coding and replication in newer storage architectures.
• Attendees will learn how replicating at a per-volume basis provides the best mix of performance and availability.
• Attendees will learn tips and best practices for incorporating new data protection methods into storage platforms.

The Lightning Memory–Mapped Database

Howard Chu, CTO, Symas

Abstract

The Lightning Memory-Mapped Database (LMDB) was introduced at LDAPCon 2011 and has been enjoying tremendous success in the intervening time. LMDB was written for the OpenLDAP Project and has proved to be the world's smallest, fastest, and most reliable transactional embedded data store. It has cemented OpenLDAP's position as world's fastest directory server, and its adoption outside the OpenLDAP Project continues to grow, with a wide range of applications including big data services, crypto-currencies, machine learning, and many others.

The talk will cover highlights of the LMDB design as well as the impact of LMDB on other projects.

Learning Objectives

• Highlight problems with traditional DB storage designs
• Explain benefits of single-level-store
• Explain corruption-proof design and implementation
• Compare and contrast leading data structures: B+tree, LSM, Fractal Trees

The Bw-Tree Key-Value Store and Its Applications to Server/Cloud Data Management in Production

Sudipta Sengupta, Principal Research Scientist, Microsoft Research

Abstract

The Bw-Tree is an ordered key-value store, built by layering a B-tree form access method over a cache/storage sub-system (LLAMA) that is lock-free and organizes storage in a log-structured manner. It is designed to optimize performance on modern hardware, specifically (i) multi-core processors with multi-level memory/cache hierarchy, and (ii) flash memory based SSDs with fast random reads (but inefficient random write performance). The Bw-Tree is shipping in three of Microsoft’s server/cloud products – as the key sequential index in SQL Server Hekaton (main memory database), as the indexing engine inside Azure DocumentDB (distributed document-oriented store), and as an ordered key-value store in Bing ObjectStore (distributed back-end supporting many properties in Bing).

Learning Objectives

• Bw-Tree data structure
• Lock-free design for high concurrency
• Log-structured storage design for flash based SSDs
• Page-oriented store (LLAMA) for building access methods on top
• Bw-Tree Applications in Production at Microsoft

IMDB NDP Advances

Gil Russell, Principal Analyst, Semiscape

Abstract

In-Memory Database appliances are rapidly evolving, becoming in effect the main operating stored image for both analytic and cognitive computing applications in the next generation of data center and cloud in-rack storage.

Co-opting of DRAM with proximal NAND-Flash mass storage being combined with Near Data Processing re-imagines the entire computing paradigm by effectively turning an entire database image into a content-addressable look alike. Candidates for Storage Class Memory are nearing market introduction and with Near Data Processing abilities will radically change Database Management Systems.

Learning Objectives

• An understanding of performance metrics for IMDB systems
• The importance of silicon photonic interconnects
• The evolution to a low latency high bandwidth DB environment
• Competing elements for market supremacy.
• Cognitive computing - the next step

Taxonomy of Differential Compression

Liwei Ren, Scientific Adviser, Trend Micro

Abstract

Differential compression (aka, delta encoding) is a special category for data de-duplication. It can find many applications in various domains such as data backup, software revision control systems, software incremental update, file synchronization over network, to name just a few. This talk will introduce a taxonomy of how to categorize delta encoding schemes in various applications. Pros and cons of each scheme will be investigated in depth.

Learning Objectives

• Why do we need differential compression?
• A mathematical model for describing the differences between two files
• A taxonomy for categorizing differential compression
• Analysis for practical applications

Design Decisions and Repercussions of Compression and Data Reduction in a Storage Array

Chris Golden, Software Engineer, Pure Storage

Abstract

All flash arrays incorporate a number of data reduction techniques to increase effective capacity and reduce overall storage costs. Compression and deduplication are two commonly employed techniques, each with multiple different strategies for implementation. Because compression and data reduction are only part of a greater data reduction strategy, one must also understand their codependent interactions with the rest of a storage system. This talk presents a structured overview of multiple different compression and deduplication technologies. The basics of each technique are presented alongside their benefits, drawbacks and impact on overall system design. This talk then augments that understanding by applying these various techniques to a sample real-world workload, demonstrating the impact of these decisions in practice.

Learning Objectives

• Gain a deeper understanding of compression, deduplication and storage layout
• Benefits and drawbacks of using data reduction techniques in a flash storage array
• Examination of co-dependent interactions between various data reduction techniques and workload
• Supplementing theory with practice via analysis of a sample workload

DAOS – An Architecture for Extreme Scale Storage

Eric Barton, Lead Architect of the High Performance Data Division, Intel

Abstract

Three emerging trends must be considered when assessing how storage should operate at extreme scale. First, continuing expansion in the volume of data to be stored is accompanied by increasing complexity in the metadata to be stored with it and queries to be executed on it. Second, ever increasing core and node counts require corresponding scaling of application concurrency while simultaneously increasing the frequency of hardware failure. Third, new NVRAM technologies allow storage, accessible at extremely fine grain and low latency, to be distributed across the entire cluster fabric to exploit full cross-sectional bandwidth. This talk describes Distributed Application Object Storage (DAOS) – a new storage architecture that Intel is developing to address the functionality, scalability and resilience issues and exploit the performance opportunities presented by these emerging trends.

Learning Objectives

• Exascale / Big Data
• Scalable Distributed Storage Systems
• Object Storage
• Persistent Memory

New Consistent Hashing Algorithms for Data Storage

Jason Resch, Software Architect, Cleversafe, Inc.

Abstract

Consistent Hashing provides a mechanism through which independent actors in a distributed system can reach an agreement about where a resource is, who is responsible for its access or storage, and even derive deterministically a prioritized list of fall-backs should the primary location be down. Moreover, consistent hashing allows aspects of the system to change dynamically while minimizing disruptions. We've recently developed a new consistent hashing algorithm, which we call the Weighted Rendezvous Hash. Its primary advantage is that it obtains provably minimum disruption during changes to a data storage system. This presentation will introduce this algorithm for the first time, and consider several of its applications.

Learning Objectives

• What is Consistent Hashing
• Traditional applications of consistent hashing
• The implementation of the new algorithm: Weighted Rendezvous Hash
• Why Weighted Rendezvous Hash is more efficient than previous algorithms
• Applications of Weighted Rendezvous Hash in data storage systems

Beyond Consistent Hashing and TCP: Vastly Scalable Load Balanced Storage Clustering

Alex Aizman, CTO and Founder, Nexenta Systems
Caitlin Bestler, Senior Director of Arch, Nexenta Systems

Abstract

Successive generations of storage solutions have increased decentralization. Early NAS systems made all the decisions on a single server, down to sector assignment. Federated NAS enabled dynamic distribution of the namespace across multiple storage serves. The first Object Clusters delegated CRUD-based management of both object metadata and data to OSDs.

Current generation of Object Clusters uses Consistent Hashing to eliminate the need for central metadata. However, Consistent Hashing and its derivatives, combined with the prevalent use of TCP/IP in storage clusters results in performance hot spots and bottlenecks, diminished scale-out capability and dis-balances in resource utilization.

These shortcomings will be demonstrated with a simulation of a large storage cluster. An alternative next generation strategy that simultaneously optimizes available IOPS "budget" of the back-end storage, storage capacity, and network utilization will be explained. Practically unlimited load-balanced scale-out capability using Layer 5 (Replicast) protocol for Multicast Replication within the cluster will be presented.

Learning Objectives

• Why neither Consistent Hashing nor TCP/IP scale
• How CCOW (Cloud Copy-on-Write) and Replicast provide for infinite scale-out
• Impact on IOPS, Storage Capacity and Network utilization
• Simulation: queuing and congestion in a 4000 node cluster Actual
• Results: measured with a more affordable cluster

Real World Use Cases for Tachyon, a Memory-Centric Distributed Storage System

Haoyuan Li, CEO, Tachyon Nexus

Abstract

Memory is the key to fast big data processing. This has been realized by many, and frameworks such as Spark and Shark already leverage memory performance. As data sets continue to grow, storage is increasingly becoming a critical bottleneck in many workloads.

To address this need, we have developed Tachyon, a memory-centric fault-tolerant distributed storage system, which enables reliable file sharing at memory-speed across cluster frameworks such as Apache Spark, MapReduce, and Apache Flink. The result of over three years of research and development, Tachyon achieves both memory-speed and fault tolerance.

Tachyon is Hadoop compatible. Existing Spark, MapReduce, Flink programs can run on top of it without any code changes. Tachyon is the default off-heap option in Spark. The project is open source and is already deployed at many companies in production. In addition, Tachyon has more than 100 contributors from over 30 institutions, including Yahoo, Tachyon Nexus, Redhat, Baidu, Intel, and IBM. The project is the storage layer of the Berkeley Data Analytics Stack (BDAS) and also part of the Fedora distribution.

In this talk, we give an overview of Tachyon, as well as several use cases we have seen in the real world.

Innovations in Data Intensive Life Sciences Research

Jay Etchings, Director of Operations, Research Computing, Arizona State University

Abstract

Amazing work is being done today in research medicine and life sciences. With the advent of next generation genome sequencing researchers now have a critical gateway to understanding the underlying molecular networks for disease (pathways). These networks are dependent on another type of network not all that much different from the molecular network. Many known factors exist to inhibit the thoughtful analyses and subsequent discoveries of public value. Genomic processing requires immense computational power and storage. Many university researchers, lacking adequate onsite resources, are faced with the challenge of designing and deploying their own infrastructure with little technical expertise or support. ASU is emerging as a leader in this development.

Learning Objectives

• How Hadoop and Spark address traditional challenges face with parallel file systems
• How Genomic data compliments precision medicine
• Biomedical Informatics is the nest IT boom
• Metadata management and Object storage for Genomic analysis primer
• How precision/personalized medicine is growing faster than storage

How Did Human Cells Build a Storage Engine?

Sanjay Joshi, CTO Life Sciences, EMC Emerging Technologies Division

Abstract

The eukaryotic cell is a fascinating piece of biological machinery – storage is at its heart, literally, within the nucleus. This presentation will tell a story of the evolution of the storage portion of the human cell and its present capacity and properties that could be "bio-mimicked" for future digital storage systems, especially deep archives.

Learning Objectives

• Biological storage concepts
• Requirements for a storage 'unit'
• Requirements for replication
• Requirements for error correction
• Power management

Apache Ignite - In-Memory Data Fabric

Nikita Ivanov, CTO, GridGain Systems

Abstract

This presentation will provide a deep dive into new Apache project: Apache Ignite. Apache Ignite is the in-memory data fabric that combines industry first distributed and fault-tolerant in-memory file system, in-memory cluster and computing, in-memory data grid and in-memory streaming under one umbrella of a fabric. In-memory data fabric slides between applications and various data sources and provides ultimate data storage to the applications.

Apache Ignite is the first general purpose in-memory computing platform in Apache Software Foundation family. We believe it will have same effect on Fast Data processing as Hadoop has on Big Data processing. Better understanding of inner details behind Apache Ignite will hopefully encourage more companies and individual committers to join the project.

Learning Objectives

• Learn about industry leading in-memory data fabric

Integrity of In-memory Data Mirroring in Distributed Systems

Tejas Wanjari, Senior Software Engineer, EMC Data Domain

Abstract

Data in memory could be in a modified state than its on-disk copy. Also, unlike the on-disk copy, the in-memory data might not be checksummed, replicated or backed-up, every time it is modified. So the data must be checksummed before mirroring to avoid network corruptions. But checksumming the data in the application has other overheads: It must handle networking functionalities like retransmission, congestion, etc. Secondly, if it delays the validation of mirrored data, it might be difficult to recover the correct state of the system.

Mirrored-data integrity as transport protocol functionality leads to modular design and better performance. We propose a novel approach that utilizes TCP with MD5 signatures to handle the network integrity overhead. Thus, the application can focus on its primary task. We discuss the evaluation and use-case of this approach (NVM mirroring in Data Domain HA) to prove its advantages over conventional approach of checksumming in the application.

Learning Objectives

• Designing efficient data-mirroring in backup and recovery systems, where reliability is prime
• Linux kernel TCP know-how for using it with MD5 option
• Analysis of conventional approach vs. the TCP MD5
• Use-case: TCP MD5 option for NVM mirroring in Data Domain HA

Learnings from Creating Plugin Module for OpenStack Manila Services

Vinod Eswaraprasad, Software Architect, Wipro

Abstract

Manila is the file sharing service for OpenStack . Manila provides the management of file shares (for example, NFS and CIFS) as a core service to OpenStack. Manila services, like all other openstack services follows a pluggable architecture, and it provides a management of a shared file system instances. This paper discusses our work on integrating a multi-protocol NAS storage device to the OpenStack Manila service. We look at the architecture principle behind the scalability and modularity of Manila services, and the analysis of interface extensions required to integrate a typical NAS head. We also take a deeper look at a NAS file share management interfaces required for a software defined storage controller within the OpenStack Manila framework.

Learning Objectives

• OpenStack File sharing service architecture
• The API and integration framework for openstack services
• NAS share management - and integration
• SDS - interfaces required for a NAS device

Apache HDFS

Jakob Homan, Distributed Systems Engineer, Microsoft

Abstract

During the past two years, HDFS has been rapidly developed to meet the needs of enterprise and cloud customers. We'll take a look at the new features, their implementations and how they address previous shortcomings of HDFS.

How to Enable a Reliable and Economic Cloud Storage Solution by Integrating SSD with LTFS – Addressing Challenges and Best Practices

Ankit Agrawal, Solution Developer, TCS
Sachin Goswami, TCS

Abstract

IT industry is constantly evolving by transforming thoughts into cutting edge products and solutions to provide better services to the customers. Linear Tape File System (LTFS) is one such file system that overcomes the drawbacks of the traditional tape storage technology such as sequential navigation. SNIA’s LTFS Technical work group is adapting to emerging market needs and developing/enhancing Liner Tape File System (LTFS) specifications for tape technology.

TCS has also started working on some ideas in LTFS space and in this proposal we will share our views on how to integrate SSD as a cache with LTFS tape system to transparently deliver the best benefits for Object Base storage. This combination will allow us to deliver a reliable and economic storage solution without sacrificing performance. We will also talk about the potential challenges in our approach and best practices that can be adopted to overcome these challenges

Learning Objectives

• Understanding LTFS specification provided by SNIA
• Understanding SSD functionality
• Understanding in integration LTFS with SSD

A Pausable File System

James Cain, Principal Software Architect, Quantel Limited

Abstract

As storage developers we are all obsessed with speed. This talk gives a different take on speed – how slow can we go? Can we even stop? If so for how long? The talk will also analyze why this is interesting, and demonstrate that the file system interface – and the way all software depends upon it – is one of the most powerful abstractions in operating systems.

The presenter will use his own implementation of an SMB3 server (running in user mode on Windows) to demonstrate the effects of marking messages as asynchronously handled and then delaying responses – in order to build up a complete understanding of the semantics offered by a pausable file system.

This exploration of the semantics of slow responses will demonstrate that researching slowing down can bear as much fruit as speeding up!

Learning Objectives

• Understanding The Inversion of Control design pattern and how it can be applied to an implementation of a file server.
• Exploring how the file system interface can be seen as a contract and that the semantics of that contract can be exploited for innovative uses.
• Demonstrating that implementing a fairly small subset of SMB3 as a server is enough to conduct research in file systems.

Storage Solutions for Tomorrow's Physics Projects

Ulrich Fuchs, Service Manager, CERN

Abstract

The unique challenges in the field of nuclear high energy physics are already pushing the limits of storage solutions today, however, the projects planned for the next ten years call for storage capacities, performance and access patterns that exceed the limits of many of today's solutions.

This talk will present the limitations in network and storage and suggest possible architectures for tomorrow's storage implementations in this field and show results of first performance tests done on various solutions (Lustre, NFS, Block Object storage, GPFS ..) for typical application access patterns.

Learning Objectives

• Shared file system and storage performance requirements in science workloads
• Setup and results of performance measurements of different file systems: the LUSTRE FS, NFS, BOS, GPFS
• Technology differences between several file systems and storage solutions

Support for New Storage Technologies by the Windows Operating System

Neal Christiansen, Principal Development Lead, Microsoft

Abstract

This will describe the changes being made to the Windows OS, its file systems and storage stack in response to new evolving storage technologies.

Learning Objectives

• How windows is adapting to new storage technologies

ZFS Async Replication Enhancements

Richard Morris, Software Engineer, Oracle
Peter Cudhea, Software Engineer, Oracle

Abstract

This presentation explores some design decisions around enhancing the zfs send and zfs receive commands to transfer already compressed data more efficiently and to recover from failures without re-sending data that has already been received.

Learning Objectives

• High level understanding - how ZFS provides an efficient platform for async replication
• Finding stability in the chaos - tension between what's stable in an archive and what isn't
• Resolving significant constraints - why something simple turned out not to be not so simple

ReFS v2: Cloning, Projecting, and Moving Data

J.R. Tipton, Development Lead, Microsoft

Abstract

File systems are fundamentally about wrapping abstractions around data: files are really just named data blocks. ReFS v2 presents just a couple new abstractions that open up greater control for applications and virtualization.

We'll cover block projection and cloning as well as in-line data tiering. Block projection makes it easy to efficiently build simple concepts like file splitting and copying as well as more complex ones like efficient VM snapshots. Inline data tiering brings efficient data tiering to virtualization and OLTP workloads.

Learning Objectives

• How ReFS v2 exploits its metadata store to project file blocks at a fine granularity
• How metadata is managed in ReFS v2
• How data movement between tiers can happen efficiently while maintaining data integrity

Achieving Coherent and Aggressive Client Caching in Gluster, a Distributed System

Poornima Gurusiddaiah, Software Engineer, Red Hat
Soumya Koduri, Red Hat

Abstract

The presentation will be about how to implement:

• File system notifications
• Leases

In a distributed system and how these can be leveraged to implement a client side coherent and aggressive caching.

Learning Objectives

• Designing File system notification in Distributed System
• Designing leases in Distributed System
• Designing client caching in Distributed file system
• Gluster and the benefits of xlator modeling

Petabyte-scale Distributed File Systems in Open Source Land: KFS Evolution

Sriram Rao, Partner Scientist Manager, Microsoft

Abstract

Over the past decade, distributed file systems based on a scale-out architecture that enables managing massive amounts of storage space (petabytes) have become commonplace. In this talk, I will first provide an overview of OSS systems (such as HDFS and KFS) in this space. I will then describe how these systems have evolved to take advantage of increasing network bandwidth in data center settings to improve application performance as well as storage efficiency. I will talk about these aspects by highlighting two novel features, multi-writer atomic append and (time-permitting) distributed erasure coding. These capabilities have been implemented in KFS and deployed in production settings to run analytic workloads.

Designing Highly Optimized Storage Accelerators with Customizable FPGA-Based Flash Server Cards

Ravi Thummarukudy, President and CEO, Mobiveil

Abstract

Previous generations of the Pure Storage FlashArray used InfiniBand RDMA as a cluster interconnect between storage controllers in a system. The current generation replaces this with PCI Express Non-Transparent Bridging. We will describe how we preserved the key attributes of high throughput, low latency, CPU offloaded data movement and kernel bypass while moving the interconnect from a discrete IB adapter to a CPU-integrated PCIe port using new technologies including Linux vfio and PCIe NTB.

Learning Objectives

• Key attributes of an RDMA transport
• Description of PCIe NTB
• Implementation of RDMA on PCIe NTB

PCI Express Non-Transparent Bridging for RDMA

Roland Dreier, Member of Technical Staff, Pure Storage

Abstract

Previous generations of the Pure Storage FlashArray used InfiniBand RDMA as a cluster interconnect between storage controllers in a system. The current generation replaces this with PCI Express Non-Transparent Bridging. We will describe how we preserved the key attributes of high throughput, low latency, CPU offloaded data movement and kernel bypass while moving the interconnect from a discrete IB adapter to a CPU-integrated PCIe port using new technologies including Linux vfio and PCIe NTB.

Learning Objectives

• Key attributes of an RDMA transport
• Description of PCIe NTB
• Implementation of RDMA on PCIe NTB

Concepts on Moving From SAS connected JBOD to an Ethernet connected JBOD (EBOD)

Jim Pinkerton, Partner Architect Lead, Microsoft

Abstract

Today’s Software Defined Storage deployments are dominated by SAS attached just-a-bunch-of-disks (JBOD), with some new drives moving to an Ethernet connected blob store interface. This talk examines the advantages of moving to an Ethernet connected JBOD, what infrastructure has to be in place, what performance requirements are needed to be competitive, and examines technical issues in deploying and managing such a product. The talk concludes with a real world example, including performance analysis.

Learning Objectives

• Understand the potential advantages of moving to an Ethernet connected JBOD (EBOD)
• Understand the architectural differences between a JBOD based approach and an EBOD based approach.
• Understand the complexities that are solved and that need to be solved, anchored in real world data, to be able to deliver an EBOD to the market.

RAIDShield: Characterizing, Monitoring, and Pro-actively Protecting Against Disk Failures

Ao Ma, Principal Engineer, EMC

Abstract

Modern storage systems orchestrate a group of disks to achieve their performance and reliability goals. Even though such systems are designed to withstand the failure of individual disks, failure of multiple disks poses a unique set of challenges. We empirically investigate disk failure data from a large number of production systems, specifically focusing on the impact of disk failures on RAID storage systems. Our data covers about one million SATA disks from 6 disk models for periods up to 5 years. We show how observed disk failures weaken the protection provided by RAID. The count of reallocated sectors correlates strongly with impending failures.

Learning Objectives

• Empirical investigation of hard disk failures in production systems
• Proactive protection of individual disk drives
• Proactive protection of RAID storage system
• Deployment results of the proactive protection in production system

Planning for the Next Decade of NVM Programming

Andy Rudoff, SNIA NVM Programming TWG, Intel

Abstract

We imagine a future where persistent memory is common in the data center. How will enterprise-class applications leverage this resource? How will middleware, libraries, and application run-time environments change? In this talk, Andy will describe how emerging NVM technologies and related research are causing a change to the software development ecosystem. Andy will describe use cases for load/store accessible NVM, some transparent to applications, others non-transparent. Starting with current examples of NVM Programming, Andy will describe where he believes this is leading us, including the likelihood that programmers in the future must comprehend numerous types of memories with different qualities and capabilities.

Why the Storage You Have is Not the Storage Your Data Needs

Laz Vekiarides, CTO and Co-founder, ClearSky Data

Abstract

How much of your data is actually hot? How much are you storing on your hottest tier? If you’re relying on traditional storage systems, the difference between those two responses could be major. Enterprises are investing in the fastest, most expensive options available for their storage without fully understanding the nature of each workload. For instance, many storage developers lack access to storage monitoring tools that base insights on hard evidence, but when you consider the breakdown of most enterprise workloads, the amount of wasted resources you may be working with can be surprising.

It’s imperative that some detailed research be conducted on the nature of enterprise workloads and the placement of hot, warm and cold data before a storage system is built. In this session, CTO of ClearSky Data Laz Vekiarides will share some of the questions every storage architect should ask, such as:

Learning Objectives

• Is your most expensive data on your most expensive storage tier?
• What sorts of latencies are your applications seeing?
• What real-world benefits can be derived from virtualized tiering technology? Where does it fail?

Innovator, Disruptor or Laggard, Where will your storage applications live? Next generation storage

Bev Crair, Vice President and General Manager, Storage Group, Intel

Abstract

With the emergence of Cloud Service Providers, new technology innovations, and elevated customer expectations, Enterprise Information Technologists continue to be faced with scalability and cost pressures. Through next-generation efficient, cost-effective scalable architectures, IT is transforming from a Cost-Center Service Provider to a valued Business Partner.

It's imperative that storage developers understand the drivers of this transition, how to leverage the open source community, embrace next generation memory storage and cloud service provider best practices in response to their infrastructure and workloads. In this session, Vice President and General Manager, Bev Crair, will discuss the leadership role Intel(r) is playing in driving the open source community for software defined storage, server based storage and upcoming technologies that will shift how storage is architected.

Learning Objectives

• Understanding the role of software development in accelerating the storage transformation
• Understanding of the technology trends in the storage industry
• Understanding the critical role of server based storage in a software defined storage
• Understanding of Intel(r) technologies for storage and the open source community

DMTF Redfish Overview

Jeff Autor, Distinguished Technologist, Hewlett-Packard

Abstract

The DMTF’s Scalable Platforms Management Forum (SPMF) is working to create and publish an open industry standard specification, called “Redfish” for simple, modern and secure systems management using RESTful methods and JSON formatting. This session will cover the design tenets, protocol and payload, expected deliverables and time frames.

Learning Objectives

• Understanding the Redfish goals
• Understanding how Redfish applies to storage topics
• Developing Redfish support
• Understanding the Redfish Open Source efforts

The State of SMI-S – The Standards Based Approach for Managing Infrastructure

Chris Lionetti, Reference Architect, NetApp

Abstract

SMI-S is the standards-based way to expose, modify, and consume the storage used in data centers. SMI-S can discover storage resources such as RAID groups and primordial disks, it can configure capabilities like thin provisioning, initiator groups and mappings for file shares or exports, and it can be used to monitor the ongoing operations of storage infrastructure.

These activities are cross-vendor and cover end-to-end operations from the host through the switching infrastructure to the storage controllers and down to the logical and physical storage devices. This session will appeal to Data Center Managers, Architects, and Development Managers, and will approach the topic from an ‘Operations’ perspective.

The audience will receive a fundamental grounding in the SMI-S and a clear understanding of its value in a production environment. This session will also address the newly created SMI-S getting started guide.

Learning Objectives

• Understand Value Prop for SMI-S adherence
• Know differing approaches to deploy SMI-S
• Appreciate how consumers gather/user SMI-S data
• Realistically Plan to develop an SMI-S provider

Enterprise-Grade Array-Based Replication and Disaster Recovery with SMI-S, Windows Server, System Center and Azure Site Recovery

Amit Virmani, Senior Software Engineer, Microsoft Corp
Jeff Li, Senior Software Engineer, Microsoft Corp

Abstract

Microsoft System Center Virtual Machine Manager(SCVMM) automates the complete end to end discovery and integration to leverage replication capabilities provided by our enterprise storage partners using SMI-S. Windows Server provides native support for SMIS providers that SCVMM can utilize. Building on top of SCVMM primitives, Azure Site Recovery provides the end to end disaster recovery and orchestration solution automating the creation and management of all target objects including storage and compute. Microsoft is working with multiple storage partners to deliver this functionality: EMC, NetApp, HP, Hitachi, IBM, Huawei, Dell Compellent and Fujitsu.

Learning Objectives

• Understand what ASR Disaster Recovery solution means – what is Planned Failover, Unplanned Failover, Test Failover, etc. Learn how SMIS provides the primitives in Replication Services profile
• Understand how SCVMM leverages the replication discovery and management profile using the developers guide distributed to partners. This includes the changes made in the Replication Services
• Deep dive into Virtual Machine Manager leveraging Pass Through service in Windows Server 2012 R2 to discover and manage replication capabilities. Capabilities include discovery, replica provisioning
• Understand how ASR orchestrates the replication automation and learn how replications groups enable multiple virtual machines to be protected and replicated together

Introducing SSM

Alan Yoder, Principal Engineer Storage Standards, Huawei/Futurewei

Abstract

SSM stands for Simple Storage Management, a small, simplified storage management interface that is compatible and interoperable with the full SMI-S management interface. It is especially suited for entry-level storage and client access to same.

Learning Objectives

• Learn what SSM is
• Learn what SSM is good for
• Learn how SSM interoperates with SMI-S

Benefits of NVMe Over Fabrics and Demonstration of a Prototype

Rob Davis, VP of Storage Technology, Mellanox Technologies

Abstract

NVMe offers a faster way to connect to solid state storage than traditional SAS and SATA interfaces, which were designed for spinning disk. It eliminates the SCSI layer and supports better bandwidth, IOPS, and latency than 12Gb SAS. However, traditional NVMe keeps the storage devices “captive” within the server or storage box and does not scale across distance, multiple storage nodes, or hundreds of PCIe devices. NVM Express, Inc. has proposed a standard to support remote access of NVMe devices across high speed, low-latency fabrics. Mellanox will present examples and prototype performance results of running the forthcoming standard over RDMA interconnects such as InfiniBand and RoCE (RDMA over Converged Ethernet).

Learning Objectives

• Understand the benefits of NVMe over SAS or SATA interfaces.
• Learn how NVMe works with RDMA on InfiniBand or RoCE fabrics.
• See test results from an NVMe Over Fabrics prototypes involving NVMe flash devices
• Understand how NVMe Over Fabrics works with 100Gb Ethernet

Implementing NVMe Over Fabrics

Wael Noureddine, VP Technology, Chelsio Communications

Abstract

NVMe is gaining momentum as the standard high performance disk interface that eliminates various bottlenecks in accessing PCIe SSD devices. NVMe over Fabrics extends NVMe beyond the confines of a PCIe fabric by utilizing a low latency network interconnect such as iWARP RDMA/Ethernet to attach NVMe devices. iWARP is unique in its scalability and reach, practically eliminating constraints on the architecture, size and distance of a storage network. This talk presents Chelsio’s open-source generic block device based implementation and benchmark results that illustrate the benefits in performance and efficiency of the new fabric, opening the way to unprecedented storage performance and scale.

Learning Objectives

• Introduction to NVMe, NVMe over Fabrics and peer-to-peer communications
• Develop a complete understanding of Chelsio block device implementation and performance results
• Learn how iWARP fits well with NVMe

A Cost Effective, High Performance, Highly Scalable, Non-RDMA NVMe Fabric

Bob Hansen, VP Systems Architecture, Apeiron Data Systems

Abstract

Large server count, scale out cluster applications require non-volatile storage performance well beyond the capabilities of legacy storage networking technologies. Until now the only solution has been to load SSDs directly into the cluster servers. This approach delivers excellent raw storage performance, but introduces many disadvantages including: single points of failure, severely limited configuration/provisioning flexibility and added solution cost. This presentation discusses a new, scalable, very high performance storage architecture that delivers all the simplicity and promise of DAS with the efficiency and capability of network storage, at an industry leading cost point.

SNIA Tutorial:
SCSI Standards and Technology Update

Marty Czekalski, President, SCSI Trade Association
Greg McSorley, Vice President, SCSI Trade Association

Abstract

SCSI continues to be the backbone of enterprise storage deployments and continues to rapidly evolve by adding new features, capabilities, and performance enhancements. This presentation includes an up-to-the-minute recap of the latest additions to the SAS standard and road maps, the status of 12Gb/s SAS deployment, advanced connectivity solutions, MultiLink SAS™, SCSI Express, and 24Gb/s development. Presenters will also provide updates on new SCSI features such as atomic writes, Zoned Block Commands (ZBC) and Storage Intelligence which provides mechanisms for improved efficiency, performance and endurance with solid state devices.

Learning Objectives

• Attendees will learn how SAS continues to grow and thrive, in part, because of the Advanced Connectivity Roadmap, which offers a solid connectivity scheme based on the versatile Mini-SAS HD connector
• The latest development status and design guidelines for 12Gb/s SAS will be discussed as well current status for extending SAS to 24Gb/s.
• Attendees will receive updates on new SCSI feature such as atomic writes, Zoned Block Commands and Storage Intelligence which provides mechanisms for improved efficiency, performance and endurance wit

Growth of the iSCSI RDMA (iSER) Ecosystem

Rob Davis, VP of Storage Technology, Mellanox Technologies

Abstract

iSCSI RDMA (iSER) has been the fastest available block storage protocol for several years but the number of commercially available storage targets has previously limited. Now new storage solutions from vendors such as NetApp are supporting iSER, along with iSER initiators in new environments such as FreeBSD. This makes it easier for both cloud service providers and enterprises to deploy iSER. In addition, improvements to the iSER and Linux SCSI layers allow faster iSER performance than before over both InfiniBand and 40Gb Ethernet links.

Learning Objectives

• Understand similarities and differences between iSER and standard iSCSI over TCP
• See the growth of the iSER ecosystem for both storage target and initiator support
• Learn about new features supported in iSER including T10-DIF, initiator/target discovery over RDMA, and performance optimization.
• See benchmark results from the latest version of iSER running over 40/56Gb Ethernet and FDR InfiniBand networks
• See metrics for using iSER with the latest flash storage

Title Pending

Pending Speaker

Abstract

Pending

Introduction to Highly Available NFS Server on Scale-Out Storage Systems Based on GlusterFS

Soumya Koduri, Senior Software Engineer, Red Hat India
Meghana Madhusudhan, Software Engineer, Red Hat

Abstract

Many enterprises still heavily depend on NFS to access their data from different operating systems and applications. NFS-Ganesha is a user-space file server that supports NFSv3, NFSv4, NFSv4.1 as well as pNFS.

GlusterFS has now added NFS-Ganesha server to its NFS stack to eventually replace native Gluster-NFS server which supports only NFSv3. The integration with NFS-Ganesha now means additional protocol support w.r.t. NFSv4, better security and authentication mechanisms for enterprise use. The upcoming release of GlusterFS (3.7) introduces Clustered or multi-head active/active NFS support using Pacemaker and Corosync for better availability. There is also tighter integration with Gluster CLI to manage NFS-Ganesha exports. This presentation is aimed at providing a basic overview of the entire solution and step-by-step configuration.

Learning Objectives

• Basic architecture walk-through of nfs-ganesha and what the integration with GlusterFS means.
• Architecture overview of the multi-head active/active highly available NFS solution.
• Step-by-step guide to configure NFS-Ganesha on GlusterFS using newly introduced CLI options.
• Requirements and best practice recommendations for HA configuration for NFSGanesha with GlusterFS.

Instantly Finding a Needle of Data in a Haystack of Large-Scale NFS Environment

Gregory Touretsky, Solutions Architect, Intel

Abstract

Intel Design environment heavily depends on a large scale NFS infrastructure with 10s of PBs of data. Global Name space helps to navigate this large environment in a uniform way from 60,000 compute servers.

But what if a user doesn't know where the piece of data he is looking for is located?

Our customers used to spend hours waiting for recursive ""grep"" commands' completion - or preferred not to bother with some less critical queries.

In this talk, we'll cover how Intel IT has identified an opportunity to provide a faster way to look for an information within this large-scale NFS environment. We'll review various open source solutions which were considered, and how we've decided to implement a mix of home-grown scalable NFS crawler with open source ElasticSearch engine to index parts of our NFS environment.

As part of this talk we'll discuss various challenges and our ways to mitigate them, including:

• crawler scalability required to index large amounts of dynamically changing data within pre-defined indexing SLA
• Index scalability and performance requirements
• Relevancy of the results presented in search queries by customers
• User interface considerations
• Security aspects of the index access control

This might be an interesting conversation for both storage vendors - covering a useful feature which might be implemented as a part of NFS environment, and for storage customers who may benefit from such capability.

Learning Objectives

• How to implement scalable indexing and search on top of large scale NFS
• Scalable crawling with controlled performance impact on shared file servers
• Security aspects of data index and search representation

Scalable Metadata in NFSv4

Matt Benjamin, CTO, CohortFS
Casey Bodley, Software Developer, CohortFS

Abstract

With NFS version 4.1, pNFS was introduced to provide clients with direct access to storage devices to reduce the bottleneck of a single NFS server.

The pNFS Metadata Striping draft applies pNFS scale-out to metadata, introducing cooperating pNFS MDS servers, striping of files and directory entries across multiple servers, a new lightweight redirection mechanism for OPEN, GETATTR, CREATE, and other mutating operations, and new parallel directory enumeration. pNFS Metastripe breaks the MDS bottleneck in pNFS, and gives NFS the ability to operate efficiently at larger scales and under more demanding metadata workloads.

Learning Objectives

• Describe metadata scalability challenges in enterprise, HPC, and cloud
• Introduce the pNFS Metastripe protocol, including changes since early proposal drafts by Eisler0
• Present insights from CohortFS implementation of metastripe in Ganesha and Ceph
• Review current state and progress of IETF draft

pNFS/RDMA: Possibilities

Chuck Lever, Linux Kernel Architect, Oracle

Abstract

The presenter will discuss possible ways to merge pNFS with persistent memory and fast storage fabrics.

Learning Objectives

• Challenges of NFS with fast storage
• What is a pNFS layout?
• Does pNFS work on RDMA transports
• Implications of pNFS/RDMA

SNIA Tutorial:
Object Drives: A New Architectural Partitioning

Mark Carlson, Principal Engineer, Industry Standards, Toshiba

Abstract

A number of scale out storage solutions, as part of open source and other projects, are architected to scale out by incrementally adding and removing storage nodes. Example projects include:

• Hadoop’s HDFS
• CEPH
• Swift (OpenStack object storage)

The typical storage node architecture includes inexpensive enclosures with IP networking, CPU, Memory and Direct Attached Storage (DAS). While inexpensive to deploy, these solutions become harder to manage over time. Power and space requirements of Data Centers are difficult to meet with this type of solution. Object Drives further partition these object systems allowing storage to scale up and down by single drive increments.

This talk will discuss the current state and future prospects for object drives. Use cases and requirements will be examined and best practices will be described.

Learning Objectives

• What are object drives?
• What value do they provide?
• Where are they best deployed?

Beyond LBA: New Directions in the Storage Interface

Abhijeet Gole, Senior Director of Engineering, Toshiba

Abstract

New storage devices are emerging that go beyond the traditional block interface and support key value protocol interfaces. In addition some of the emerging devices include capabilities to run applications on the device itself. This talk will explore the paradigm shift introduced by these new interfaces and modes of operation of storage devices.

Learning Objectives

• The developer will learn the key technical details of the OpenKinetic protocol as an example
• Learn how key value protocols map to the semantics that modern days developers are familiar with
• Learn how your software can be modified to take advantage of these new interfaces and the impact of running storage applications on the devices
• Leave with an in depth understanding of the new interfaces and how they apply to your project
• Leave with an understanding of the current adoption by customers and other developers

Designing SSD-Friendly Applications

Zhenyun Zhuang, Senior Performance Engineer, LinkedIn

Abstract

SSD is being increasingly adopted for improved application performance. SSD works quite differently from its HDD counterpart. Hence, many conventional applications that are designed and optimized for HDD may not fit well to SSD characteristics. In particular, developers typically know little about SSD and simply treat SSD as a "faster" HDD. In this talk, we will present a set of guidelines of how to design SSD-friendly applications which not only maximize the application performance, but maximize the SSD life.

Learning Objectives

• Developers can gain thorough understanding of SSD performance
• Developers are able to design improved applications, data structures and algorithms to maximize performance with SSD

Load-Sto-Meter: Generating Workloads for Persistent Memory

Doug Voigt, Distinguished Technologist, HP

Abstract

New persistent memory technologies allow IO to be replaced with memory mapped files where the primary operations are load, store, flush and fence instructions executed by CPU’s. This creates a new need for software generate well understood workloads made up of those operations to characterize implementations of persistent memory related functionality. This session describes a proposal for such a workload generator which could play a role for PM solutions similar to IOMeter for IO.

Learning Objectives

• Learn how pure load/store workloads can be generated
• Learn about parameters that should govern the creation of pure load/store workloads

Application-Level Benchmarking with SPEC SFS 2014

Nick Principe, Senior Software Engineer, EMC
Vernon Miller, Performance Engineer, IBM

Abstract

A technical deep-dive into the four SPEC SFS 2014 workloads at different levels of the storage stack, and how client performance and configuration can affect benchmark results. Multiple storage protocols will be addressed, including NFS, SMB, and FC – yes, you CAN test a block storage array with a file-level benchmark!

Learning Objectives

• Deep analysis of storage solution performance
• Effects of protocol and operating system on storage performance measurement
• Observe workload changes as traffic flows from client through storage to disk

Online Cache Analysis And Its Applications For Enterprise Storage Systems

Irfan Ahmad, CTO, CloudPhysics

Abstract

It is well-known that storage cache performance is non-linear in cache size and the benefit of caches varies widely by workload. This means that no two real workload mixes have the same cache behavior! Existing techniques for profiling workloads don’t measure data reuse, nor does they predict changes in performance as cache allocations are varied. Since caches are a scarce resource, workload-aware cache behavior profiling is highly valuable with many applications.

We will describe how to make storage cache analysis efficient enough to be able to put directly into a commercial cache controller. Based on work published at FAST '15, we'll show results including computing miss ratio curves (MRCs) on-line in a high-performance manner (~20 million IO/s on a single core).

The technique enables a large number of use cases in all storage device. These include visibility into cache performance curves for sizing the cache to actual customer workloads, troubleshooting field performance problems, online selection of cache parameters including cache block size and read-ahead strategy to tune the array to actual customer workloads, and dynamic MRC-guided cache partitioning which improve cache hit ratios without adding hardware. Furthermore, the work applies to all types of application caches not just those in enterprise storage systems.

Learning Objectives

• Storage cache performance is non-linear
• Benefit of caches varies widely by workload mix
• Working set size estimates don't work for caching
• How to make storage cache analysis available in a commercial cache controller
• New use cases for cache analysis in enterprise storage systems

NVDIMM-SSDs Tested to the SNIA SSD Performance Test Specification

Eden Kim, CEO, Calypso Systems

Abstract

The performance benefits of parallel processing technology have led the migration of existing RDBMS applications to big data technologies such as Hadoop and Hive. This migration brings in additional challenges to catch up performance of parallel RDBMS using parallelism for data processing in commodity based nodes’ cluster- this raises the need to replace the traditional file systems such as HDFS with parallel file systems such as Lustre. Moreover, convergence of HPC with Big data motivates further to have unified file system to avoid data transfer across different subsystems.

In this presentation, we share performance comparison of HDFS and Luster for FSI, Telecom and Insurance SQL workload evaluating the performance of the application on an integrated stack with Hive and Lustre through Hive extensions such as Hadoop Adapter for Lustre (HAL) developed by Intel, while comparing the performance against the Hadoop Distributed File System (HDFS). The environment used for this evaluation shall be a 16 nodes HDDP cluster hosted in the Intel Big Data Lab in Swindon (UK). The cluster will be divided into two clusters. One 8 node cluster was set up with CDH 5.0.2 and HDFS and another 8 node was set up with CDH 5.0.2 connected to Lustre through Intel HAL. We use Intel Enterprise Edition for Lustre 2.0 for the experiment based on Lustre 2.5. Both the systems will be evaluated on performance metric ‘query average response time’ for FSI workload.

Learning Objectives

• Performance comparison of MR appln
• Performance scalability of HDFS
• Performance scalability of Lustre
• Performability of Lustre

Storage Performance Analysis for Big Data Processing

Da Qi Ren, Staff Research Engineer, Huawei Technologies
Zane Wei, Director, Huawei Technologies

Abstract

End to end big data benchmarking has become an extreme attention of ICT industry, the related techniques are being investigated by numerous hardware and software vendors. Storages, as one of the core components of a data center system, need specially designed approaches to measure, evaluate and analyze their performance. This talk introduces our methods to create the storage performance model based on workload characterization, algorithm level behavior tracing and capture, and software platform management. The functionality and capability of our methodology for quantitative analysis of big data storage have been validated through benchmarks and measurements performed on real data center system.

Learning Objectives

• Storage Performance Measurement and Evaluation
• Scalable and Distributed Storage Systems
• Best practices architecture for Big Data
• Performance analysis

Preparing Applications for Persistent Memory

Doug Voigt, Distinguished Technologist, HP

Abstract

New persistent memory technologies promise to revolutionize the way applications store data. Many aspects of application data access will need to be revisited in order to get full advantage of these technologies. The journey will involve several new types of libraries and ultimately programming language changes. In this session we will use the concepts of the SNIA NVM Programming Model to explore the emerging landscape of persistent memory related software from an application evolution point of view.

Learning Objectives

• Learn how applications can navigate the transitions created by persistent memory
• Learn how additional software and services can help.

Managing the Next Generation Memory Subsystem

Scott Kirvan, Software Architect, Intel

Abstract

New memory technologies are emerging which bring substantial performance and reliability benefits, but these benefits are only achieved with careful provisioning and on-going management of the memory subsystem. Non-volatile memory technologies in particular have unique characteristics that require rethinking memory and storage management. This talk begins with an overview of emerging memory device types with a focus on non-volatile DIMMs. We’ll cover the management concepts and features of these new technologies and put them in the context of overall memory subsystem and server management. The talk concludes with an overview of SNIA, DMTF and other standards that are being introduced to drive interoperability and encourage the development of memory subsystem management tools.

Learning Objectives

• Discuss the features of emerging memory technologies and the system management challenges that result from these new features
• Discover the concepts, practices and tools that administrators can use to discover, provision, and manage the growing complexity of the memory subsystem
• Review the standardization efforts, documentation, and open source code available to developers looking to get started with memory management development projects

SNIA Tutorial:
The NVDIMM Cookbook: A Soup-to-Nuts Primer on Using NVDIMMs to Improve Your Storage Performance

Jeff Chang, VP Marketing and Business Development, AgigA Tech
Arthur Sainio, Senior Director Marketing, Smart Modular

Abstract

Non-Volatile DIMMs, or NVDIMMs, have emerged as a go-to technology for boosting performance for next generation storage platforms. The standardization efforts around NVDIMMs have paved the way to simple, plug-n-play adoption. If you're a storage developer who hasn't yet realized the benefits of NVDIMMs in your products, then this session is for you! We will walk you through a soup-to-nuts description of integrating NVDIMMs into your system, from hardware to BIOS to application software. We'll highlight some of the "knobs" to turn to optimize use in your application as well as some of the "gotchas" encountered along the way.

Learning Objectives

• Understand what an NVDIMM is
• Understand why an NVDIMM can improve your system performance
• Understand how to integrate an NVDIMM into your system

Remote Access to Ultra-low-latency Storage

Tom Talpey, Architect, Microsoft

Abstract

A new class of ultra-low latency storage is emerging, including Persistent Memory (PM), as well as advanced nonvolatile storage technologies such as NVMe. The SNIA NVM TWG has been exploring these technologies and has more recently prepared a white paper for requirements of remotely utilizing such devices. Remote Direct Memory Access (RDMA), arbitrated by file and block storage protocols, is a clear choice for this access, but existing RDMA and storage protocol implementations incur latency overheads which impact the performance of the solution. And while raw fabric block protocols can address latency overheads, they do not address data integrity, management and sharing.

This talk explores the issues, and outlines a path-finding effort to make small, natural extensions to RDMA and upper layer storage protocols to reduce these latencies to acceptable, minimal levels, while preserving the many advantages of the storage protocols they extend.

Learning Objectives

• Learn key technologies enabling remote access to new storage media, such as NVM and PM
• Understand the issues in making full use of PM technologies remotely, with today’s protocols
• Explore a path to fully access the benefits of remote access to PM devices in the future

Solving the Challenges of Persistent Memory Programming

Sarah Jelinek, Senior SW Engineer, Intel

Abstract

Programming with persistent memory is hard, similar to the type of programming a file system developer does because of the need to write changes out in a way that maintains consistency. Applications must be re-architected to change data stored in two tiers (DRAM and storage) into three tiers (DRAM, pmem and storage). This presentation will review key attributes of persistent memory as well as outline architectural and design considerations for making an application persistent memory aware. This discussion will conclude with examples showing how to modify an application to provide consistency when using persistent memory.

Learning Objectives

• Introduce how persistent memory differs from DRAM and standard storage for storing application data
• Show examples of the architectural considerations for making an application persistent memory aware
• Give examples of how to modify an existing application to utilize persistent memory
• Discuss the open source Non-Volatile Memory Library (NVML) available on GitHub for use to help with persistent memory programming

RDMA with PM: Software Mechanisms for Enabling Persistent Memory Replication

Chet Douglas, Principal SW Architect, Intel

Abstract

With the emergence of persistent memory, the need to replicate data across multiple clusters arises. RDMA to persistent memory provides a mechanism to replicate data remotely but requires SW to implicitly make previously written data persistent. This presentation will review key HW components involved in RDMA and introduce several SW mechanisms that can be utilized with RDMA with PM. The discussion will conclude with a review of performance implications of each solution and methods that can be utilized to model the latencies associated with RDMA and PM.

Learning Objectives

• Introduce HW Architecture concepts of Intel platforms that will affect RDMA usages with PM
• Introduce SW Mechanisms that can be utilized in RDMA Application SW to make RDMA Write data persistent
• Review detailed sequences, platform performance implications, and Pros and Cons for each proposed SW Mechanism
• Intel platform configuration details for using RDMA with PM
• Overview of Intel’s SW Application for evaluating RDMA performance and SW modeling of new Intel CPU instructions for use with PM

Progress Report On Advanced Non Volatile Storage Technologies

Thomas Coughlin, President, Coughlin Associates
Edward Grochowski, Storage Consultant, Self Employed

Abstract

Today, HDD areal densities are nearing 1 Terabit per sq. in. and Flash memories are applying lithographic exposures much smaller than 28 nm, or are advancing to 3D structures. These products require new and demanding process techniques to maintain storage market growth and cost competitiveness. Alternative new non volatile memories and storage technologies as STT RAM, RRAM, PCM and several others are becoming more attractive to meet this growing demand for storage and memory bytes. This study will address the status of NVM device technologies and review requirements in process, equipment and innovations. Progress in implementing these devices as well as future concerns to achieve economic implementation will be outlined. The dependency on CMOS driver devices for NVM will be discussed to attain a high density memory or storage alternative. A concluding assessment in implementation of NVM will be made in the context of HDD and Flash memories.

The Long-Term Future of Solid State Storage

Jim Handy, General Director, Objective Analysis

Abstract

Today solid-state storage is an extension of established storage technologies extended by hiding flash behind existing hardware & software protocol layers. Over time these layers will be abandoned in favor of new architectures.

This presentation will examine research of new solid state memory and storage types, and new means of integrating them into highly-optimized computing architectures. This will lead to a discussion of the way that these will impact the market for computing equipment.

Learning Objectives

• Understanding how solid state storage arrived at its current state
• A background on the changes in store for memory chips
• How computing architectures will change to squeeze the highest performance from these chips
• How the market will react to these changes

Nonvolatile Memory (NVM), Four Trends in the Modern Data Center, and the Implications for the Design of Next Generation Distributed Storage Platforms

David Cohen, System Architect, Intel
Brian Hausauer, Hardware Architect, Intel

Abstract

There are four trends unfolding simultaneously in the modern Data Center: (i) Increasing Performance of Network Bandwidth, (ii) Storage Media approaching the performance of DRAM, (iii) OSVs optimizing the code path of their storage stacks, and (iv) single processor/core performance remains roughly flat. A direct result of these trends is that application/workloads and the storage resources they consume are increasingly distributed and virtualized. This, in turn, is making Onload/Offload and RDMA capabilities a required feature/function of distributed storage platforms. In this talk we will discuss these trends and their implications on the design of distributed storage platforms.

Learning Objectives

• Highlight the four trends unfolding in the data center
• Elaborate on the implication of these trends on design of modern distributed storage platforms
• Provide details on how onload/offload mechanisms and RDMA become feature/function requirements for these platforms in the near-future

Developing Software for Persistent Memory

Thomas Willhalm, Senior Staff Software Engineer, Intel
Karthik Kumar, Senior Staff Software Engineer, Intel

Abstract

NVDIMMs provide applications the ability to access in-memory data that will survive reboots: this is a huge paradigm shift happening in the industry. Intel has announced new instructions to support persistence. In this presentation, we educate developers on how to take advantage of this new kind of persistent memory tier. Using simple practical examples [1] [2], we discuss how to identify which data structures that are suited for this new memory tier, and which data structures are not. We provide developers a systematic methodology to identify how their applications can be architected to take advantage of persistence in the memory tier. Furthermore, we will provide basic programming examples for persistent memory and present common pitfalls.

Learning Objectives

• NVDIMMs have the potential to be a game changer for applications, as they offer the ability to access “in-memory data” that will survive reboots.
• In this presentation, we educate developers on how to take advantage of this new kind of persistent memory tier.
• Furthermore, we will provide basic programming examples for persistent memory and present common pitfalls.

Using iSCSI or iSER?

Ásgeir Eiriksson, Chief Technology Officer, Chelsio Communications

Abstract

This talk will demystify the relationship and relative performance and capabilities of iSCSI and iSER. The talk provides an introduction to iSER and its position in an iSCSI environment, and presents performance results to compare the two protocols when both are processed in hardware within an HBA. The talk concludes with a set of recommendations on deploying iSCSI and iSER within storage networks.

Learning Objectives

• Develop an understanding of iSCSI and iSER protocol stacks with comparison of capabilities for each
• Learn to recognize the performance benefits and benchmark results
• Have a clear understanding of when to use iSER

Linux SMB3 and pNFS - Shaping the Future of Network File Systems

Steven French, Principal System Engineer, Samba team/Primary Data

Abstract

Network File Systems, needed for accessing everything from low end storage, to Windows and Mac servers, to high end NAS, continue to evolve. NFS and SMB, the two dominant network storage protocols, also continue to improve with exciting new features in their most recent dialects. And the Linux clients continue to improve their implementation of these protocols, recently adding security and performance enhancements for SMB3 and new pNFS layout types along with the NFSv4.2 support in the NFS client.

This presentation will discuss some of the recent changes in network file system support in Linux including enhanced CIFS/SMB2/SMB3 support in the kernel client, and also new developments in the NFS client. It will also discuss in progress work on new protocol features for improved performance, clustering scalability, reliability and availability. It will also compare and contrast some of the key features of the SMB3 and NFS Linux clients.

Learning Objectives

• Understanding key features and limitations of the SMB3 support in Linux client
• Understanding key features and limitations of the NFS client in Linux client
• Understanding which protocol (and dialect) is better for common use cases
• Understanding key differences between NFSv4.2 and SMB3.1
• Understanding common SMB3 client configuration choices and why they are useful

Move Objects to LTFS Tape Using HTTP Web Service Interface

Matt Starr, Chief Technical Officer, Spectra Logic
Jeff Braunstein, Developer Evangelist, Spectra Logic

Abstract

Tape has always been a reliable, low-cost, green medium for long-term storage needs. However, moving objects to tape has sometimes been challenging and expensive. The DS3 protocol, which is an extension of the S3 protocol popularized by Amazon, provides easy storage to tape through HTTP web services. Additionally, DS3 uses the open Linear Tape File System (LTFS) format to store the objects on tape, making the data readable by many applications. With DS3, developers can easily create applications that move data to tape.

Learning Objectives

• Understand the DS3 HTTP protocol and how it can be used to move data to tape.
• Learn the difference between DS3 and S3.
• Review the software development kits (SDK) available to simplify DS3 application development.
• See the sample client applications that have been built using the DS3 SDKs.
• Understand the different components needed for DS3 application development.

Introduction to CoprHD: An Open Source Software Defined Storage controller - deep dive for developers

Anjaneya Chagam, Principal Engineer, Intel
Urayoan Irizarry, Consultant Software Engineer, EMC

Abstract

CoprHD is an open source software defined storage controller based on EMC's ViPR Controller. Software Defined Storage (SDS) has significant impact on how companies deploy and manage public and private cloud storage solutions to deliver on-demand storage services while reducing the cost. Similar to Software Defined Networking (SDN), SDS promises to simplify management of diverse provider solutions and ease of use. CoprHD open source SDS controller centralizes management and automation of multi-vendor storage systems to deliver on-demand policy driven storage services.

This presentation will cover CoprHD controller overview, architecture, driver and plug-in development that will help in jump starting your community development.

Learning Objectives

• Learn Software Defined Storage framework for managing cloud wide storage services
• Understand how to get engaged in CoprHD community for code contributions
• Understand CoprHD controller internals and integration with Orchestration frameworks (e.g., OpenStack, VMWare)
• Gain in-depth exposure to tools and techniques to write custom plug-ins using REST APIs
• Learn how to write drivers that will expose storage system advanced features using driver interfaces and controller extensions

Software Defined Storage Based on Direct Attached Storage

Vinod Shankar, Lead Software Developer, Microsoft
Claus Joergensen, Principal Program Mgr, Microsoft

Abstract

Software defined storage solutions can (and should) be based on industry-standard hardware! This talk will cover the technical architecture of the solution from the lead developer’s viewpoint, with design decisions explained, and optimization evaluated. We will also enumerate the wire protocol. We will demonstrate the end-to-end solution, scale and performance.

Hackers, Attack Anatomy and Security Trends

Ted Harrington, Executive Partner, Independent Security Evaluators

Abstract

Practical experience from implementing the OASIS Key Management Interoperability Protocol (KMIP) and from deploying and interoperability testing multiple vendor implementations of KMIP form the bulk of the material covered. Guidance will be provided that covers the key issues to require that your vendors address and how to distinguish between simple vendor tick-box approaches to standard conformance and actual interoperable solutions.

Learning Objectives

• In-depth knowledge of the core of the OASIS KMIP
• Awareness of requirements for practical interoperability
• Guidance on important of conformance testing

Mobile and Secure: Cloud Encrypted Objects Using CDMI

David Slik, Technical Director, Object Storage, NetApp

Abstract

Data wants to live in the cloud, and move freely between enterprises, phones, homes and clouds, but one major obstacle remains: How can your data be protected against alteration and disclosure? This session introduces the Cloud Encrypted Object Extension to the CDMI standard, which permits encrypted objects to be stored, retrieved, and transferred between clouds. Originating out of work to make CDMI usable for Electric Medical Records (EMR) application, Cloud Encrypted Objects are a standards-based way to encrypt data, verify integrity, and provide access to secured content, such that objects can freely move between clouds in a cross-protocol manner.

Learning Objectives

• Learn how Cloud Encrypted Objects are used by a client
• Learn how Cloud Encrypted Objects can move between clouds
• Learn about access control and delegation for Cloud Encrypted Objects
• Learn how Cloud Encrypted Objects can be stored and accessed from file systems, CDMI, S3, Swift and other repositories

OpenStack Swift On File: User Identity For Cross Protocol Access Demystified

Dean Hildebrand, IBM Master Inventor and Manager | Cloud Storage Software, IBM
Sasikanth Eda, Software Engineer, IBM

Abstract

Swift on File enables the swift object store hosted over clustered file system to have file as well as object access for the same data. Such multi protocol access enables various use-cases where data can be ingested via object and processed for analytics over file protocols (SMB/NFS/POSIX). In another manifestation, data can be accessed or shared by the user interchangeable via different protocols enabling user data sync n share across protocols.

For some of these use-cases, there is a strong need to have common User Identity management across object and file protocols so that one can leverage the underlying common file system features like quota management per user or group, per user/group placement policies on data or even have common authorization across file and object . In order to achieve this, the approaches need to ensure that objects created by an user via Swift is associated with the user's user ID (UID) and group ID (GID) which is same when the object is accessed by that user via file protocols like NFS/SMB/POSIX (where typically the ID's are stored in a centrally ID mapping server like Microsoft AD or LDAP).

The proposed presentation discusses in detail the various issues and nuances associated with having common ID management across Swift object access and file access and presents an approach to solve them without changes in core Swift code by leveraging powerful SWIFT middleware framework.

Learning Objectives

• Understanding OpenStack SWIFT Object Store and File access challenges
• Learning about OpenStack middleware which will help developers on its usage
• Learning ID management concepts for File and Object
• Learning the different approaches to solve SWIFT on File ID management problem
• Learning the algorithm that developers can deploy to have common ID management between object (SWIFT) and FILE

Multi-Vendor Key Management with KMIP

Tim Hudson, CTO and Technical Director, Cryptsoft

Abstract

Practical experience from implementing the OASIS Key Management Interoperability Protocol (KMIP) and from deploying and interoperability testing multiple vendor implementations of KMIP form the bulk of the material covered. Guidance will be provided that covers the key issues to require that your vendors address and how to distinguish between simple vendor tick-box approaches to standard conformance and actual interoperable solutions.

Learning Objectives

• In-depth knowledge of the core of the OASIS KMIP
• Awareness of requirements for practical interoperability
• Guidance on important of conformance testing

Network Bound Encryption for Data-at-Rest Protection

Nathaniel McCallum, Senior Software Engineer, Red Hat

Abstract

Setting up a system to store sensitive data is the easy part. Protecting that data from prying eyes is much harder. Warranty repair? Retiring old disks? Sure, you can store your data on encrypted disks. But now you get to manage all the disk encryption keys, creating a high-risk target for active attackers.

In this talk we will introduce Petera, an open source project which implements a new technique for binding encryption keys to a network. This technique provides secure decentralized storage and management of decryption keys so that disk encryption can become entirely transparent and automatic.

Learning Objectives

• Outline a disk's full life-cycle
• Identify data vulnerability points
• Demonstrate how to use encryption to protect data-at-rest
• Survey encryption key management
• Use asymmetric cryptography to reduce management complexities Objective5

SMB 3.1.1 Update

Greg Kramer, Senior Software Engineer, Microsoft
Dan Lovinger, Principal Software Engineer, Microsoft

Abstract

The SMB3 ecosystem continues to grow with the introduction of new clients and server products, a growing deployment base, and new generations of networking technologies. This talk covers the changes to the SMB3 protocol in Windows 10 and Windows Server 2016, the design considerations, and how the changes will affect both protocol implementers and customers. The challenges and performance of 100Gb Ethernet and RDMA solutions on the next-generation Storage Spaces Direct (S2D) will be presented for the first time.

Samba and SMB3: Are We There Yet?

Ira Cooper, Principal Software Engineer, Red Hat

Abstract

Like passengers on a long car ride, the one question on everyone's mind regarding Samba and SMB3 is, "Are we there yet?"

This talk will take you on a tour of how Samba will go from its current nominal support of SMB3 to more comprehensive support of SMB3. You will be given an overview of Samba's architecture, design, and the implementation status of key SMB3 features including Witness, Multichannel, SMB Direct, and Persistent Handles.

By the end, you will know exactly where we are and how far we have to go.

Learning Objectives

• Current status of SMB3 support in Samba
• Architecture and design of SMB3 features in Samba
• Challenges faced during the implementation of SMB3 in Samba so far
• The roadmap for SMB3 support in Samba going forward

Tuning an SMB Server Implementation

Mark Rabinovich, Rand D Manager, Visuality Systems

Abstract

Server platforms differ from a low-end NAS solution to a high-end storage. All of them require an SMB server solution, although. Such a solution, to serve all of them, must be highly customizable. We will discuss the methods of SMB Server parameterization to meet the wide range of requirements. This discussion will emphasize on both instrumentation and performance figures. A special topic will be dedicated to measuring SMB performance over RDMA.

Learning Objectives

• SMB3 server implementation
• SMB scalability
• SMB performance
• SMB over RDMA

Azure File Service: ‘Net Use’ the Cloud

David Goebel, Software Engineer, Microsoft

Abstract

Microsoft Azure has provided REST endpoints for blobs, tables, and queues since its inception. This is an efficient and simple stateless storage API for new applications. However, there is a very large installed base of mature applications, especially enterprise and vertical, which are written to a conventional file API such as Win32 or the C run-times. Azure File Service provides [MS-SMB2] compliant file shares with the same high availability as Azure’s REST endpoints since the backing store for both transient handle state and files data is, under the hood, Azure tables and blobs. As a bonus, the file share namespace is also exposed via REST, allowing simultaneous and coherent access to file data from both endpoints. This talk will relate the experience and challenges of designing and implementing a wire compliant continuously available SMB server where the backing store is not even a conventional file system, let alone NTFS.

Learning Objectives

• Learn how an SMB sever can be built on top of something other than a conventional file system.
• Gain an appreciation of the complexities involved in durably committing what is usually considered volatile handle state which must be both highly available and high performance.
• Be inspired by the possibilities of immediately running existing applications unmodified against the cloud while simultaneously leveraging REST access to the application’s data.

SMB3 Multi-Channel in Samba

Michael Adam, Principal Software Engineer, Red Hat

Abstract

The implementation of advanced SMB3 features is a broad and important set of topics on the Samba roadmap. One of these SMB3 features that is currently being actively worked on is Multi-Channel, a kind of channel bonding at the SMB level intended to increase both performance and fault-tolerance of SMB sessions. It is not only one of the most generally useful features of SMB3 but also a prerequisite for enabling RDMA as a transport for SMB with SMB Direct.

This talk will provide details about the current project to finish the implementation of SMB3 Multi-Channel in Samba, explaining the challenges for development and how they are solved. The presentation will include demos. The talk will conclude with a brief outlook how SMB Direct support can be added to Samba.

Learning Objectives

• Refresher on Multi-Channel
• State of implementation of Multi-Channel in Samba
• Challenges for Samba to implement Multi-Channel
• Design of Multi-Channel in Samba
• Outlook to SMB Direct support

The Past, Present and Future of Samba Messaging

Volker Lendecke, Developer, Samba Team / SerNet

Abstract

Pending

Calling the Witness: SMB3 Failover with Samba/CTDB

Günther Deschner, Developer, RedHat / Samba Team
Jose Rivera, Software Engineer, Red Hat / Samba Team

Abstract

Pending

SMB 3.0 Transparent Failover for EMC Isilon OneFS

Adam Ruprecht, Principal Software Engineer, Isilon Storage Division, EMC

Abstract

EMC Isilon OneFS operating system powers a file system that scales to more than twenty petabytes of data in a single namespace. Transparent failover capabilities of SMB 3.0 are very attractive to provide continuous. non-disruptive availability of this data to the users. However as one can imagine, there are many challenges to build this capability into the scale out architecture of this magnitude. We want to share the approach we took, and challenges we overcame in the process.

Learning Objectives

• Fundamentals of SMB 3.0 failover
• Configuration options to fit workloads running non-server application data
• Isilon implementation of SMB 3.0 failover, and challenges overcome"

SMR – The Next Generation of Storage Technology

Mike Alexenko, Senior Director Market Development Cloud/Cold Storage, HGST

Abstract

Shingled Magnetic Recording (SMR) is the next generation storage technology for continued improvement in HDD areal density, and offers new opportunities for open compute environments. In massive, scale-out cold storage applications such as active archive, social media and long-term data storage, SMR HDD-based solutions offers the highest density, lowest TCO and leading $/TB.

This speaking session will clearly articulate the difference in SMR drive architectures and performance characteristics, and will illustrate how the open source community has the distinct advantage of integrating a host-managed platform that leverages SMR HDDs. Further, HGST will discuss how SMR presents the possibility for unprecedented storage capacities, maintains a familiar form factor, and creates a lower-power envelope so architects can create responsive cold storage data pools that can be accessed in near real-time.

Learning Objectives

• Demonstrate how leveraging a SMR HDD provides advantages to a host-managed platform
• Show how SMR is enabling cold storage data pools on disk, by vastly increasing the amount of archive data that can be actively accessed
• Provide an insight into the future of SMR – what does this mean for next steps of the data center

Host Managed SMR

Albert Chen Engineering Program Director, WDC
Jim Malina, Technologist, WDC

Abstract

Any problem in computer science can be solved with another layer of indirection. Shingle magnetic recording is no different – the only “difficulty” is to determine where to add the additional layer of indirection/abstraction to enable maximum flexibility and efficiency. Let's go over the various SW/FW paradigms that attempt to abstract away SMR behavior (e.g. user space library, device mapper, SMR aware file system, enlightened application). Along the way, we will also explore what deficiencies are holding back SMR adoption in (e.g. ATA sense data reporting) the data center.

Learning Objectives

• Host managed SMR support in Linux
• Linux host managed SMR simulator
• Linux host managed SMR device mapper
• What deficiencies are holding back SMR adoption in (e.g. ATA sense data reporting) the data center
• Various SW/FW paradigms that attempts to abstract away SMR behavior

SNIA Tutorial:
FS Design Around SMR: Seagate’s Journey and Reference System with EXT4

Adrian Palmer, Drive Development Engineering, Seagate Technologies

Abstract

SMR is a game changer drive technology, embraced by all major manufacturers. SMR changes fundamental assumptions of file system management. This long-help abandonment of Random-Writes now makes drives behave as sequential-access tape.

Seagate is leading the way in providing a standards compliant IO stack for use with the new drives. Using the new ZAC/ZBC commands to make and maintain a file system is essential for performant operation. Seagate is sharing lessons learned from modifying EXT4 for use with SMR. This effort is called the SMR Friendly File System (SMRFFS).

Learning Objectives

• Forward-write only considerations for the block allocation scheme
• Zones/BlockGroup/AllocationGroup alignment and use Superblock, and other required write-in-place management schemes

An SMR-Aware Append-Only File System

Stephen Morgan, Senior Staff Research Engineer, Huawei
Chi-Young Ku, Huawei

Abstract

The advent of shingled magnetic recording (SMR) is bringing significant changes to modern file system design. Update-in-place data structures are no longer practicable; log structuring is becoming the de facto game. We report on simulations of a new SMR aware file system for append-only or circular write-only environments that merges log-structured design with traditional journaling to the advantage of both techniques. For example, sequential read performance should be better with SAFS than with a pure LFS because with SAFS, compaction moves blocks of data to contiguous zones. And, like a pure LFS, write performance should be high because writes are converted to appends to a single zone at a time. In this talk, we discuss the effects that SMR is having on basic file system design, how we arrived at our hybrid design, simulations of the design, and results we’ve obtained to date, especially a comparison of the performance of a simulation of SAFS, a traditional journaling file system, and an LFS, all under Linux.

Learning Objectives

• SMR Disk Technology Overview and its Effect on File Systems
• SMR Disk Technology and Log-structuring File Systems
• SMR Disk Technology and Append-only File Systems

Strategies for Using Standard File Systems on SMR Drives

Hannes Reinecke, Senior Engineer, SUSE

Abstract

SMR (shingled Media Recording) drives are posed to become the de facto standard for high-density disk drives. The technology behind these drives sets new challenges to existing storage stacks by introducing new concepts like strict sequential write ordering, zone management etc.

While the ultimate goal for SMR drives is to use a file system natively, currently none of the standard file systems can run without modifications.

In this presentations I'll be outlining a zone-based caching strategy and a remapping strategy for SMR drives. I will be presenting the advantages and disadvantages for each of these, and will be presenting a sample implementation under Linux.

Additionally I'll be presenting the results for running unmodified btrfs, xfs, and ext4 file systems using both of these strategies.

Learning Objectives

• Problems for file systems on SMR drives
• Strategies for using file systems on SMR drives
• Advantages and disadvantages for each of these strategies

Implement Object Storage with SMR based Key-Value Store

Qingchao Luo, Massive Storage Chief Architect, Huawei

Abstract

Object storage technology is suitable for cloud storage and cold storage market, because of its simplicity and scalability. Most of the workload on object storage is write once with few modification, then read a lot. So HDD SMR technology can be leveraged in object storage, since it provide cost efficient media but need sequential write without write in place requirement. This presentation introduces how to design log structured Key value Store based on SMR HDD, then build competitive Object storage.

Integrating Cooperative Flash Management with SMR Technology for Optimized Tiering in Hybrid Systems

Alan Chen, Principal Software Architect, Radian Memory Systems

Abstract

Integrating Cooperative Flash Management (CFM) with SMR drives can achieve unprecedented efficiencies for data tiering in hybrid systems. As an alternative to Flash-Translation-Layers (FTLs) found in conventional SSDs, Cooperative Flash Management (CFM) can provide dramatic improvements in latency, IOPS, bandwidth, and endurance, including an order of magnitude advantage in Quality-of-Service, the most critical metric for Flash storage applications. CFM enables optimizing garbage collection and segment cleaning policies at the system level, opening up a new design space for data center applications. Because CFM and SMR host-managed drives are based upon a similar premise, concepts from the ZBC standard can map to either technology, and be utilized to integrate the two technologies to extend this new system design space into highly optimized data tiering.

Learning Objectives

• Understanding Cooperative Flash Management: what is it, advantages and limitations
• Why Flash FTL strategies don't readily apply to SMR drives
• Mapping ZBC concepts to Flash
• Leveraging host-managed SMR efforts into Cooperative Flash Management
• System design for integrating SMR and Cooperative Flash Management

Standardizing Storage Intelligence and the Performance and Endurance Enhancements it Provides

Bill Martin, Principal Engineer Storage Standards, Samsung
Changho Choi, Principal Engineer, Samsung Semiconductor

Abstract

Storage Intelligence allows Solid State Storage to work together with applications to provide enhanced performance and endurance of storage devices. Today standardization of initial features is nearly complete in the SCSI standard and is moving forward in the NVMe standard with SATA standardization close behind that. This presentation will describe the details that Storage Intelligence is standardizing today and bringing to the standardization process in the near future. Current work involves intelligent placement of data on the storage device, intelligent management of garbage collection, and management of the over provisioning space on the storage device. Future work will add In Storage Compute in the SNIA Object Drive TWG. Each of these four features will be described in detail.

Learning Objectives

• Define the features of Storage Intelligence
• Define the state of standards development in SCSI, NVMe, and SATA
• Why are additional controls over storage necessary

SCSI Protocol Testing

Ronnie Sahlberg, iSCSI hacker, Twitter

Abstract

Automated SCSI/iSCSI testing and protocol validation. This is a presentation aimed at iSCSI and SCSI implementors. libiscsi is a multi-platform iSCSI initiator. It comes with the most comprehensive test suite in the industry for validating the SCSI protocol. 2 Years ago I gave a presentation on iSCSI testing for protocol compliance. A lot has happened since and a lot of things are planned for the future. This will be a presentation of the current state of the libiscsi test suite for SCSI and iSCSI protocol. Why you as an implementor should use it and how it will make your SCSI target better.

Learning Objectives

• Testing
• SCSI testing
• iSCSI testing
• OpenSource resources for SCSI/iSCSI

Object and Open Source Storage Testing: Finally, a viable approach

Tim Van Ash, VP of Product Management, Load DynamiX

Abstract

Interest in object-based and software defined storage, such as CEPH, OpenStack Swift, SNIA CDMI and Amazon S3, is expanding rapidly. Is it still just for non-mission critical or archiving applications or can it really be used for more performance-sensitive production application workloads? If so, how can one prove that these newer storage approaches can handle such workloads? Where are the performance limits? What are the testing parameters that the industry should be most concerned about? This session will discuss such topics and propose a new approach to testing the performance of object-based and open source storage.

Learning Objectives

• Are object-based and software defined storage, such as CEPH, OpenStack Swift, SNIA CDMI and Amazon S3t still just for non-mission critical or archiving applications
• Understand their performance limits
• Learn testing parameters that the industry should be most concerned about

Parallelizing a Distributed Testing Environment

Teague Algie, Software Developer, Cleversafe

Abstract

Insuring software correctness is important in all development environments, but it is critical when developing systems that store mission-critical data. A common bottleneck in the development cycle is the turn-around time for automated regression tests. Yet as products mature, lines of code increase, and features are added, the complexity and number of tests required tends to grow dramatically.

This hampers quick detection and correction of errors, leading to development delays and missed deadlines. To address this problem, we embarked on a path to optimize and parallelize our automated testing framework; in this presentation we detail the result of our effort and the gains we achieved in streamlining our development process.

Learning Objectives

• How to scale a test framework as the product increases in complexity
• Identifying where time is spent in your test framework
• Making the build system resilient to isolated failures
• The utility of virtual machines in accelerating testing

Avoiding Common Storage Development Pitfalls: A Review of the Approaches and Lessons Learned with the VMware vSphere Platform

Scott Davis, Chief Technology Officer, Infinio Systems

Abstract

Developing system-software technology for the VMware vSphere platform can be challenging, as there are a lot of constraints for development partners. Yet, we all know early-platform architecture design decisions can have far-reaching impact on product development options and future capabilities. When building an integrated storage product for VMware, one of the most important decisions is which architectural approach to take to interface with vSphere: virtual appliances, using the Pluggable Storage Architecture (PSA), kernel mode drivers/extensions or soon-to-be VMware's API for IO Filtering (VAIO). In this session, Scott Davis, CTO of Infinio, will weigh each approach’s benefits and challenges for the audience, as well as the trade offs associated with across virtual appliance, kernel-mode and hybrid architectures. Davis will also share lessons learned on the differences across developing for NFS, VMFS and VSAN data store types, as well as the pitfalls and best practices for implementing VAAI support.

Learning Objectives

• Understand several approaches to VMware storage development: virtual appliance, PSA, kernel mode and VAIO
• Learn how to evaluate the benefits and challenges of different architectures, specifically a virtual appliance, kernel-mode operation and a hybrid model
• Find out how developing on NFS, VMFS and VSAN data store types are different challenges
• Get insight on the best practices for VAAI implementation

Seamless Live Virtual Machine Migration by Mitigating Shared Storage Resource Constraint

Sangeeth Keeriyadath, Senior Staff Software Engineer, IBM
Prasanth Jose, Senior Staff Software Engineer, IBM

Abstract

Virtual Machine(VM) Migration is a widely acknowledged feature of most top-selling virtualization solutions; helping businesses tackle the hardware maintenance and server consolidation challenges without the need to affect solution availability. To reap the advantages of this flexibility, businesses have to plan their server networking and storage infrastructure, which includes cabling layout, well in advance. Providing connectivity and sharing of same storage resources across servers is a daunting task and often proves to be a bottleneck for the ability to migrate a VM, to a not planned destination server. The objective of this paper is to provide efficient workable solution for migrating VMs across different servers; in different kinds of data center layout, where storage may not be shared and/or have heterogeneous( i.e. FC / iSCSI / SAS / FCoE) connectivity. We are showcasing enhanced VM migration solution.

We were able to successfully extend existing VM Migration solution to achieve VM migration across servers with multiple kinds of storage connectivity and/or servers without common shared storage.

This implementation provides flexibility to businesses in migrating a VM anywhere in their data center irrespective of storage connectivity type and/or shared storage. This will provide big savings for businesses and enhances their flexibility for better management of data center and cloud based services.

Learning Objectives

• Live Machine Migration without shared storage

I/O Virtualization in Enterprise SSDs

Zhimin Ding, Principle Engineer Design, Toshiba

Abstract

As PCIe-based SSDs become more and more powerful, they are increasingly being used in a virtualized server environment. IO virtualization is an efficient method for VMs to share the resources of the SSD as it allows VMs to directly communicate with the SSD virtual functions instead of going through the hyper-visor layer, thus improving throughput and reducing latency.

In this talk, the author will present methods to manage resource sharing within an IO virtualization-enabled PCIe SSD, with an NVMe front end. The goal is to achieve maximum utilization of SSD internal resources with minimum overhead, while providing customers the flexibility to configure the number of virtual functions and the capabilities associated with each virtual function. More specifically, the presenter will discuss unified architecture that allows both structurally separable and structurally non-separable resources to be shared by virtual functions within an IOV-enabled PCIe SSD.

Learning Objectives

• IO Virtualization is important for enterprise SSD
• Managing shared resource in SSD controller contributes to better IO virtualization performance
• There is an unified architecture to manage different types of shared resources in IO virtualized SSD