Break Out Sessions and Agenda Tracks Include:

Note: This agenda is a work in progress. Check back for updates on additional sessions as well as the agenda schedule.

Moving Data Protection to the Cloud: Trends, Challenges and Strategies

Abstract

There are various new ways and advantages to perform data protection using the Cloud. However, Developers need to carefully study all the alternative approaches and the experiences of others (good and bad) to avoid the pitfalls, especially during the transition from strictly local resources to cloud resources. At this BoF we will discuss:

Learning Objectives

• Critical cloud data protection challenges
• How to use the cloud for data protection
• Pros and cons of various cloud data protection strategies
• Experiences of others (good and bad) to avoid common pitfalls
• Cloud standards in use – and why you need them

Getting Started with the CDMI Conformance Test Program

Abstract

The SNIA Cloud Data Management Interface (CDMI) is an ISO/IEC standard that enables cloud solution vendors to meet the growing need of interoperability for data stored in the cloud and provides end users with the ability to control the destiny of their data, ensuring hassle-free data access, data protection and data migration from one cloud service to another. The CDMI Conformance Test Program (CTP) is now available. Administered by Tata Consulting Services, the CDMI CTP validates a cloud product’s conformance to the CDMI standard. Come to this Birds of a Feather session to learn what the CTP program entails, details on the testing service that is offered, how to get the CTP process started, and pricing. Please note: The availability of CDMI conformance testing at the Cloud Plugfest happening at SDC.

Enabling Persistent Memory Applications with NVDIMMs

Abstract

Non-Volatile DIMMs, or NVDIMMs, have emerged as a go-to technology for boosting performance for next generation storage platforms. The standardization efforts around NVDIMMs have paved the way to simple, plug-n-play adoption. Join the SNIA SSSI NVDIMM Special Interest Group for an interactive discussion on "What's Next?" - what customers, storage developers, and the industry would like to see to improve and enhance NVDIMM integration and optimization.

Kinetic Open Storage – Scalable Objects

Philip Kufeldt, Toshiba
Erik Riedel, EMC

Abstract

Open Kinetic is an open source Collaborative Project formed under the auspices of the Linux Foundation dedicated to creating an open standard around Ethernet-enabled, key/value Kinetic devices.

Storage Architectures for IoT

Mark Carlson, Principal Engineer, Industry Standards, Toshiba

Abstract

The Internet of Things (IoT) is expected to produce massive amounts of data streaming in from sensors. This data needs to be stored and analyzed, sometimes in real time. What are the best storage architectures for this use case? Is Hyper-Converged an answer? What about In-Storage Compute? Come to this BoF to learn what ideas are out there and contribute your own.

Storage for Containers

Abhishek Kane, Senior Software Engineer, Symantec Corporation

Abstract

This talk explores the essential requirements to host mission critical database applications in containers. Specifically, it focuses on the following capabilities that the container ecosystem needs to provide to host such databases:

1. Scalable and persistent data volumes
2. High Availability for containers
3. Migration of containers across hosts
4. Disaster Recovery capability

This talk will demonstrate a case study on how the above goals can be met for Docker ecosystem. Docker ecosystem is not completely evolved to meet the needs of mission critical databases to be run in Docker containers. As a result, there is hesitation in moving enterprise class and mission critical databases from physical/virtual machine platforms to containers. Elaborating on each of the above objectives, the talk intends to inspire confidence to deploy mission critical databases in Docker containers.

Container Data Management Using Flocker

Madhuri Yechuri, Software Engineer, ClusterHQ

Abstract

Containerized micro services call for state be sticky when the application relocates across compute nodes. State associated with a container might involve secret keys used for trusted communication, log files that user wants to data mine at a later point, or data files that are part of application logic. System administrator who sets application relocate policy has no insight into possible state associated within an application running inside a container.

Flocker solves the problem of container data loss when an application relocates, voluntarily or involuntarily, across compute nodes as part of initial placement, maintenance mode, or load balancing workflows performed by orchestrators like Docker Swarm, Mesosphere Marathon, or Kubernetes.

This session includes an overview of Flocker, and a demonstration of Flocker in action using Docker Swarm as the orchestrator of choice.

SNIA and OpenStack: Standards and Open Source

Mark Carlson, SNIA Technical Council Vice Chair

Abstract

The SNIA has created a number of educational materials for OpenStack Storage, which have become some of the most popular content produced. The SNIA has now created a Task Force to investigate a new focused set of activities around OpenStack which may result in a new group targeted at the adoption of storage industry standards in this open source project. Come join the members of this new task force in discussing the requirements and needs in this space.

NAS Benchmarking and SFS 2014 Forum

Nick Principe, EMC Corporation & SPEC

Abstract

Join several SPEC SFS subcommittee members for discussions about SFS development work and an open Q&A session - bring your questions and feedback! We would also like to follow onto the very successful NAS Benchmarking session with an open Q&A with some of the presenters of that tutorial.

Active Directory Integration War Stories

Oliver Jones, Senior Software Engineer, EMC Isilon
Steven Danneman, Software Engineering Manager, EMC Isilon

Abstract

Every storage server eventually has to provide Active Directory authentication. Implementing an AD client can be as complicated as implementing an SMB or NFS server, with lots of gotchas along the way. Isilon will share our stories about the evolution (through many bugs) of our interoperability with Active Directory and we’d love to hear yours. Come for a guided, collaborative, session.

Using SPEC SFS with the SNIA Emerald Program for EPA Energy Star Data Center Storage Program

Wayne Adams, Chair, SNIA Green Storage Initiative
Carlos Pratt, Chair, SNIA Green Storage TWG

Abstract

The next storage platform category to be added into the EPA Data Center Storage program is NAS. Come learn what it takes in setting up a SNIA Emerald NAS testing environment with the SPEC SFS tools , the additional energy related instrumentation and data collection tools. Become involved in SNIA technical work to validate the test methodologies in prep for 2016. Don’t wait to be kicked in the “NAS” when an Energy Star rating gates selling your NAS solutions.

NVMe Over Fabrics

Panel: Dave Minturn, Intel; Idan Burstein, Mellanox; Christoph Helllwig, Qingbo Wang, HGST
Moderator: Zvonimir Bandic, HGST

Abstract

This one hour session with panel of experts from the industry will focus on explaining a need for new storage networking protocols for both NAND and emerging Non-volatile memory devices. The pressure to reduce network latency to scale comparable with new solid state devices requires rethinking and reengineering of storage networking protocols. We will discuss the benefits of NVMe over fabrics protocol that utilizes RDMA networking, and present recent measured prototyping data. Panel of experts will be available to answer questions from attendees.

Using CDMI to Manage Swift, S3, and Ceph Object Repositories

David Slik, Technical Director, NetApp, Inc

Abstract

The Cloud Data Management Interface is designed to provide namespace-based management functionality for the superset of object, file and block protocols. This makes it ideally suited for use with common protocols such as NFS, CIFS, iSCSI, Swift and S3. This session provides an overview of how CDMI interoperates with these protocols, and how the use of CDMI as a management protocol adds value to multi-protocol systems. Concrete examples and use cases from end-users and vendors will be highlighted.

Learning Objectives

• Learn how to use CDMI to manage object repositories
• Learn how to use CDMI to manage file systems
• Learn how to use CDMI to manage block storage systems
• Learn how CDMI works with multi-protocol systems

Unistore: A Unified Storage Architecture for Cloud Computing

Yong Chen, Assistant Professor, Texas Tech University

Abstract

Emerging large-scale applications on Cloud computing platform, such as information retrieval, data mining, online business, and social network, are data- rather than computation-intensive. Storage system is one of the most critical components for Cloud computing. The traditional hard disk drives (HDD) are current dominant storage devices in Clouds, but are notorious for long access latency and failure prone. The recently emerged storage class memory (SCM) such as Solid State Drives provides a new promising storage solution of high bandwidth, low latency, and mechanical component free, but with inherent limitations of small capacity, short lifetime, and high cost. This talk will introduce an ongoing effort from Texas Tech University and Nimboxx Inc. of building an innovative unified storage architecture (Unistore) with the co-existence and efficient integration of heterogeneous HDD and SCM devices for Cloud storage systems. We will introduce the Unistore design principle and rationale. We will also discuss the prototyping implementation with newly designed data distribution and placement algorithm. This talk is intended for SNIA/SDC general attendees.

The Developer's Dilemma: Do-It-Yourself Storage or Surrender Your Data?

Luke Behnke, VP of Product, Bitcasa

Abstract

Creating an app isn’t simple. Early in the process of designing the app, decisions have to be made around how app data will be stored, and for most developers the cloud is an obvious choice. At this point, developers need to make an important choice: invest time, energy and resources in creating their own DIY file systems that sits on top of public cloud infrastructure; or take the shortcut and use a cloud storage API, and surrender their users’ data to popular cloud storage services. In this session, Bitcasa CEO, Brian Taptich will outline the impact of this dilemma on the future functionality and user experience of an app, and also discuss why the next generation of apps will require better file systems that offer broad capabilities, performance, security and scalability, and most importantly, developer control of user data and experience.

Learning Objectives

• To explain the benefits of utilizing the cloud for developers
• To dissect the various options developers have when utilizing cloud storage
• Why choosing the right platform is imperative for user experience and privacy
• Why owning user data is important if developers want to own the customer

How to Test CDMI Extension Feature Like LTFS, Data Deduplication, and OVF, Partial – Value Copy Functionality: Challenges, Solutions and Best Practice?

Sachin Goswami, Solution Architect and Storage COE Head Hi Tech, TCS

Abstract

The Cloud Storage space has been outperforming the industry expectations as is evident in several Industry report. SNIA provided Cloud Data Management Interface (CDMI) specification is increasingly being adopted as a standard across the cloud.

The popularity of the CDMI specification can be judged by the present cloud storage market being flooded with CDMI server based products offered by many big and small cloud storage vendors. The SNIA Cloud Storage Technical Workgroup has been unceasingly working to address all storage challenges that exist in the storage domain. It is striving to provide support and solution for Data Deduplication, Open virtualization format (OVF), Partial Upload, server side partial value copy and LTFS as a primary cloud storage, managing Latency as well as backup and archival solution. TCS is focusing on maturing the Conformance Test Suite by adding more enhancements. In this proposal we will share the approach TCS will adopt to overcome the challenges in testing of LTFS integration with CDMI, Data Deduplication, partial upload on Server and Open Vitalization format (OVF) of CDMI and Non-CDMI based scenarios of the cloud products. Additionally, we will also be sharing challenges faced / learnings gathered from testing of CDMI Products for conformance. These learnings will help serve as a ready reference for organizations developing LTFS, Data Deduplication, OVF and partial upload in CDMI, Non-CDMI based product suite.

Learning Objectives

• Understanding to develop test specification on LTFS Export and Test.
• Understanding how to develop Test Specification on Server Side Partial-Value Copy Specification and Test.
• Understanding to develop test specification on OVF and Partial Upload and Test.

Big Data Analytics on Object Stoage - Hadoop Over Ceph Object Storage with SSD Cache

Yuan Zhou, Software Engineer, Intel Asia R&D

Abstract

Cloud object store provides the ability to store objects across multiple datacenters over a straightforward HTTPS REST API. The namespace is hierarchical and can be searched. Objects can be arbitraiy large and numerous. The deployment can also be done on a commodity-harware based. This makes them an attractive option for archiving large amounts of data that are produced in science and industry. To analyze the data, advanced analytics such as MapReduce can be used. However, copying the data from the object store into distributed file system that the analytics system requires directly on object stores greatly improves usability and performance. In this work, we study the possibility of running Hadoop over Ceph Object Storage and identify common problems.

GlusterFS - The Thrilla in Manila

Ramnath Sai Sagar, Cloud Technical Marketing Manager, Mellanox
Veda Shankar, Technical Marketing Manager, RedHat

Abstract

With an estimated 15 billion devices connected to internet in 2015 generating exabytes of data, there is a huge influx in the amount of data generated, that puts a severe stress on the underlying storage. This explosion in data growth along with corresponding expenditure on infrastructure has catalyzed the need for a fundamental shift in the way we look at storage. The solution to this problem is Software Defined Storage (SDS), which is also the natural fit for the cloud model. OpenStack is fairly mature for block storage (Cinder) and object storage (Swift), but many current business applications require file storage. Enter Manila - the new service that provides an automated, on-demand and scalable service for delivering shared and distributed file systems all using a standardized API native to OpenStack. From the set of available Manila drivers, we’ll focus today on GlusterFS, the founding SDS in the Manila project and an open, and proven distributed storage system. In this presentation, we’ll review Manila backed by the GlusterFS scale-out storage system and explore the value of RDMA in SDS system like GlusterFS in certain use cases.

What You Need to Know on Cloud Storage

David Slik, Technical Director, NetApp
Mark Carlson, Principal Engineer, Industry Standards, Toshiba

Abstract

This session assumes no prior knowledge on cloud storage and is intended to bring a storage developer up to speed on the concepts, conventions and standards in this space. The session will include a live demo of a storage cloud operating to reinforce the concepts presented.

Using REST API for Management Integration

Brian Mason, MTS-SW, NetApp

Abstract

Integration is key to managing storage systems today. Customers do not want a vendor lock-in or vendor specific management tools. They want to use their best in class management tools and have various storage systems integrate into their management tools. A REST API to your storage system is an absolute must in today's market. REST is the common denominator for management integration. Fortunately it is rather simple to create a REST API. It is a little harder to get one just right and to get the documentation done in a usable form.

Learning Objectives

• What is a REST API? How are they different from previous API protocols? Why are they so useful?
• Technology Primer for REST
• How to build a REST API
• Documentation Standards
• Using a REST API as a client

SNIA Tutorial:
Windows Interoperability Workshop

Christopher Hertel, SW Senior Program Engineer, Samba Team / Dell Compellent

Abstract

Windows and POSIX are different, and bridging the gap between the two—particularly with Network File Systems—can be a confusing and daunting endeavor ...and annoying, too.

This tutorial will provide an overview of the SMB3 network file protocol (the heart and soul of Windows Interoperability) and describe some of the unique and powerful features that SMB3 provides. We will also point out and discuss some of the other protocols and services that are integrated with SMB3 (such as PeerDist), and show how the different pieces are stapled together and made to fly. The tutorial will also cover the general structure of Microsoft's protocol documentation, the best available cartography for those lost in the Interoperability Jungle. Some simple code examples will be used sparingly as examples, wherever it may seem clever and useful to do so.

Learning Objectives

• Become familiar with the Windows Interoperability Ecosystem
• Better understand Microsoft's Specifications
• Identify Windows-specific semantic details

Next Generation Data Centers: Hyperconverged Architectures Impact On Storage

Mark OConnell, Distinguished Engineer, EMC

Abstract

A modern data center typically contains a number of specialized storage systems which provide centralized storage for a large collection of data center applications. These specialized systems were designed and implemented as a solution to the problems of scalable storage, 24x7 data access, centralized data protection, centralized disaster protection strategies, and more. While these issues remain in the data center environment, new applications, new workload profiles, and the changing economics of computing have introduced new demands on the storage system which drive towards new architectures, and ultimately towards a hyperconverged architecture. After reviewing what a hyperconverged architecture is and the building blocks in use in such architectures, there will be some predictions for the future of such architectures.

Learning Objectives

• What is a hyperconverged architecture
• How hyperconverged architectures differ from traditional architectures
• What technologies are being used to build hyperconverged architectures
• What workloads are appropriate for hyperconverged architectures

PCI Express: Driving the Future of Storage

Ramin Neshati, PCI-SIG Board Member and Marketing Chair, PCI-SIG

Abstract

The data explosion has led to a corresponding explosion in the demand for storage. At the same time, traditional storage interconnects such as SATA are being replaced with PCI Express (PCIe)-attached storage solutions. Leveraging PCIe technology removes the performance bottlenecks and provides long-term bandwidth and performance scalability as PCIe evolves from 8GT/s bit rate to 16GT/s and beyond. PCIe-attached storage delivers a robust solution that is supported natively in all Operating Systems and a wide array of form factors either chip-to-chip or through expansion modules and daughter cards.

Learning Objectives

• Gain insight into PCI Express technology and how it is used in storage solutions
• Learn how PCI Express technology advancements in lowering active and idle power can be used in your storage solution
• Learn how PCIe 3.0 and PCIe 4.0 provides a strategic solution for storage attachment
• Understand the work the PCI-SIG is doing in new form factors for driving broad adoption in storage applications

Next Generation Low Latency Storage Area Networks

Rupin Mohan, Chief Technologist - Storage Networking, HP

Abstract

In this session, we will present a current (FC, FCoE and iSCSI) and future state (iSER, RDMA, NVMe, and more) of the union of the next generation low latency Storage Area Networks (SAN's) and discuss how the future of SAN's protocols will look like for block, file and object storage.

The Pros and Cons of Developing Erasure Coding and Replication Instead of Traditional RAID in Next-Generation Storage Platforms

Abhijith Shenoy, Engineer, Hedvig

Abstract

Scale-out, hyperconverged, hyperscale, software-defined, hybrid arrays – the list of scalable and distributed storage systems is rapidly growing. But all of these innovations require tough choices on how best to protect data. Moreover, the abundance of 4- 8- and even 10-TB drives makes the traditional approach of RAID untenable because repairing drive failures can take days and even weeks depending on the architecture and drive capacity. New approaches that balance performance with availability are needed. Erasure coding and replication are emerging, rapidly maturing techniques that empower developers with new data protection methods.

This session will discuss the pros and cons of erasure coding and replication versus traditional RAID techniques. Specifically, this session will discuss the performance vs. availability tradeoffs with each technique as well as present and in-depth look at using tunable replication as the ideal data protection solution, as proven by large-scale distributed systems.

Learning Objectives

• Attendees will learn why RAID isn’t adequate in next-gen storage architectures.
• Attendees will learn the pros and cons of erasure coding and replication in newer storage architectures.
• Attendees will learn how replicating at a per-volume basis provides the best mix of performance and availability.
• Attendees will learn tips and best practices for incorporating new data protection methods into storage platforms.

The Lightning Memory–Mapped Database

Howard Chu, CTO, Symas

Abstract

The Lightning Memory-Mapped Database (LMDB) was introduced at LDAPCon 2011 and has been enjoying tremendous success in the intervening time. LMDB was written for the OpenLDAP Project and has proved to be the world's smallest, fastest, and most reliable transactional embedded data store. It has cemented OpenLDAP's position as world's fastest directory server, and its adoption outside the OpenLDAP Project continues to grow, with a wide range of applications including big data services, crypto-currencies, machine learning, and many others.

The talk will cover highlights of the LMDB design as well as the impact of LMDB on other projects.

Learning Objectives

• Highlight problems with traditional DB storage designs
• Explain benefits of single-level-store
• Explain corruption-proof design and implementation
• Compare and contrast leading data structures: B+tree, LSM, Fractal Trees

The Bw-Tree Key-Value Store and Its Applications to Server/Cloud Data Management in Production

Sudipta Sengupta, Principal Research Scientist, Microsoft Research

Abstract

The Bw-Tree is an ordered key-value store, built by layering a B-tree form access method over a cache/storage sub-system (LLAMA) that is lock-free and organizes storage in a log-structured manner. It is designed to optimize performance on modern hardware, specifically (i) multi-core processors with multi-level memory/cache hierarchy, and (ii) flash memory based SSDs with fast random reads (but inefficient random write performance). The Bw-Tree is shipping in three of Microsoft’s server/cloud products – as the key sequential index in SQL Server Hekaton (main memory database), as the indexing engine inside Azure DocumentDB (distributed document-oriented store), and as an ordered key-value store in Bing ObjectStore (distributed back-end supporting many properties in Bing).

Learning Objectives

• Bw-Tree data structure
• Lock-free design for high concurrency
• Log-structured storage design for flash based SSDs
• Page-oriented store (LLAMA) for building access methods on top
• Bw-Tree Applications in Production at Microsoft

IMDB NDP Advances

Gil Russell, Principal Analyst, Semiscape

Abstract

In-Memory Database appliances are rapidly evolving, becoming in effect the main operating stored image for both analytic and cognitive computing applications in the next generation of data center and cloud in-rack storage.

Co-opting of DRAM with proximal NAND-Flash mass storage being combined with Near Data Processing re-imagines the entire computing paradigm by effectively turning an entire database image into a content-addressable look alike. Candidates for Storage Class Memory are nearing market introduction and with Near Data Processing abilities will radically change Database Management Systems.

Learning Objectives

• An understanding of performance metrics for IMDB systems
• The importance of silicon photonic interconnects
• The evolution to a low latency high bandwidth DB environment
• Competing elements for market supremacy.
• Cognitive computing - the next step

Taxonomy of Differential Compression

Liwei Ren, Scientific Adviser, Trend Micro

Abstract

Differential compression (aka, delta encoding) is a special category for data de-duplication. It can find many applications in various domains such as data backup, software revision control systems, software incremental update, file synchronization over network, to name just a few. This talk will introduce a taxonomy of how to categorize delta encoding schemes in various applications. Pros and cons of each scheme will be investigated in depth.

Learning Objectives

• Why do we need differential compression?
• A mathematical model for describing the differences between two files
• A taxonomy for categorizing differential compression
• Analysis for practical applications

Design Decisions and Repercussions of Compression and Data Reduction in a Storage Array

Chris Golden, Software Engineer, Pure Storage

Abstract

All flash arrays incorporate a number of data reduction techniques to increase effective capacity and reduce overall storage costs. Compression and deduplication are two commonly employed techniques, each with multiple different strategies for implementation. Because compression and data reduction are only part of a greater data reduction strategy, one must also understand their codependent interactions with the rest of a storage system. This talk presents a structured overview of multiple different compression and deduplication technologies. The basics of each technique are presented alongside their benefits, drawbacks and impact on overall system design. This talk then augments that understanding by applying these various techniques to a sample real-world workload, demonstrating the impact of these decisions in practice.

Learning Objectives

• Gain a deeper understanding of compression, deduplication and storage layout
• Benefits and drawbacks of using data reduction techniques in a flash storage array
• Examination of co-dependent interactions between various data reduction techniques and workload
• Supplementing theory with practice via analysis of a sample workload

DAOS – An Architecture for Extreme Scale Storage

Eric Barton, Lead Architect of the High Performance Data Division, Intel

Abstract

Three emerging trends must be considered when assessing how storage should operate at extreme scale. First, continuing expansion in the volume of data to be stored is accompanied by increasing complexity in the metadata to be stored with it and queries to be executed on it. Second, ever increasing core and node counts require corresponding scaling of application concurrency while simultaneously increasing the frequency of hardware failure. Third, new NVRAM technologies allow storage, accessible at extremely fine grain and low latency, to be distributed across the entire cluster fabric to exploit full cross-sectional bandwidth. This talk describes Distributed Application Object Storage (DAOS) – a new storage architecture that Intel is developing to address the functionality, scalability and resilience issues and exploit the performance opportunities presented by these emerging trends.

Learning Objectives

• Exascale / Big Data
• Scalable Distributed Storage Systems
• Object Storage
• Persistent Memory

New Consistent Hashing Algorithms for Data Storage

Jason Resch, Software Architect, Cleversafe, Inc.

Abstract

Consistent Hashing provides a mechanism through which independent actors in a distributed system can reach an agreement about where a resource is, who is responsible for its access or storage, and even derive deterministically a prioritized list of fall-backs should the primary location be down. Moreover, consistent hashing allows aspects of the system to change dynamically while minimizing disruptions. We've recently developed a new consistent hashing algorithm, which we call the Weighted Rendezvous Hash. Its primary advantage is that it obtains provably minimum disruption during changes to a data storage system. This presentation will introduce this algorithm for the first time, and consider several of its applications.

Learning Objectives

• What is Consistent Hashing
• Traditional applications of consistent hashing
• The implementation of the new algorithm: Weighted Rendezvous Hash
• Why Weighted Rendezvous Hash is more efficient than previous algorithms
• Applications of Weighted Rendezvous Hash in data storage systems

Beyond Consistent Hashing and TCP: Vastly Scalable Load Balanced Storage Clustering

Alex Aizman, CTO and Founder, Nexenta Systems
Caitlin Bestler, Senior Director of Arch, Nexenta Systems

Abstract

Successive generations of storage solutions have increased decentralization. Early NAS systems made all the decisions on a single server, down to sector assignment. Federated NAS enabled dynamic distribution of the namespace across multiple storage serves. The first Object Clusters delegated CRUD-based management of both object metadata and data to OSDs.

Current generation of Object Clusters uses Consistent Hashing to eliminate the need for central metadata. However, Consistent Hashing and its derivatives, combined with the prevalent use of TCP/IP in storage clusters results in performance hot spots and bottlenecks, diminished scale-out capability and dis-balances in resource utilization.

These shortcomings will be demonstrated with a simulation of a large storage cluster. An alternative next generation strategy that simultaneously optimizes available IOPS "budget" of the back-end storage, storage capacity, and network utilization will be explained. Practically unlimited load-balanced scale-out capability using Layer 5 (Replicast) protocol for Multicast Replication within the cluster will be presented.

Learning Objectives

• Why neither Consistent Hashing nor TCP/IP scale
• How CCOW (Cloud Copy-on-Write) and Replicast provide for infinite scale-out
• Impact on IOPS, Storage Capacity and Network utilization
• Simulation: queuing and congestion in a 4000 node cluster Actual
• Results: measured with a more affordable cluster

Real World Use Cases for Tachyon, a Memory-Centric Distributed Storage System

Haoyuan Li, CEO, Tachyon Nexus

Abstract

Memory is the key to fast big data processing. This has been realized by many, and frameworks such as Spark and Shark already leverage memory performance. As data sets continue to grow, storage is increasingly becoming a critical bottleneck in many workloads.

To address this need, we have developed Tachyon, a memory-centric fault-tolerant distributed storage system, which enables reliable file sharing at memory-speed across cluster frameworks such as Apache Spark, MapReduce, and Apache Flink. The result of over three years of research and development, Tachyon achieves both memory-speed and fault tolerance.

Tachyon is Hadoop compatible. Existing Spark, MapReduce, Flink programs can run on top of it without any code changes. Tachyon is the default off-heap option in Spark. The project is open source and is already deployed at many companies in production. In addition, Tachyon has more than 100 contributors from over 30 institutions, including Yahoo, Tachyon Nexus, Redhat, Baidu, Intel, and IBM. The project is the storage layer of the Berkeley Data Analytics Stack (BDAS) and also part of the Fedora distribution.

In this talk, we give an overview of Tachyon, as well as several use cases we have seen in the real world.

Where Moore's Law Meets the Speed of Light: Optimizing Exabyte-Scale Network Protocols

Yogesh Vedpathak, Software Developer, Cleversafe

Abstract

Scalability is critically important to distributed storage systems. Exabyte-scale storage is already on the horizon and such systems involve tens of thousands of nodes. But today’s Internet protocols were never designed to handle such cases. In systems this big it's impossible to maintain connections and sessions with every storage device. However, multiple round trip connection setups, TLS handshakes, and authentication mechanisms, compounded with the unyielding speed of light and geo-dispersed topologies create a perfect storm for high latency and bad performance. In this presentation we explore the available options to achieve security, performance, and low latency in a system where persistent sessions are an unaffordable luxury.

Learning Objectives

• Limitations of today’s Internet protocols in large distributed systems
• How to implement a secure, connectionless, single-round-trip network protocol
• What the network topology will look like in an exabyte-scale globally dispersed storage system

Implications of Emerging Storage Technologies on Massive Scale Simulation Based Visual Effects

Yahya H. Mirza, CEO/CTO, Aclectic Systems Inc

Abstract

As the feature film industry moves towards higher resolution frames sizes from 4K to 8K and beyond, physically based visual effects such as smoke, fire, water, explosions, etc. demand higher resolution simulation grids. A common rule of thumb for final renders is that one voxel is utilized per pixel. Thus a 4K (4096 x 2160 pixels) frame may require simulation on a massive sized grid, resulting in significant compute and I/O costs. The result leads to unacceptable turn-around times and increased production costs. This presentation will overview the tools, production pipeline and production relevant open source and proprietary physical simulation software utilized by major feature film studios to create blockbuster or “tent pole” productions.

Aclectic Systems Inc. (Acletic) is developing ColossusTM, a custom hardware / software integrated solution to dramatically speed up massive scale physically based visual effects. To achieve our performance goals, Aclectic is taking a systems approach which accelerates simulation, volume rendering and I/O. This presentation will overview the tools, production pipeline and production relevant open source software utilized by major feature film studios. Throughout, a discussion will be intertwined about how emerging storage technologies such as FLASH, NVMe and NVMe over Fabric could play a part in a future integrated solution used to lower production costs.

How Did Human Cells Build a Storage Engine?

Sanjay Joshi, CTO Life Sciences, EMC Emerging Technologies Division

Abstract

The eukaryotic cell is a fascinating piece of biological machinery – storage is at its heart, literally, within the nucleus. This presentation will tell a story of the evolution of the storage portion of the human cell and its present capacity and properties that could be "bio-mimicked" for future digital storage systems, especially deep archives.

Learning Objectives

• Biological storage concepts
• Requirements for a storage 'unit'
• Requirements for replication
• Requirements for error correction
• Power management

Apache Ignite - In-Memory Data Fabric

Dmitriy Setrakyan, VP of Engineering, GridGain Systems

Abstract

This presentation will provide a deep dive into new Apache project: Apache Ignite. Apache Ignite is the in-memory data fabric that combines industry first distributed and fault-tolerant in-memory file system, in-memory cluster and computing, in-memory data grid and in-memory streaming under one umbrella of a fabric. In-memory data fabric slides between applications and various data sources and provides ultimate data storage to the applications.

Apache Ignite is the first general purpose in-memory computing platform in Apache Software Foundation family. We believe it will have same effect on Fast Data processing as Hadoop has on Big Data processing. Better understanding of inner details behind Apache Ignite will hopefully encourage more companies and individual committers to join the project.

Learning Objectives

• Learn about industry leading in-memory data fabric

Integrity of In-memory Data Mirroring in Distributed Systems

Tejas Wanjari, Senior Software Engineer, EMC Data Domain

Abstract

Data in memory could be in a modified state than its on-disk copy. Also, unlike the on-disk copy, the in-memory data might not be checksummed, replicated or backed-up, every time it is modified. So the data must be checksummed before mirroring to avoid network corruptions. But checksumming the data in the application has other overheads: It must handle networking functionalities like retransmission, congestion, etc. Secondly, if it delays the validation of mirrored data, it might be difficult to recover the correct state of the system.

Mirrored-data integrity as transport protocol functionality leads to modular design and better performance. We propose a novel approach that utilizes TCP with MD5 signatures to handle the network integrity overhead. Thus, the application can focus on its primary task. We discuss the evaluation and use-case of this approach (NVM mirroring in Data Domain HA) to prove its advantages over conventional approach of checksumming in the application.

Learning Objectives

• Designing efficient data-mirroring in backup and recovery systems, where reliability is prime
• Linux kernel TCP know-how for using it with MD5 option
• Analysis of conventional approach vs. the TCP MD5
• Use-case: TCP MD5 option for NVM mirroring in Data Domain HA

Learnings from Creating Plugin Module for OpenStack Manila Services

Vinod Eswaraprasad, Software Architect, Wipro

Abstract

Manila is the file sharing service for OpenStack . Manila provides the management of file shares (for example, NFS and CIFS) as a core service to OpenStack. Manila services, like all other openstack services follows a pluggable architecture, and it provides a management of a shared file system instances. This paper discusses our work on integrating a multi-protocol NAS storage device to the OpenStack Manila service. We look at the architecture principle behind the scalability and modularity of Manila services, and the analysis of interface extensions required to integrate a typical NAS head. We also take a deeper look at a NAS file share management interfaces required for a software defined storage controller within the OpenStack Manila framework.

Learning Objectives

• OpenStack File sharing service architecture
• The API and integration framework for openstack services
• NAS share management - and integration
• SDS - interfaces required for a NAS device

Leveraging BTRFS, Linux and Open Source in Developing Advanced Storage Solutions

Suman Chakravartula, Maintainer, Rockstor

Abstract

The future of Linux filesystems is here with the emergence of BTRFS. Other advancements in Linux combined with BTRFS provide a robust OS platform for developers. Features from CoW snapshots, robust software raid, data protection to compression, dedup and efficient replication -- just to name a few, are accessible to developers. These Linux OS level advancements combined with proven application level open source tools and libraries give developers a lot of horsepower and raw material to build creative and powerful solutions.

Learning Objectives

• Learn to develop with BTRFS
• Learn about new storage related advancements in Linux
• Learn about challenges developing scalable storage solutions using open source
• Learn about application level opensource tools and libraries that help storage development
• Learn about open source storage ecosystem

Apache HDFS: Latest Developments and Trends

Jakob Homan, Distributed Systems Engineer, Microsoft

Abstract

During the past two years, HDFS has been rapidly developed to meet the needs of enterprise and cloud customers. We'll take a look at the new features, their implementations and how they address previous shortcomings of HDFS.

How to Enable a Reliable and Economic Cloud Storage Solution by Integrating SSD with LTFS – Addressing Challenges and Best Practices

Ankit Agrawal, Solution Developer, TCS
Sachin Goswami, TCS

Abstract

IT industry is constantly evolving by transforming thoughts into cutting edge products and solutions to provide better services to the customers. Linear Tape File System (LTFS) is one such file system that overcomes the drawbacks of the traditional tape storage technology such as sequential navigation. SNIA’s LTFS Technical work group is adapting to emerging market needs and developing/enhancing Liner Tape File System (LTFS) specifications for tape technology.

TCS has also started working on some ideas in LTFS space and in this proposal we will share our views on how to integrate SSD as a cache with LTFS tape system to transparently deliver the best benefits for Object Base storage. This combination will allow us to deliver a reliable and economic storage solution without sacrificing performance. We will also talk about the potential challenges in our approach and best practices that can be adopted to overcome these challenges

Learning Objectives

• Understanding LTFS specification provided by SNIA
• Understanding SSD functionality
• Understanding in integration LTFS with SSD

A Pausable File System

James Cain, Principal Software Architect, Quantel Limited

Abstract

As storage developers we are all obsessed with speed. This talk gives a different take on speed – how slow can we go? Can we even stop? If so for how long? The talk will also analyze why this is interesting, and demonstrate that the file system interface – and the way all software depends upon it – is one of the most powerful abstractions in operating systems.

The presenter will use his own implementation of an SMB3 server (running in user mode on Windows) to demonstrate the effects of marking messages as asynchronously handled and then delaying responses – in order to build up a complete understanding of the semantics offered by a pausable file system.

This exploration of the semantics of slow responses will demonstrate that researching slowing down can bear as much fruit as speeding up!

Learning Objectives

• Understanding The Inversion of Control design pattern and how it can be applied to an implementation of a file server.
• Exploring how the file system interface can be seen as a contract and that the semantics of that contract can be exploited for innovative uses.
• Demonstrating that implementing a fairly small subset of SMB3 as a server is enough to conduct research in file systems.

Storage Solutions for Tomorrow's Physics Projects

Ulrich Fuchs, Service Manager, CERN

Abstract

The unique challenges in the field of nuclear high energy physics are already pushing the limits of storage solutions today, however, the projects planned for the next ten years call for storage capacities, performance and access patterns that exceed the limits of many of today's solutions.

This talk will present the limitations in network and storage and suggest possible architectures for tomorrow's storage implementations in this field and show results of first performance tests done on various solutions (Lustre, NFS, Block Object storage, GPFS ..) for typical application access patterns.

Learning Objectives

• Shared file system and storage performance requirements in science workloads
• Setup and results of performance measurements of different file systems: the LUSTRE FS, NFS, BOS, GPFS
• Technology differences between several file systems and storage solutions

Storage Class Memory Support in the Windows Operating System

Neal Christiansen, Principal Development Lead, Microsoft

Abstract

This will describe the changes being made to the Windows OS, its file systems and storage stack in response to new evolving storage technologies.

Learning Objectives

• How windows is adapting to new storage technologies

ZFS Async Replication Enhancements

Richard Morris, Principal Software Engineer, Oracle
Peter Cudhea, Principal Software Engineer, Oracle

Abstract

This presentation explores some design decisions around enhancing the zfs send and zfs receive commands to transfer already compressed data more efficiently and to recover from failures without re-sending data that has already been received.

Learning Objectives

• High level understanding - how ZFS provides an efficient platform for async replication
• Finding stability in the chaos - tension between what's stable in an archive and what isn't
• Resolving significant constraints - why something simple turned out not to be not so simple

ReFS v2: Cloning, Projecting, and Moving Data

J.R. Tipton, Development Lead, Microsoft

Abstract

File systems are fundamentally about wrapping abstractions around data: files are really just named data blocks. ReFS v2 presents just a couple new abstractions that open up greater control for applications and virtualization.

We'll cover block projection and cloning as well as in-line data tiering. Block projection makes it easy to efficiently build simple concepts like file splitting and copying as well as more complex ones like efficient VM snapshots. Inline data tiering brings efficient data tiering to virtualization and OLTP workloads.

Learning Objectives

• How ReFS v2 exploits its metadata store to project file blocks at a fine granularity
• How metadata is managed in ReFS v2
• How data movement between tiers can happen efficiently while maintaining data integrity

Achieving Coherent and Aggressive Client Caching in Gluster, a Distributed System

Poornima Gurusiddaiah, Software Engineer, Red Hat
Soumya Koduri, Red Hat

Abstract

The presentation will be about how to implement:

• File system notifications
• Leases

In a distributed system and how these can be leveraged to implement a client side coherent and aggressive caching.

Learning Objectives

• Designing File system notification in Distributed System
• Designing leases in Distributed System
• Designing client caching in Distributed file system
• Gluster and the benefits of xlator modeling

Petabyte-scale Distributed File Systems in Open Source Land: KFS Evolution

Sriram Rao, Partner Scientist Manager, Microsoft

Abstract

Over the past decade, distributed file systems based on a scale-out architecture that enables managing massive amounts of storage space (petabytes) have become commonplace. In this talk, I will first provide an overview of OSS systems (such as HDFS and KFS) in this space. I will then describe how these systems have evolved to take advantage of increasing network bandwidth in data center settings to improve application performance as well as storage efficiency. I will talk about these aspects by highlighting two novel features, multi-writer atomic append and (time-permitting) distributed erasure coding. These capabilities have been implemented in KFS and deployed in production settings to run analytic workloads.

High Resiliency Parallel NAS Cluster

Richard Levy, CEO and President, Peer Fusion

Abstract

The PFFS is a POSIX compliant parallel file system capable of high resiliency and scalability. The user data is dispersed across the cluster with no replication thus providing significant savings.The resiliency level is selected by the user. Peer failures do not disrupt applications as the cluster automatically performs on the fly repairs as required for read and write operations to complete successfully (applications can read and write data from failed peers). There are two main protocols for communication between peers: the CLI protocol for namespace type commands (e.g. link, unlink, symlink, mkdir, rmdir, etc.) and the MBP protocol for file I/O. Both protocols are highly efficient and produce very little chatter. They rely on multicast and inference to preserve efficient scalability as the peer count grows large. The software is highly threaded to parallelize network I/O, disk I/O and computation. Gatewa