SNIA Developer Conference September 15-17, 2025 | Santa Clara, CA
The Challenges in Creating a Clustered Software Defined Fileserver from Scratch on HCI
Abstract
This presentation will outline the trials and tribulations encountered over the last 5 years creating the first software defined Fileserver on our hyperconverged platform. We believe that this architecture brings value and goodness as on a private cloud platform but poses unique challenges. Working closely in partnership with developers and performance engineers, we gleaned insights by performance experimentation troubleshooting. These insights were then passed on to architectural changes as we iterated on the design. Its been a fun journey, mostly due to the people and technology we were able to build.
Learning Objectives
- How being virtualized affects fileserver performance and function.
- The challenges to Integrate and layer one platform onto another
- The value of pay as you grow file services
Related Sessions
How to Connect a SAS System
The new Serial Attached SCSI (SAS)-4.1 (INCITS 567) technology is being deployed in the market for use in practically every industry, including hyperscale data centers, banking, education, government, healthcare and manufacturing. It maintains backwards compatibility with previous-generation SAS implementations, which means that older drives will be compatible with newer storage controller and SAS expander products. In this session, you’ll be introduced to the SAS Integrator’s Guide: a quick reference to the menu of standard connectors and cables required to assemble a SAS system, from the SAS drive (a hard drive or a solid-state drive) out to the enclosure level, including the associated box-to-box cabling.
From NASD to DeltaFS: CMU and Los Alamos's Efforts in Building Large-Scale Filesystem Metadata
It has been a tradition that, every once in a while, we stop and reassess whether we need to build our next filesystems differently. A key previous effort was made by the Carnegie Mellon University's NASD project, which decoupled filesystem data communication from metadata management and leveraged object storage devices for scalable data access. Now, as we enter into the exascale age, once again, we need bold ideas to advance parallel filesystem performance if we are to keep with up the rapidly increasing scale of today's massively-parallel computing environments. In this presentation, we introduce DeltaFS, a research project at Carnegie Mellon University and Los Alamos National Lab. DeltaFS is based on the premise that at exascale and beyond, synchronization of anything global should be avoided. Conventional parallel filesystems, with fully synchronous and consistent namespaces, mandate synchronization with every file create and other filesystem metadata operations. This must stop. At the same time, the idea of dedicating a single filesystem metadata service to meet the needs of all applications running on a single shared computing environment, is archaic and inflexible. This too must stop. DeltaFS allows parallel computing jobs to self-commit their namespace changes to logs later published to a registry, avoiding the cost of global synchronization. Followup jobs selectively merge logs produced by previous jobs as needed, a principle we term No Ground Truth which allows for scalable data sharing without requiring a global filesystem namespace. By following this principle, DeltaFS leans on the parallelism found when utilizing resources at the nodes where job processes run, improving metadata operation throughput as job processes increase. Synchronization is limited to an as-needed basis that is determined by the needs of followup jobs, through an efficient, log-structured format that lends itself to deep metadata writeback buffering and deferred metadata merging and compaction. Our evaluation shows that no ground truth enables more efficient inter-job communication, reducing overall workflow runtime by significantly improving client metadata operation latency and resource usage.
Netflix Drive for Media Assets
Netflix Studios produces petabytes of media content accounting for billions of media assets. These assets are managed, created, edited, encoded, and rendered by artists working on a multitude of workstation environments that run on cloud, from different parts of the globe. Artists working on a project may only need access to a subset of the assets from a large corpus. Artists may also want to work on their personal workspaces on intermediate content, and would like to keep only the final copy of their work persisted on cloud. Ever wondered about the architecture that works for this scale and provides artists with a secure, performant and seamless storage interface? In this talk, we present Netflix Drive, a Cloud Drive for Studio Media applications and a generic paved path solution for storing and retrieving all assets in Netflix. Netflix Drive ties together disparate data and metadata stores in a cogent form for creating and serving assets. Talk Structure: In this talk, we will share with the audience how Netflix Drive is an extensible, scalable, performant, hybrid architecture for managing Studio and Media assets. We explore how Media pipelines leverage the dynamic namespace design provided by Netflix Drive to expose pertinent assets to artists. We also highlight different instance types of Netflix Drive that open up several integrations with tools and workstations used by Studio artists. Key Takeaways: As studio applications generate and consume assets, there is a need to design scalable architectures that work in cloud and on-premise, provide a globally consistent view of data, and integrate seamlessly with artist workflows. In this talk, attendees would learn about an extremely performant and scalable file system built using FUSE to provide an intuitive interface to artists, and how multiple data & metadata stores, which can be on-premise or in cloud, can be plugged into Netflix Drive’s ecosystem. Attendees will also learn how different instances of Netflix Drive can be used by different studio applications and workflows to store and retrieve pertinent content.
A Tiering-Based Global Deduplication for a Distributed Storage System
Reducing the amount of data is a huge advantage of saving a total cost of ownership for a distributed storage system. To do this, a deduplication method which removes redundant data is being used as one of the promising solutions to save storage capacity. However, in practice, traditional deduplication methods designed for a local storage system is not suitable for a distributed storage system due to several challenging issues. First, I/O overhead due to additional data and metadata processing can have a huge impact on performance, and the deduplication ratio is not high enough due to data distributed across multiple nodes. Second, it is not easy to design efficient metadata management for deduplicated data along with legacy metadata management due to scale-out characteristics. To address these challenges, in this talk, we propose a global deduplication method with a multi-tiered storage design and self-contained metadata structure. A tiering with deduplication-aware replacement policy can improve a deduplication efficiency by filtering out more important chunks which have high deduplication ratio. In addition, by adopting a self-contained metadata structure, it can also provide compatibility with existing storage features, recovery and snapshot. As a result, our proposed tiering-based global deduplication can reduce I/O traffic and save storage cost for a distributed storage system.
Fine Grain Encryption Control for Enterprise Applications
The Key Per IO (KPIO) project is a joint initiative between the NVM Express® and TCG Work Groups (WGs) to define a new KPIO Security Subsystem Class (SSC) under TCG Opal SSC for NVMe® class of Storage Devices. Self-Encrypting Drives (SED) perform continuous encryption on user accessible data based on contiguous LBA ranges per namespace. This is done at interface speeds using a small number of keys generated/held in persistent media by the storage device. KPIO will allow large numbers of encryption keys to be managed and securely downloaded into the NVM subsystem. Encryption of user data then occurs on a per command basis (each command may request the use a different key). This provides a finer granularity of data encryption that enables a granular encryption scheme in order to support the following use cases:
1) Easier support of European Union’s General Data Protection Regulations’ (GDPR) “Right to be forgotten”.
2) Easier support of data erasure when data is spread over many disks (e.g., RAID/Erasure Coded)
3) Easier support of data erasure of data that is mixed with other data needing to be preserved.
4) Assigning an encryption key to a single sensitive file or host object.
The presentation will include a brief introduction to the architectural differences between traditional SEDs and the KPIO SSC, followed by an overview of the proposed TCG KPIO SSC specification, and the features in the NVMe commands to allow use of KPIO. The talk will conclude by summarizing the current state of the standardization proposals with in NVM Express.org and the TCG Storage WG.
TCG Storage Workgroup Status Update
This is an overview of the new standards work being defined in the storage work group of the TCG. This includes overview of the TCG Opal SSC, SIIS (Storage Interface Interactions Specifications), Configurable Namespace Locking, and Key Per IO. The session may also touch upon some of the enhancements being worked on in the work group such as Settable Trylimits and Persistence feature set.
