SNIA Developer Conference September 15-17, 2025 | Santa Clara, CA
Design Modern Object Store Server for Lustre file system in the Era of Solid State Storage and Persistent Memory
Abstract
Fast, scalable parallel file system performance is a key enabler of massively parallel computing as well as of emerging big data and machine learning applications. Released almost two decades ago, Lustre has long been the storage solution of choice for many supercomputing data centers. But as the world slowly retire rotational disks in favor of fast SSDs and persistent memory for their performance tiers, Lustre is becoming increasingly unable to fully utilize available storage bandwidth due to its old, disk-oriented object storage server designs based on Ext4 derived ldiskfs. In this talk, we will first introduce the high level design of Lustre key networking and storage components (Object Storage Server and Object Storage Target). Then we will describe the limitations and shortcomings we consider that are becoming major impedance to further Lustre innovations and hurdle to faster cadence of development. In order to address these issues, we propose a completely new OSS/OST architecture and implementation by natively incorporating latest advancements in hardware (SSDs, Persistent memory), software (SPDK/PMDK, KV-Store) and other relevant technologies such as flexible Erasure Coding data protection scheme. We believe the benefits of this proposal will lower the barrier to future innovative contributions and can invigorate developer community activities, which is essential to ensure Lustre’s continued success in end user adoption and support.
Learning Objectives
- To understand the architecture of Lustre file system and the high-level design of key networking and storage components (OSS/OST) and its limitations & shortcomings.
- To gain insight into our technical proposal and engineering approaches to incorporate latest advancements in hardware, software and other technologies to design and implement a new modern OSS/OST.
- To learn the current status of our ongoing project and effort in implementing the proposal, including the lessons we have learned so far.
Related Sessions
Highly Scalable, Masterless, Distributed Filesystem at Rubrik
Rubrik is a cybersecurity company protecting mission critical data for thousands of customers across the globe including banks, hospitals, and government agencies. SDFS is the filesystem that powers the data path and makes this possible. In this talk, we will discuss challenges in building a masterless distributed filesystem with support for data resilience, strong data integrity, and high performance which can run across a wide spectrum of hardware configurations including cloud platforms. We will discuss the high level architecture of our FUSE based filesystem, how we leverage erasure coding for maintaining data resilience and checksum schemes for maintaining strong data integrity with high performance. We will also cover the challenges in continuously monitoring and maintaining the health of the filesystem in terms of data resilience, data integrity and load balance. Further we will go over how we expand and shrink resources online from the filesystem. We will also discuss the need and challenge of providing priority natively in our filesystem to support a variety of workloads and background operations with varying SLA requirements. Finally, we will also touch on the benefits and challenges of supporting encryption, compression, and de-duplication natively in the filesystem.
Scalable Metadata in Distributed File Systems: Revisiting the GoogleFS Design for Exabyte-Scale Namespaces
GoogleFS introduced the architectural separation of metadata and data, but its reliance on a single active master imposed fundamental limitations on scalability, redundancy, and availability. This talk presents a modern metadata architecture, exemplified by SaunaFS, that eliminates the single-leader model by distributing metadata across multiple concurrent, multi-threaded servers. Metadata is stored in a sharded, ACID-compliant transactional database (e.g., FoundationDB), enabling horizontal scalability, fault tolerance through redundant metadata replicas, reduced memory footprint, and consistent performance under load. The result is a distributed file system architecture capable of exabyte-scale operation in a single namespace while preserving POSIX semantics and supporting workloads with billions of small files.
Choosing Your AI Storage Protocol: A Deep Dive into SMB and NFS Performance, Tuning, and Overhead
The performance of network file protocols is a critical factor in the efficiency of the AI and Machine Learning pipeline. This presentation provides a detailed comparative analysis of the two leading protocols, Server Message Block (SMB) and Network File System (NFS), specifically for demanding AI workloads. We evaluate the advanced capabilities of both protocols, comparing SMB3 with SMB Direct and Multichannel against NFS with RDMA and multistream TCP configurations. The industry-standard MLPerf Storage benchmark is used to simulate realistic AI data access patterns, providing a robust foundation for our comparison. The core of this research focuses on quantifying the performance differences and identifying the operational and configuration overhead associated with each technology.
New Transports in Samba: QUIC and SMB-Direct Support
The Samba file server is evolving beyond traditional TCP-based transport. This talk introduces the latest advancements in Samba's networking stack, including full support for SMB over QUIC, offering secure, firewall-friendly file sharing using modern internet protocols. We’ll also explore the ongoing development of SMB over SMB-Direct (RDMA), aimed at delivering low-latency, high-throughput file access for data center and high-performance environments. Join us for a deep dive into these transport innovations, their architecture, current status, and what's next for Samba’s high-performance networking roadmap.
Samba 2025: Enterprise-Ready, Cloud-Optimized
Samba is evolving to meet the demands of modern enterprise IT. The latest advancements bring critical SMB3 capabilities that boost scalability, reliability, and cloud readiness. With features like SMB over QUIC, Transparent Failover, and SMB3 Directory Leases now arriving, Samba is positioning itself as a robust solution for secure, high-performance file services across data centers and hybrid cloud environments. Learn how these enhancements can future-proof your infrastructure - without vendor lock-in.
