SNIA Developer Conference September 15-17, 2025 | Santa Clara, CA
GoogleFS introduced the architectural separation of metadata and data, but its reliance on a single active master imposed fundamental limitations on scalability, redundancy, and availability. This talk presents a modern metadata architecture, exemplified by SaunaFS, that eliminates the single-leader model by distributing metadata across multiple concurrent, multi-threaded servers. Metadata is stored in a sharded, ACID-compliant transactional database (e.g., FoundationDB), enabling horizontal scalability, fault tolerance through redundant metadata replicas, reduced memory footprint, and consistent performance under load. The result is a distributed file system architecture capable of exabyte-scale operation in a single namespace while preserving POSIX semantics and supporting workloads with billions of small files.
This presentation delves into the development and implementation of a multi-petabyte archive storage solution, focusing on the challenges and architectural choices made to accommodate Host Managed Shingled Magnetic Recording (HM-SMR) drives in a software-defined storage environment. We will discuss the sequential write constraints of SMR drives and compare various HM-SMR libraries from a software engineering standpoint.
In this talk, we'll delve into the transformative potential of selective write-grouping within software-defined storage (SDS) systems, setting it against the backdrop of the Popular Data Concentration (PDC) approach. Both strategies are compatible with Shingled Magnetic Recording (SMR) and Conventional Magnetic Recording (CMR) disks with an erasure coding. Write-grouping confines write operations to a limited subset of drives, allowing inactive drives to power down and slash energy consumption by up to 43% in SMR configurations.
