MarFS, Marchive, and GUFI - Long Term Storage Strategies at LANL

This talk will provide an overview of scalable data management solutions in development and production at Los Alamos National Lab. The talk will target three primary efforts: MarFS, Marchive, and GUFI. MarFS is a near-POSIX scalable namespace over flexible data stores with a focus on data resiliency, metadata scalability, and IO shaping for efficient disk utilization. Marchive (MarFS archive) is a natural extension of the MarFS architecture for archival systems. The Grand Unified File Indexing (GUFI) System is a hybrid indexing capability using both file system trees and embedded SQL to enable a fast and efficient file metadata indexing system that can be used by both system administrators and users due to its unique approach to securing access to the index.

Learning Objectives:
1. MarFS design goals, architecture overview, and production experience
2. Marchive (MarFS archive) design goals, data protection scheme, and current status
3. GUFI (Grand Unified File Index) design, implementation, and usage