Beyond POSIX: Building a Protocol-Native VFS for User-Space NAS

Modern NAS protocols are not simply remote wrappers around POSIX. NFSv4 and SMB3 expose richer client intent through compound operations, stable handles, stateful opens, leases, delegations, durable handles, replay behavior, and metadata operations. S3 adds object semantics that further strain a filesystem-shaped internal API. When a user-space NAS server immediately decomposes these requests into blocking open(), read(), write(), and stat()-style calls, it loses information that could have improved correctness, scheduling, caching, and backend execution.

This talk explores the architecture of a protocol-native user-space VFS: an API where NAS operations, compound structure, stable object identity, protocol state, replay handling, and metadata semantics are first-class concepts.  That can help backends avoid redundant lookups, batch metadata work, reason about idempotency, and, where appropriate, map related operations onto transaction-like backend units while maintaining protocol-specific failure semantics.

The session will also examine why asynchronous, completion-driven APIs are a better fit than blocking syscall-style interfaces for high-operation-rate NAS services. Event-driven VFS design affects buffer ownership, backend plugins, scheduling, cache interaction, and the ability to multiplex protocols and transports efficiently. We will also discuss how user-space NAS designs can leverage modern hardware and transport capabilities, including TCP acceleration, RDMA, io_uring, and zero-copy I/O.

These lessons are drawn from building Chimera, an open-source multi-protocol NAS stack serving NFS, SMB, and S3 from a shared user-space VFS.