Azure Files: Design Challenges for the Biggest File Server in the World

Stevens Creek

Wed Sep 18 | 11:35am

Abstract

For over nine years, Microsoft Azure has provided completely managed file shares in the cloud.
Azure Files provides SMB3, NFS4.1 and REST based access to file shares.

This talk will present the evolution of architecture of Azure Files to serve applications with higher performance and scale needs, based on Azure Storage architecture under the hood, not on a conventional file system -- let alone NTFS. We will focus on features which provide the availability and reliability guarantees despite the constant din of hardware underneath suffering failures and needing replacement. Leveraging the Continuous Availability features of SMB3, users are able to access always available and highly reliable file shares. We will also deep dive into our approach to achieve security at cloud scale.

This talk will highlight recently added features and the engineering challenges of making significant changes and additions to data schemas and the code that manipulates it, while still allowing access to those many petabytes of data, or breaking the semantics that applications depend on. Additionally, this talk will discuss the significance of receiving an ack for the write requests from Azure File Server, how we build back state to continue serving data when client reconnects and customer focused challenges and how we overcame them.

Learn about the evolution of fully managed file shares built on top of something other than a conventional file system, while still serving live customer traffic.
Gain an understanding of the challenges in durably committing what is typically seen as volatile handle state, ensuring it remains both highly available and performant.
Learn how Azure Files is iterating to improve scale, performance, and security
See through the eyes of the customer what the expectations are for the public cloud of the 21st century.