CSI Driver Design: Bringing a Parallel File System to Containerized Workloads

Library Content Type:
Publish Date: 
Tuesday, September 28, 2021
Event Name: 
Event Track:

The advent of cloud and the everything as a service (XaaS) model requires storage developers to rethink how their products are consumed. Organizations are looking to develop infrastructure and processes that are agnostic from any one cloud vendor, including their own on-premises datacenters. Container orchestrators (COs) like Kubernetes enable this ideal by allowing entire application deployments to be packaged up (in containers and manifests) and moved from environment to environment with relative ease. Container Storage Interface (CSI) provides a standard for exposing arbitrary block and file storage to these COs, but user experience for a particular CSI driver is heavily dependent on its implementation. Storage developers must carefully consider the key attributes of a storage product when developing a driver to ensure a cloud-like experience. Parallel file systems provide an excellent opportunity to take storage software often perceived as complex and expose advanced functionality and capabilities through a CO’s simple and familiar interfaces. In this presentation, Eric Weber and Joe McCormick will break down the thought processes and design considerations that went into developing a CSI driver for BeeGFS. We will walk through how the CSI spec’s gRPC endpoints were mapped onto BeeGFS specific functions and discuss specific CSI/CO functionality and sub-features that do or do not make sense in the context of BeeGFS. While the CSI spec is intended to be CO agnostic, we will tend to discuss our design in the context of Kubernetes. In addition to existing BeeGFS CSI driver functionality, we will also discuss potential future features in order to give our audience a well-rounded view of the available capabilities for container storage.

  • The participant will recognize the basic challenges associated with developing a CSI driver for an arbitrary storage platform.
  • The participant will recognize the unique challenges a parallel file system presents for CSI driver development.
  • The participant will understand the design decisions the BeeGFS CSI driver team made to overcome the above challenges.
  • The participant will be better prepared to design a CSI driver for a new/different storage platform.

Watch video: