Ceph Storage in a World of AI/ML Workloads

Abstract

Artificial intelligence and Machine learning (AI/ML) is a hot topic in every business at the moment, and there is a growing dialog about what constitutes an Open Model, is it the weights? Is it the data?

Those are important questions, but equally important is ensuring that the tooling and frameworks to train, validate, fine-tune, and perform inference are open source. Storage systems are a crucial component of these workflows, how can open-source solutions address the needs for high capacity and high performance? Open source solutions like Ceph can provide almost limitless scaling capabilities, both for performance and capacity. In this webinar learn how Ceph can be used as the backing store for AI/ML workloads.

The LTFS Bulk Transfer standard defines a method by which a set of files, directories and objects from a source system can be transferred to a destination system. The bulk transfer of large quantities of data is well suited for LTFS due to the economic and environmental characteristics of tape.

We’ll cover:

• The demands of AI on storage systems
• How open source Ceph storage fits into the picture
• How to approach Ceph cluster scaling to meet AI’s needs
• How to get started with Ceph