Using Reinforcement Learning to Optimize Storage Decisions

Effective use of distributed storage systems requires real-time decision making: what nodes to read from, where to write new data, and when to schedule maintenance operations to name a few. Effectively using available resources is everyone's goal, but in systems as complex and dynamic as distributed storage, the number of variables makes it impossible for any developer to work out every possible situation in advance. Therefore, making optimum decisions requires building intelligent logic into the storage application. But optimizing the logic and getting the information to base decisions on is not easy. In this talk we show that many decision problems in distributed storage are solved by the "multi-armed bandit" model, a well researched approach in reinforcement learning. We also explain how we've put multi-armed bandits to use in our product, to create adaptive agents that make performance optimizing storage decisions in real-time.

Learning Objectives

The importance of adaptive intelligence in large scale distributed storage systems
The basics of the exploration/exploitation trade off
Statistical approaches to the "Multi-armed Bandit" and "Thompson sampling"
How to implement distributed on-line and real-time learning