Automated storage tiering using machine learning

In the current era there is a general transition from a human developed automation storage system to machine learning based intelligent storage system. Many storage solutions support multiple tiers of storage coming from on premise and cloud storage in a single namespace. The storage tiers can vary in various dimensions such as performance, resiliency, cost. The storage solutions also provide mechanisms to move objects from one tier to a different tier depending on various parameters, such as last access time, last modification time, access frequency, size of the object. There are thresholds to those parameters and objects are moved when the thresholds are crossed. But administrators are expected to set the threshold values, which puts the onus on the administrators to set the values properly to make optimal use of the storage. The presentation will talk about the challenges for an administrator to set the correct threshold value and will propose a machine learning based approach to help the administrator in finding the threshold value. The machine learning based approach (neural networks/ARIMA, clustering techniques) uses historical data to create a model to predict future access for the objects and use the predicted values to either do the automated tiering or suggest proper threshold values. The presentation will also cover some of the results obtained by applying some techniques on in-house data.