Decomposing Compute to Grow Computational Storage | SNIA

Abstract

Recent hardware trends have introduced a variety of heterogeneous compute units while also bringing network and storage bandwidths within an order of magnitude of memory subsystems. In response, developers have used increasingly exotic solutions to extract more performance from hardware; typically relying on static, design-time partitioning of their programs which cannot keep pace with storage systems that are layering compute units throughout deepening hierarchies of storage devices. We argue that dynamic, just-in-time partitioning of computation offers a solution for emerging data systems to overcome ever-growing data sizes in the face of stalled CPU performance and memory bandwidth.

We are prototyping a computational storage system (CSS) that adopts a database perspective to utilize computational storage devices (CSx). We discuss our approach, "decomposable queries," for passing a portion of a query plan (sub-plan) down a hierarchy of CSx and allowing each device to dynamically determine how much of the sub-plan to propagate and execute. We briefly describe microbenchmarks used to evaluate aspects of our initial progress for a bioinformatics use case, from which we observed unexpected slowdowns on our CSDs (Seagate's research-only KV drives). We then explain how decomposable queries will allow our prototype CSS to transparently benefit from future improvements to CSx and accommodate new types of CSx, and how these are two capabilities are necessary for deployment of CSx in production.