Accelerating Near Real-time Algorithms Using Disaggregated Computational Storage

Computational storage can bring unique benefits in increasing the efficiency of CPU utilization in a data processing system. Here we discuss the benefits of leveraging computational storage in a disaggregated storage environment. We demonstrate the ability of the solution to complement the CPU by taking away tasks that benefit from in-situ processing within the storage, thereby improving the overall system performance while lowering the TCO. Disaggregated storage is particularly attractive when using computational storage since scaling storage naturally yields to scaling of tasks that can be accelerated using computational storage. We experimented with accelerating the S3 Select functionality using our disaggregated computational storage (DCS) platform. Data tagging and partitioning utilizing sharding aspect of DCS platform further enhances ability to provide even greater performance for large data processing with parallel execution.