Storage Industry: The Changing Paradigm Leading to Computational Storage

The Storage industry has been witnessing a paradigm shift with the emergence of new and disruptive technologies. In the past three years, the Storage space has witnessed a spurt in areas like all-flash & NVMe, Predictive Storage Analysis, Cloud Storage, Data Protection, IoT Analytics, Computational Storage, and AI-driven Storage. But where did all this begin? It all started with magnetic disks coming to the fore for filesystems to manage data and deliver performance. SAN brought with it flexibility, sharing, and improved performance. This was followed by ‘Shared Nothing’, which came in with a purpose to divide & conquer, and also to enable parallel processing. Solid State Drives (SSDs) made an appearance to help cut down service times considerably. By now, the CPUs and buses have experienced a manifold increase in speed. With great development, come greater challenges. The challenge faced by technologists was cutting down the overheads in the IO subsystems and the various kernel processes involved in servicing these IOs. One thing led to the other, and now we have NVMe. To boost storage performance and exploit the might of NVMe, a mechanism was devised to couple an application directly with the storage subsystem, bypassing the kernel. This led to concerns for data transfer times and hence the initiative to club Compute and Storage together as a unit – resulting in the birth of Computational Storage – the great hybrid with a massive potential to disrupt the market.