Where Moore's Law Meets the Speed of Light: Optimizing Exabyte-Scale Network Protocols

Scalability is critically important to distributed storage systems. Exabyte-scale storage is already on the horizon and such systems involve tens of thousands of nodes. But today’s Internet protocols were never designed to handle such cases. In systems this big it's impossible to maintain connections and sessions with every storage device. However, multiple round trip connection setups, TLS handshakes, and authentication mechanisms, compounded with the unyielding speed of light and geo-dispersed topologies create a perfect storm for high latency and bad performance. In this presentation we explore the available options to achieve security, performance, and low latency in a system where persistent sessions are an unaffordable luxury.

Learning Objectives

Limitations of today’s Internet protocols in large distributed systems
How to implement a secure, connectionless, single-round-trip network protocol
What the network topology will look like in an exabyte-scale globally dispersed storage system