Bare-Metal Abstractions for Modern Hardware

Hardware is getting smarter every day. GPUs, hardware accelerated networks, and non-volatile memories are increasingly replacing capabilities offered today by operating systems and software libraries. Leveraging them can yield orders of magnitude improvements in latency and throughput, but often only through the use of complex specialized SDKs instead of simple hardware-agnostic abstractions like socket or files. We present experimental abstractions that attempt to recover some of this simplicity, generality and interoperability, while remaining low overhead and allowing device-to-device data transfers and control flow. Our work-in-progress demos include data storage using hardware erasure codes present in recent network adapters, streaming data from storage to GPUs using RDMA, and executing a deep learning distributed compute graph entirely in hardware using GPUDirect Async.

Learning Objectives:
1. Review the state of programming models and software abstractions
2. Review real life challenges with managing GPU farms and running machine learning workloads
3. Explore candidate programming models simplifying said challenges