Enabling Heterogeneous Memory in Python

Library Content Type:
Publish Date: 
Tuesday, September 28, 2021
Event Name: 
Event Track:
Abstract: 

Adopting new memory technologies such as Persistent Memory and CXL-attached Memory is a challenge for software. While libraries and frameworks (such as PMDK) can help developers build new software around emerging technology, legacy software faces a more severe challenge. At IBM Research Almaden we are exploring a new approach to managing heterogeneous memory in the context of Python. Our solution, PyMM, focuses on ease-of-use and is aimed primarily at the data science community. This talk will outline PyMM and discuss how it is being used to manage Intel Optane persistent memory. We will review the PyMM programming abstractions and some early data science use-cases. PyMM is currently an early research prototype with open source availability.

  • Understand the emergence of heterogeneous memories (e.g., Optane, CXL-attached)
  • Understand the challenges facing integration of legacy s/w with new memory technology
  • Introduction and demonstration of PyMM
  • Outline of some existing use cases.

Watch video: