LPDDR Memory: A New CPU Memory Choice for AI Inference

As Large Language Models (LLM) evolve and become integrated across various networks, there is a growing demand for more advanced AI training and inference hardware to support their capabilities. The compute power for these LLMs is becoming increasingly constrained by memory bandwidth and power. AI Inference is now the dominant workload in modern computing, demanding new architectural approaches to handle data movement, retention, and processing.

LPDDR SDRAM, originally developed for low-power mobile products, is emerging as an alternative CPU memory solution for AI inference workloads. LPDDR offers a good balance of power efficiency, performance, capacity, and cost. LPDDR5X, now deployed in AI data centers in SOCAMM2 form factor, has demonstrated improved system performance with lower power on many of the latest LLMs.  This presentation will look at implementation options, performance benchmarks, and the evolution to LPDDR6 for next generation systems.