There is a demand for Hierarchical memory as use cases demand higher amounts of memory capacity and bandwidth as AI/ML, database applications scale up and scale out. Software-Defined Memory (SDM) is an emerging HW-SW co-design architecture paradigm that provides software abstraction between applications and underlying memory resources with dynamic memory provisioning to achieve the desired SLA. With emergence of newer memory technologies and faster interconnects, it is possible to optimize memory resources deployed in cloud infrastructure while achieving best possible TCO.