Demystifying Data Flows During LLM Training
This presentation walks through how data flows through GPUs during Large Language Model training. There is an introduction to LLM neural networks and how they map to the GPU arrays. We look at the challenges of current GPU topologies and look at how these topologies will evolve with the introduction of UEC and UALink.