From 15% to 90% GPU Utilization: Fix the Data Pipeline, Not the Model

Why watch

Tarun Sunkaraneni from Amazon AGI demonstrates how to increase GPU utilization from 15% to 90% by fixing data pipeline bottlenecks in multimodal training. The talk covers diagnosing IO-bound training with Qwen3-VL-style models, implementing concurrency with asyncio and Ray actors, prefetching strategies, zero-copy transfers via Ray's object store, and scaling across nodes to overcome single-machine network card bottlenecks. Ideal for ML engineers and data scientists optimizing training infrastructure.