Beyond VLLM: Distributed LLM Inferencing With Llm-d on Kubernetes - Ravindra Patil, Red Hat

This talk introduces llm-d, a cloud-native project enabling distributed LLM inference across multiple GPUs and Kubernetes nodes. It contrasts vLLM's single-node efficiency with llm-d's multi-node capabilities, covering architecture, design trade-offs, and a practical demo. Ideal for platform engineers and ML infrastructure developers building production LLM systems.