Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod - Chris Vela, Runpod

Why watch

Runpod engineer Chris Vela demonstrates data-driven vLLM configuration for LLM inference, introducing BenchPod for live benchmarking without service restarts. The video shows Kimi K3 inference on B300 hardware, concrete tuning of max batch tokens to reduce inter-token latency, and challenges whether teams actually need expensive frontier models. For inference engineers and MLOps practitioners optimizing LLM deployments.