Evals in AI: A Deep Dive — Tejas Kumar, IBM

Why watch

Tejas Kumar demonstrates building production-ready AI evaluation systems, starting with simple assertions that fail on nuanced customer support scenarios and evolving into LLM judges with inherent biases. The workshop covers creating evaluation datasets, calibrating judges against position bias and self-preference, setting CI gates at 80% agreement thresholds, and using OpenRAG for real-time policy retrieval. Ideal for AI engineers building reliable evaluation pipelines for LLM-powered systems.