How to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada

Why watch

This video demonstrates how to test AI agents in enterprise environments where no public benchmarks exist. Nathanael Weill presents a reverse methodology: write the answer first, generate input from it, then verify the agent finds the correct result. He covers five techniques for building test sets including planted rule violations, removed clauses, and contract templates, plus explains why deterministic code-based verification is preferred over LLM judges when possible.