How We Built an Agent That Improves Itself — Zubin Aysola, Weights & Biases

Why watch

Zubin Aysola demonstrates how Weights & Biases built ARIA, an agent that analyzes its own production traces, converts them into offline eval tasks, identifies bugs like missing SDK calls, writes fixes, and benchmarks improvements. The team maintains byte-identical agents in production and research, defined via YAML for parallel testing, and runs 886 simulated multi-turn tasks. Every production miss becomes a new task, creating a flywheel for continuous self-improvement. Ideal for engineers building AI agent evaluation systems.