How to Test Your Agents on Real Tasks With Harbor - Xander Song, Arize AI

Why watch

This video demonstrates Harbor, an open-source framework for evaluating AI agents on realistic tasks using isolated Docker sandboxes. It covers task definition, verifiers, oracles for pre-run validation, and running agent tests at scale with Phoenix integration. The presenter shows how Arize AI's Phoenix team uses Harbor to test prompt changes and detect regressions, making it essential for ML engineers building production agent systems.