Agentic Search vs Vector Search for Coding Agents: We Ran the Eval — Braintrust

Why watch

Jess Wang from Braintrust demonstrates how to build and run evals for AI coding agents by comparing agentic search (grep, find, file reading) against vector search over embeddings. Using Claude Code on Microsoft's TypeScript Go repo fix PRs and the repo's own test suite as scorer, both approaches achieved identical accuracy but vector search cost 4x more due to missing contextual connections between code chunks. This talk is for AI engineers and developer advocates building or evaluating AI coding tools.