Eval Set

A curated collection of representative tasks with known correct outcomes used to measure AI agent performance. Eval sets are run before every prompt change, model upgrade, and deployment to catch regressions early. A good eval set covers common cases, known edge cases, and historical failures—and grows over time as new failure modes are discovered in production.