Evaluates an agent's quality systematically

Building agents · competency building-agents/evaluates-agents

Taught in: the Building agents course

Draws on: Evaluation and testing

Learning objectives

Turns "good" into scorable criteria (base)

Claim	Why	Example
Each criterion in the rubric is something two people would score the same way on the same output.	A criterion that depends on the grader's taste measures the grader.	"Every claim in the answer is supported by a cited passage" replaces "the answer is well grounded".
The rubric is written from real outputs: good ones, bad ones and the arguments about them.	Criteria written from imagination miss the failures that occur and include failures that do not.	The learner reads thirty transcripts, notes what made the bad ones bad, and writes a criterion for each recurring kind.
Each criterion has a scale with anchored points, and the anchors are examples.	"Rate 1 to 5" without anchors gives different 3s from different graders and from the same grader on different days.	"Accuracy: 0 = contains a false statement, 1 = correct but incomplete, 2 = correct and complete", with one example transcript for each.
The learner checks the rubric by having a model grade with it and comparing the results with human grades on a sample.	A rubric that a model applies differently from a person is not ready to run at scale.	The model judge agrees with the human on 26 of 30 samples, so the learner rewrites the two criteria behind the four disagreements.

Served by: A model as the grader, Turning good into a rubric

Builds a representative input set with expected qualities (base)

Claim	Why	Example
The set covers the kinds of inputs that occur in use, in roughly their real proportion, plus the rare cases that matter most.	An agent tuned on easy or typical inputs looks good and fails on the input that costs the most.	The support agent set has 60 routine questions, 20 ambiguous ones, 10 in other languages and 10 that should be refused.
Each item records the expected qualities of a good answer instead of one exact answer.	Agents produce many correct answers, and matching one string fails all the others.	"Must mention the 30-day window, must not promise a refund, must offer the form link" instead of a reference reply.
The set grows from production failures and is versioned with the agent.	Every failure that reached a user is a test case the set was missing.	A user got a wrong answer about a holiday closure, so the question and the expected qualities go into the set with a link to the incident.
The learner keeps a part of the set that is never used for tuning.	A set the team iterates against becomes a set the agent has been fitted to.	Twenty percent of the items are held back and scored only at release time.

Served by: Reading failures one by one, A golden set is the agent's regression suite

Grades the path the agent took, not only the final answer (base)

Claim	Why	Example
The learner scores the steps the agent took (which tools, in what order, with what arguments) as well as the final answer.	A correct answer reached by luck or by a dangerous path is a failure the final-answer score cannot see.	The agent answered correctly after calling the payment API three times with the wrong account, and the trajectory score fails it.
The learner writes down what a good path looks like for each task type: which tools it should use, which it should not, and roughly how many steps.	Without an expected path there is nothing to grade the actual path against.	"A lookup question should call search once and answer; more than three tool calls is a warning sign."
The learner reads a sample of full transcripts by hand every release, however good the scores.	Automated grading finds the failures it was built to find, and the new failure mode is in the transcripts.	The scores are unchanged, and the transcripts show the agent has started apologizing in every reply since the prompt change.

Served by: Reading failures one by one, Recording and grading the path the agent took

Chooses what to fix from counted failures (base)

Claim	Why	Example
Before changing the agent, the learner reads the failed runs of a period, such as a week, and counts them by the kind of step where each went wrong.	Without a count, the team fixes the failure it remembers best, which is often the latest or the one a manager forwarded.	Of 20 failed runs, 11 are search misses, 4 are made-up amounts, 3 hit the step limit and 2 answer in the wrong language.
The learner fixes the most frequent kind first, with a change aimed at the part of the agent that failed.	A targeted fix can turn every failure of its kind into a pass, and a general change such as a larger model often does not reach a fault in a tool or in the search.	Five of ten failures are dates the tool could not read, so the learner writes the date format into the parameter description before touching the stop rule.
The learner checks the fix on the next batch of runs and counts the kinds again.	A fix can miss its target or break runs that passed, and once it works another kind becomes the most common.	In the week after the date fix, no run fails on a date the tool could not read, and search misses are now the largest kind, so they are the next fix.

Served by: Reading failures one by one

Chooses metrics that cannot improve without the real quality improving (expert)

Claim	Why	Example
For each metric, the learner can describe a way to raise it without making the agent better, and either closes that way or pairs the metric with one that catches it.	A metric that can be gamed will be, by the team or by the model, without anyone intending it.	"Answers per hour" is paired with "answers accepted by the user", because the first rises when the agent answers badly and fast.
The learner checks metric changes against a held-out set and against human judgment on a sample.	A rise on the set the team tunes against may come from the tuning rather than from the agent.	The rubric score rose five points, but on the held-out set it rose one, so the learner reports one.
The learner prefers metrics tied to the user's outcome over metrics about the agent's output.	Output metrics measure the agent's behavior, and the goal is the user's result.	"Ticket resolved without reopening within seven days" replaces "response contains the policy link".

Served by: A model as the grader

Alignment

Framework	Code	Asks	Objectives here
Ng, AI engineering skills map	Building and deploying AI applications	Build, evaluate, ship	writes-a-rubric, builds-a-golden-set, grades-trajectories, prioritizes-fixes, avoids-gamed-metrics
Brilliant Coding with AI	VER-7	Measure quality and evaluate AI systems	writes-a-rubric, builds-a-golden-set, grades-trajectories, prioritizes-fixes, avoids-gamed-metrics