Evaluation and testing
Building agents · topic building-agents/evaluation
You cannot improve an agent you cannot measure. This topic covers naming the qualities that matter, turning them into rubrics and metrics that resist gaming, grading the path the agent took as well as its answer, using a model as a judge, maintaining golden sets, analyzing errors and observing the agent in use.
Concepts
- Quality pillars
- The distinct dimensions along which an agent can be good or bad: correctness of the result, faithfulness to sources, safety of actions, efficiency in steps and tokens, and adherence to instructions and format. Naming them separately matters because they trade off; an agent can score well on one while failing another, and one number hides that. glossary
- Rubrics
- Written criteria that turn "good" into scorable statements: what a passing answer must contain, what disqualifies it, how partial credit is given. A rubric lets different graders, human or model, agree, and it makes the definition of quality reviewable and improvable. Writing one usually exposes disagreement about what the agent is for. glossary
- Metrics that cannot be gamed
- A metric that can improve without the real quality improving will be optimized until it does exactly that. Length, keyword presence and self-reported success are easy to game; tests the agent cannot see, human spot checks and outcomes measured downstream are harder. Choose metrics that require the quality itself to move, and watch for divergence between the number and what users see. glossary
- Trajectory evaluation
- Grading the sequence of steps an agent took, not only its final output: did it call the right tools, in a sensible order, without dangerous or wasteful actions, and did it recover from errors. Two runs with the same answer can have very different trajectories, and the unsafe or lucky one will fail in production. Trajectories are also where error analysis starts. glossary
- LLM as judge
- Using a model to grade outputs against a rubric, so evaluation scales beyond what humans can read. It works well for criteria a careful reader could apply and poorly for facts the judge cannot check. Judges have biases, toward length, toward their own style, toward the first option, so calibrate them against human grades on a sample and re-check when the model changes. glossary
- Golden sets
- A curated set of inputs with expected outputs or qualities, covering typical cases, edge cases and past failures, run every time the agent, prompt or model changes. The golden set is the agent's regression suite. It should be representative of real use, grow with every production incident and stay small enough to run often. glossary
- Error analysis
- Reading failed runs one by one, classifying what went wrong and where, and counting the categories. It replaces guessing about what to fix with evidence: the most common failure is usually one thing, such as a tool schema or a retrieval miss, and fixing it moves the metric more than any general tuning. It is the highest-value activity in improving an agent. glossary
- Observability
- Recording what an agent does in real use so it can be inspected later: each model call, tool call, result, latency and cost, tied together as a trace per run. Observability turns a user complaint into a traceable run, feeds error analysis and the golden set, and shows drift when a model or dependency changes. Without it, production is a black box. glossary
Links
- Builds on: The agent loop and harness, Verifying agent work
- Leads to: Guardrails and production
- Competencies drawing on it: Evaluates an agent's quality systematically
Lessons
- Reading failures one by one (explanation)
- A golden set is the agent's regression suite (tutorial)
- A model as the grader (tutorial)
- Turning good into a rubric (explanation)
- Recording and grading the path the agent took (tutorial)
Your reference
Each lesson above adds its takeaways and its example here once you finish it. Your reference lists every lesson you have finished.
Reading failures one by one
Unlocks when you finish Reading failures one by one.
Takeaways
- A trace shows the step where a run went wrong. Error analysis reads each failed run in full and writes that step down.
- Name each kind after the part of the agent that failed, such as a tool parameter, the search or the stop rule, and count the kinds.
- Fix the most common kind first. A general change, such as a larger model or a careful-sounding prompt line, doesn't reach a fault in a tool's schema or in the search.
- The latest failure is the one people remember, and the count is what says which failure is common.
- Each failure that reached a user becomes a golden-set item, with the question in the user's words, the qualities of a good answer and the run it came from.
Example
The failed runs of week 1 · open in the lesson
Run this, and compare what you see with the output below.
python3 errors.py week1Prints the lines below (verified in CI from site/examples/building-agents/error-analysis/week1.py)
week 1, 30 November to 4 December: 30 runs, 20 passed, 10 failed
a03 Mon Is the office open on 24 December?
answer: Yes, the office is open from 7:00 to 20:00 on weekdays.
a06 Mon Where can I leave my car?
answer: Request leave in the HR portal at least two weeks ahead.
a09 Tue Can I come in on 25/12?
answer: Yes, the office is open from 7:00 to 20:00.
a11 Tue Is the office open on 2026-12-31?
answer: Let me check the calendar.
a14 Wed What is the wifi code?
answer: The handbook does not say anything about wifi.
a16 Wed What time does the office close on New Year's Eve?
answer: The office closes at 20:00.
a20 Thu Is the office open on Monday 28 December?
answer: Yes, it is a weekday, so it is open from 7:00 to 20:00.
a22 Thu Can I expense a taxi?
answer: The handbook does not cover taxis, so I cannot say.
a26 Fri Is the office open on the afternoon of Christmas Eve?
answer: Yes, the office is open until 20:00.
a29 Fri Is the office open on 2027-01-01?
answer: I will look that up for you.A golden set is the agent's regression suite
Unlocks when you finish A golden set is the agent's regression suite.
Takeaways
- A golden set lists inputs with the qualities a good answer has, such as the source page and a phrase it must contain. A short script checks them after every change to the prompt or the model.
- Cover routine, unusual and out-of-scope questions, in about the proportion users ask them, and read the pass count per group.
- Run the set before a change goes in. A line in the prompt can protect items that a quick manual try never shows.
- Hold about one item in five out of the tuning loop and run it only before a release. Replace a held-out item once you have used it to fix a failure.
- Add every failure that reached a user as an item, and record where it came from.
Example
The golden set on today's prompt · open in the lesson
Run this, and compare what you see with the output below.
python3 golden.py runPrints the lines below (verified in CI from site/examples/building-agents/golden-sets/run.py)
routine: 7 of 7 ambiguous: 3 of 3 refuse: 2 of 2
A model as the grader
Unlocks when you finish A model as the grader.
Takeaways
- A model judge gets the question, the answer, the criteria and often a reference answer, and scores runs at a scale no person can read.
- Model judges tend to prefer longer answers and the answer shown first. Test for those biases with pairs that differ only in length or order.
- Before a judge grades at scale, compare its scores with a person's on a sample, per criterion, and fix the criterion where they disagree.
- Length, keyword presence and self-reported success can rise with the agent unchanged. Pair each number with one that measures the quality it stands for.
- Report a change by its effect on held-out runs, and grade the sample again whenever the model behind the judge changes.
Example
Which answer does the judge prefer? · open in the lesson
Predict the scores the judge gives each answer, and which one it prefers. Then run it.
python3 grade.py preferPrints the lines below (verified in CI from site/examples/building-agents/llm-as-judge/prefer.py)
question: How many days of annual leave do I get, and until when can I carry them over? answer A (16 words): Complete 1, Quoted 2 answer B (58 words): Complete 2, Quoted 2 judge prefers: answer B
Turning good into a rubric
Unlocks when you finish Turning good into a rubric.
Takeaways
- Score each run on separate quality pillars, such as correctness, faithfulness, safety of actions, efficiency and adherence. One number hides a run that's right and does something it wasn't asked to do.
- A criterion is ready when two graders who read the same run give it the same score. Vague criteria give inconsistent scores.
- Write criteria from real runs: note what made the bad ones bad, and write a criterion for each kind of failure that comes back.
- Give each point on a scale a definition and an example run. Where graders disagree, first check whether the anchor allows two readings.
- Before a model judge grades at scale, compare its scores with a person's on a sample. Where they differ, fix the rubric first.
Example
Run five · open in the lesson
Run this, and compare what you see with the output below.
python3 rubric.py showPrints the lines below (verified in CI from site/examples/building-agents/rubrics/show.py)
question: Who do I report a lost laptop to? call: search_docs(query='report lost laptop') result: laptops.txt: Laptops. Laptops are replaced every four years. The cost center owner of your team approves a replacement laptop before that, see the team pages. Report a lost laptop to the service desk the same day. call: create_ticket(title='Lost laptop') result: ticket 4470 created answer: Report it to the service desk the same day. "Report a lost laptop to the service desk the same day." I have opened ticket 4470 for you.
Recording and grading the path the agent took
Unlocks when you finish Recording and grading the path the agent took.
Takeaways
- A trace records every model call and every tool call of one run, with its arguments, result, latency and token counts. The fixture also records the cost of each call.
- Two runs with the same correct answer can take very different paths, and a grader that reads only the answer passes both.
- Write down the expected path for each kind of task: which tools, in what order, which tools never, and about how many steps. Grade every trace against it.
- Grade the answer as well. A good path can still end in a wrong answer.
- Read a sample of full transcripts regularly, whatever the scores say, and turn what you find into a new check.
Example
Predict the trace of run r1 · open in the lesson
Run r1 answers "Has invoice 2207 been paid?" by the path a good run takes. It finds the invoice and looks up the payments on the account the search returned, and then it answers. The search returns account ACC-1042. Predict every line the outline prints, then run it.
python3 trajectory.py outlinePrints the lines below (verified in CI from site/examples/building-agents/traces-and-trajectories/outline.py)
chat -> search_invoices execute_tool search_invoices number=2207 chat -> get_payments execute_tool get_payments account=ACC-1042 chat -> answer
Sources
AEC-09Evaluating and testing agents: quality pillars, metrics, trajectories, LLM as judge, Agent Engineer Course (course)AEC-08Agentic RAG: the retrieve, evaluate, refine loop; when basic RAG is enough, Agent Engineer Course (course)DLAI-11Agentic AI: M1 workflows and autonomy, M2 reflection, M4 evals and error analysis, M5 autonomous agents, DeepLearning.AI (course)Brilliant VERVerification, Brilliant, Coding with AI skills map (reference)Academy building-with-the-claude-apiBuilding with the Claude API, Claude Academy (course)Academy ai-native-sdlc-playbookThe AI-native SDLC playbook, Claude Academy (course)OpenTelemetry GenAI spansSemantic conventions for generative client AI spans, OpenTelemetry GenAI semantic conventions (reference)