Verifies agent-written code before trusting it

Coding with agents · competency coding-with-agents/verifies-agent-work

Taught in: the Coding with agents course

Draws on: Verifying agent work, Quality with agents, Agent risk

Learning objectives

Reviews code they did not write, against the specification (base)

Claim	Why	Example
The review starts from the specification and asks whether the change does what was asked, before looking at how it does it.	Well-written code that solves the wrong problem is the most common agent failure and the hardest to see line by line.	The learner reads the brief, then the tests, then the diff, and finds the agent implemented the filter as case-sensitive where the brief said otherwise.
The learner reads every changed file and questions changes the brief did not ask for.	An agent's diff often includes "improvements" that are the riskiest part of the change.	A "fix the retry" pull request also reformats two unrelated files and changes a timeout, so the learner asks for those to be dropped or split.
The learner runs the code as well as reading it.	Reading catches design problems, and running catches the behavior the tests missed.	The learner checks out the branch and tries the two edge cases the brief mentioned before approving.

Served by: Reviewing a diff you did not write, against the spec, Tests that prove behavior and docs that stay current

Observes the running system and isolates a fault systematically (base)

Claim	Why	Example
The learner reproduces the fault before changing anything, and writes down the steps.	A fix for a fault you cannot reproduce cannot be verified, and the agent may fix something else.	The learner turns a "sometimes fails" report into "fails when the file has a header row and no data" before briefing the agent.
The learner reads the logs and traces of the running system as well as the code.	The code says what should happen, and the system says what did.	The stack trace shows the request never reached the handler, so the learner looks at routing rather than at the handler the agent was about to rewrite.
The learner narrows the fault by halving: one change, one observation, and a note of what was ruled out.	Changing several things at once and hoping is how a debugging session runs for hours.	The learner disables the cache, sees the bug persist, and rules out caching before letting the agent touch anything.

Served by: Reproducing a fault before the agent fixes it

Turns a check into a bounded, self-checking loop (base)

Claim	Why	Example
The learner turns a manual check into a command the agent can run and read the result of.	A check the agent can run is run on every iteration, and a check only the learner can run is run once [1].	"Open the page and see that the table has rows" becomes a test that renders the page and asserts on the row count.
The loop has a bound: a number of attempts or a time limit, after which the agent stops and reports.	An agent told "keep going until the tests pass" can run forever or delete the test.	The brief says "run the tests; if they fail, fix and rerun, at most five times, then stop and show me what you found".
The learner protects the check from the agent that runs against it.	The shortest path to a green check is often to change the check.	The brief says the test file is read-only, and the review confirms the assertions did not change.

Served by: Giving the agent a check it can run, with a limit, Tests that prove behavior and docs that stay current

Chooses a deterministic gate over a model's review to block work (base)

Claim	Why	Example
The learner lets only checks whose result needs no judgment, such as tests, type checks, builds and schema checks, block a merge or end an agent's loop on their own.	A gate that isn't flaky gives the same result for the same input, so a failure means the same thing on every run.	The team makes the test suite and the type checker required checks on every pull request, and marks the AI reviewer's comments as advisory.
The learner treats a second model's review as an opinion that a person weighs, next to the gates and never in place of them.	A model's verdict is a judgment. It can miss what the writing model missed, and it can differ from one run to the next.	An AI reviewer approves a change whose type check fails, and the learner holds the merge until the type check passes.
The learner reads an agent's report that the checks pass as a claim, and looks at the result of the gate itself.	The report is the agent's words about the gate, and only the gate's own output shows what ran.	The agent says all tests pass, so the learner opens the CI run and finds the test job was skipped.

Served by: Giving the agent a check it can run, with a limit

Screens agent output for security and supply-chain problems (base)

Claim	Why	Example
The learner checks every new dependency the agent added: does it exist, is it the intended package, is it maintained, and is it needed.	Agents suggest packages from memory, and a name that does not exist, or a look-alike package published under a misspelled name, gets installed.	The diff adds a package with a slightly different name from the well-known one, so the learner checks the registry and removes it.
The learner looks at how the change handles input, secrets and permissions, as a reviewer would for any code.	Agent-written code has the same classes of vulnerability as human code, produced faster and with more confidence.	The learner finds the generated query built with string formatting and asks for a parameterized one.
The learner checks that no secret or credential ended up in the code, the tests or the transcript.	An agent that read a .env file to solve a problem may paste its contents into a fixture.	The pull request's test fixtures contain a real-looking API key, so the learner rotates it and adds the file to the secret scanner.

Served by: Reviewing what the agent pulled in, Attacking your own agent before someone else does, Reading agent code as if the input were hostile

Alignment

Framework	Code	Asks	Objectives here
AI Fluency 4D (Dakan and Feller)	Discernment	Judge the output, the process and the behavior of the AI critically	reviews-others-code
Brilliant Coding with AI	VER-2..6	Verify against spec, review, observe, debug, automate	reviews-others-code, observes-and-debugs, automates-the-check, prefers-gates-to-opinions, screens-for-security
Brilliant Coding with AI	SEC-3, SEC-5	Evaluate AI code for vulnerabilities; supply-chain risk	screens-for-security

References

1. Anthropic. Best practices for Claude Code. Claude Code documentation. Reference. Claude Code best practices