Verifying outputs
Safety · topic safety/verification
Verification is the habit of checking AI output before relying on it, in proportion to the cost of being wrong. This topic covers routine checking habits, following claims back to their sources, treating endorsed answers differently from raw ones, and extending "trust but verify" to actions agents take.
Concepts
- Checking habits
- Small routines that catch model errors before they spread: read the output against the request you made, look for claims you did not supply, run the code, open the link, do one calculation by hand. The depth scales with stakes. A habit is better than a rule because it runs even when you are in a hurry, which is when errors slip through. glossary
- Source checking
- Following a claim, citation or quote back to where it is supposed to come from and confirming it says what the model reported. Models produce references that look right more easily than references that are right. Anything with a page number, URL, statute, version or study behind it should be opened before it goes to anyone else. glossary
- Endorsed answers
- An answer that a qualified human has reviewed and marked as correct, distinct from raw model output. Some teaching and support systems show the mark so readers know which answers carry human accountability. The idea transfers to teams: separate what the model said from what a person has checked, and make the difference visible in the artifact. glossary
- "Trust but verify" for agents
- When an agent acts rather than answers, verification moves from reading text to checking effects: what files changed, what was sent, what was deleted. Let the agent proceed on low-stakes steps, then inspect the result against what you asked for, and require a pause before any step that is hard to reverse. Trust is granted per action, not per tool. glossary
Links
- Builds on: Recognizing failure
- Leads to: Verifying agent work, Agent risk, Delegating to an agent
- Competencies drawing on it: Verifies AI output before relying on it
Lessons
- Asking a model to disagree (tutorial)
- Checking habits that run when you are in a hurry (tutorial)
- Checking what an agent changed (tutorial)
- Following a claim back to its source (tutorial)
- Marking what a person has checked (explanation)
Your reference
Each lesson above adds its takeaways and its example here once you finish it. Your reference lists every lesson you have finished.
Asking a model to disagree
Unlocks when you finish Asking a model to disagree.
Takeaways
- A model tends to agree with a plan you have shown you prefer, and it gives feedback that matches the opinion you stated about your own work. A yes to a leading question is agreement, and agreement isn't evidence.
- Ask for the strongest counter-argument before you ask for confirmation, in a chat that hasn't agreed with you yet. The model's own earlier yes is context, and it pulls the next answer toward yes.
- When you want an assessment, strip your opinion out of the question: describe the situation, name the options and ask for a comparison.
- A counter-argument prompt that also carries your verdict gets a counter-argument and a rebuttal of it. Ask for the case against, and nothing else, and answer it yourself afterwards.
- The fix for a leading answer is in the question. When you use a model's judgment as a check, the question sets the quality of the check.
Example
I've decided to merge our three short customer surveys into one longer survey that goes out once a year. Makes sense, right?
Yes, that makes a lot of sense. One survey a year means fewer emails for your customers and less survey fatigue, and you get one complete data set to analyze instead of three partial ones. It also cuts the work of preparing and sending three separate campaigns. Sounds like a solid decision.
Checking habits that run when you are in a hurry
Unlocks when you finish Checking habits that run when you are in a hurry.
Takeaways
- Before a model's output leaves your desk, read it against the request and look for claims you did not supply. Then open one link and redo one calculation.
- A total is a specific like any other, and its source is the arithmetic. One wrong row total in a table can put two wrong numbers on a slide, and the table looks the same either way.
- The check you are about to skip is the one you run. Errors get through at the moment a routine loses a step, which is when you are busy.
- The routine is the same every time, and its depth comes from the cost of being wrong: whether the output can be undone, and whether its readers check it themselves.
- A habit is better than a rule because it runs when you are in a hurry, which is when a rule gets skipped.
Example
This lesson has no runnable example or prompt block.
Checking what an agent changed
Unlocks when you finish Checking what an agent changed.
Takeaways
- When an agent acts, verification moves from reading its text to checking its effects: what changed, what was deleted, what was sent.
- Compare the before and after with the request, one effect at a time. A deleted file shows only in the before list, and a sent message shows in neither.
- The agent's summary is a raw answer about its own work. Read the effects first and the summary afterwards.
- Let the agent take steps you can undo on its own and check them afterwards. Put the pause before the step that can't be undone, such as a send, a permanent deletion or a payment.
- Trust is granted per action, by what the action costs if it goes wrong, and never once for the whole tool.
Example
Which file is gone? · open in the lesson
One file from the before list isn't in the after list under any name or in any folder. Type its name as the before list shows it.
Prints: notes-thursday-copy.txt
Following a claim back to its source
Unlocks when you finish Following a claim back to its source.
Takeaways
- A reference can fail in two ways. The source does not exist, or the source exists and says something else. The second is harder to catch, because the title and the number both check out.
- Source checking is a walk: open the source itself, find the passage, read what it says, and compare it with what the text says it says, one sentence each.
- Stopping at a title match or a number match is the pitfall. Only the sentence around the number tells you what it counts.
- A claim you cannot check before the deadline is marked unverified or taken out. It never stays in looking like a checked one.
- Anything with a page number, URL, statute or study behind it gets opened before it goes to anyone else.
Example
This lesson has no runnable example or prompt block.
Marking what a person has checked
Unlocks when you finish Marking what a person has checked.
Takeaways
- To the next reader, an answer a person has checked and one nobody has checked look the same until someone marks the difference, and an unmarked claim reads as a checked one.
- An endorsed answer is one a qualified person has reviewed and marked, shown apart from raw model output. On CS50's course forum, staff endorse the AI tutor's answers they have checked, so students can see which ones a person stands behind.
- A useful mark says who checked, when, and against what. "Checked" on its own says none of these.
- A missing mark means nobody marked the answer, which is a different thing from wrong. At CS50, the authors suspect a count of endorsements alone understated the tutor's accuracy. A count of marks in your team measures how much reviewing happened.
- Put the mark in the process, as a template line, a review step or a checklist item, and keep it light enough to fill in on a busy day. A heavy check gets ticked without being done, and then everyone assumes it happened.
- Sample a few documents a month to see whether the check is done, and make it normal to report a caught mistake. A habit held by one person leaves with that person.
Example
This lesson has no runnable example or prompt block.
Sources
CS50xCS50's Introduction to Computer Science, Harvard CS50 (course)Academy ai-fluency-framework-foundationsAI Fluency: Framework and foundations, Claude Academy (course)