Stopping the loop on purpose

In this lesson we take the loop from When a tool fails and give it every reason it needs to stop. The loop so far ends when the model answers, when the step budget runs out, or after two tool errors in a row. Then we swap in three fake models that never answer and watch which rule ends each run. The step limit catches the last one late, and the exercise adds the check that catches it early.

Every example runs against a fixture with a fake model, so you can predict the output and check yourself. The complete program is site/examples/building-agents/stopping-the-loop/agent.py in the repository. Its run function is the loop from the previous lesson with the harness checks gathered in one function, check, that returns the name of the rule that ended the run. The fixture prints one line per tool round, round 1: read_file(path='notes/1.md') -> {...}, with the tool result in repr, then the stop reason, and an answer line only when there is one. The last example prints its lines while the loop runs, and that output has no user: line.

The rules that end a run

A harness combines stop conditions, and each one exists because the others can miss. The model answering without a tool request is the natural end, and the Claude API reports it as the stop reason end_turn, against tool_use for a reply that wants a tool run [1]. A turn limit catches a model that keeps finding one more thing to do [2]. A token or cost budget catches a run whose turns are few but large. A passing check ends a run whose goal the harness can test itself, such as the tests going green, whatever the model thinks. A human watching the run stops it when they see it going wrong. The API has its own limits on its side: max_tokens caps one reply, and pause_turn says that a server-side tool loop reached its iteration limit and the client may send the reply back to continue [1]. Your loop knows about its own budgets, so it adds the rest.

This lesson’s harness has the model’s answer, the step limit max_steps, the error budget max_errors from the previous lesson, a character budget in place of a token budget because the fake model has no tokens, and the KeyboardInterrupt a person sends with Ctrl-C. After each round, check reads the message list and names the first rule that matches.

def check(messages, spent: int, max_errors: int, budget: int) -> Optional[str]:
    if errors_in_a_row(messages) >= max_errors:
        return "too_many_errors"
    if repeated(messages):
        return "repeated_state"
    if spent > budget:
        return "budget_spent"
    return None

The loop calls it after it has appended the round, and it wraps the whole run in a try so that Ctrl-C becomes a stop reason too.

def run(question, model, tools=TOOLS, max_steps=4, max_errors=2, budget=5000, progress=None):
    messages = [{"role": "user", "content": question}]
    spent = 0
    try:
        for step in range(1, max_steps + 1):
            reply = model(messages)
            if "answer" in reply:
                return outcome("end_turn", reply["answer"], messages)
            tool = tools[reply["tool"]]
            result = tool["fn"](**reply["args"])
            messages.append({"role": "assistant", "content": reply})
            messages.append({"role": "tool", "content": result, "is_error": is_error(result)})
            if progress is not None:
                progress(step, reply, result)
            spent += len(str(messages))
            reason = check(messages, spent, max_errors, budget)
            if reason is not None:
                return outcome(reason, None, messages)
    except KeyboardInterrupt:
        return outcome("interrupted", None, messages)
    return outcome("max_steps", None, messages)

repeated and progress are explained where the lesson adds them. Until then, repeated returns False for every run on this page except the one that is built to trigger it. The character budget never ends a run on this page either: four rounds of the first fixture put spent at 2062, well under 5000, and the other runs stay under it too.

A model that never answers

The first fake model always finds one more file to read. It counts the tool rounds in the message list and asks for the next note.

def read_file(path: str) -> dict:
    return {"ok": True, "text": f"contents of {path}"}

def model_never_answers(messages):
    n = len(rounds(messages)) + 1
    return {"tool": "read_file", "args": {"path": f"notes/{n}.md"}}

Every call succeeds, so the error count never moves, and every request names a new path. max_steps is 4 in this fixture.

Checkpoint · predict

Predict which rule fires

What does this print?

show(run("Summarize the notes.", model=model_never_answers))

What is printed?

Output verified in CI from site/examples/building-agents/stopping-the-loop/never_answers.py.

CheckHintSkip

Four rounds, then max_steps. Each round on its own looks like normal work, and a model that reads one file too many looks the same as one that is nearly done. The step limit ends the run without needing to understand it, and that is what makes it the rule every loop has.

Pitfall: The model is the only judge of done

The first version of most loops is while True with a break when the reply has no tool request [2]. That loop has one stop rule, and the model owns it. With the model above, show never prints, because run never returns, and the terminal is empty while the bill grows. Rule: every loop has at least one stop rule that the model cannot influence, and the harness reports which rule ended the run.

Two ways a loop runs in circles

The second and third models work on a failing test. The tool applies a change and runs the tests, and in this fixture the test never passes. The tool itself succeeds every time, so the error budget from the previous lesson sees nothing, because a test failure is a result, and a tool error is a tool that could not do its job.

def apply_fix(change: str) -> dict:
    return {"ok": True, "tests": "1 failed: test_total"}

def model_retries_forever(messages):
    return {"tool": "apply_fix", "args": {"change": "round the total"}}

Repeating the same action after the same result is the first loop failure mode the source lesson names for planning agents [3], and the defense is to remember what was tried. rounds gives the request and result of every tool round, and repeated compares the last round with the one before it.

def rounds(messages) -> list:
    return [
        (messages[i]["content"], messages[i + 1]["content"]) for i in range(1, len(messages) - 1, 2)
    ]

def repeated(messages) -> bool:
    done = rounds(messages)
    return len(done) >= 2 and done[-1] == done[-2]

Example · run it

The same fix twice

Run this, and compare what you see with the output below.

show(run("Make the failing test pass.", model=model_retries_forever))

Output

user: 'Make the failing test pass.'
round 1: apply_fix(change='round the total') -> {'ok': True, 'tests': '1 failed: test_total'}
round 2: apply_fix(change='round the total') -> {'ok': True, 'tests': '1 failed: test_total'}
stop: repeated_state

Output verified in CI from site/examples/building-agents/stopping-the-loop/retries.py.

The run ends after round 2, and the stop reason says why. The third model is smarter by one step. It reads its own last request and tries the other fix, so no two rounds in a row are the same.

def model_alternates(messages):
    done = rounds(messages)
    if done and done[-1][0]["args"]["change"] == "round the total":
        return {"tool": "apply_fix", "args": {"change": "truncate the total"}}
    return {"tool": "apply_fix", "args": {"change": "round the total"}}

Example · run it

Two fixes, in turn

Run this, and compare what you see with the output below.

show(run("Make the failing test pass.", model=model_alternates))

Output

user: 'Make the failing test pass.'
round 1: apply_fix(change='round the total') -> {'ok': True, 'tests': '1 failed: test_total'}
round 2: apply_fix(change='truncate the total') -> {'ok': True, 'tests': '1 failed: test_total'}
round 3: apply_fix(change='round the total') -> {'ok': True, 'tests': '1 failed: test_total'}
round 4: apply_fix(change='truncate the total') -> {'ok': True, 'tests': '1 failed: test_total'}
stop: max_steps

Output verified in CI from site/examples/building-agents/stopping-the-loop/alternates.py.

Round 3 is round 1 again, and repeated misses it because it only looks one round back. The step limit catches this run one round later than a wider check would. With max_steps at forty, the character budget ends the same run with budget_spent after round 7, because the message list grows with every round and spent adds its whole length each time. The exercise at the end of the lesson adds the check that catches the alternation.

Progress a person can see

show prints the trace after run returns. A run that has no working stop rule never returns, so the person at the terminal sees nothing until they lose patience. The progress parameter of run is called after every round with the step number, the request and the result, and the fixture’s print_round writes the same line show would, while the loop is running.

def print_round(step: int, request: dict, result: dict) -> None:
    print(line(step, request, result), flush=True)

A person who reads those lines can press Ctrl-C. The try in run turns the key press into the stop reason interrupted, and the caller still gets the message list. In the fixture, watch_and_stop does what you would do at the terminal: it prints the line and raises KeyboardInterrupt after round 2.

Example · run it

A person stops the run

Run this, and compare what you see with the output below.

result = run("Summarize the notes.", model=model_never_answers, progress=watch_and_stop)
print(f"stop: {result['stop']}")

Output

round 1: read_file(path='notes/1.md') -> {'ok': True, 'text': 'contents of notes/1.md'}
round 2: read_file(path='notes/2.md') -> {'ok': True, 'text': 'contents of notes/2.md'}
stop: interrupted

Output verified in CI from site/examples/building-agents/stopping-the-loop/interrupted.py.

The lines appeared one at a time, and the run ended two rounds before the step limit, because someone could see it going in circles. Every stop reason on this page is a name the caller can branch on, and interrupted is the one that says a person made the call.

Checkpoint · order

Order one iteration

Put the steps of one iteration of the loop in order, from the model call to the decision to go round again.

1. Send the message list to the model↑↓
2. Return end_turn with the answer when the reply holds no tool request↑↓
3. Run the requested tool with the arguments from the reply↑↓
4. Append the request and the result to the message list↑↓
5. Run the harness checks over the new message list: errors in a row, a repeated round, the budget↑↓
6. Return the stop reason the checks named, or start the next step if the step limit allows↑↓

CheckHintSkip

Exercise

Copy agent.py from the repository. Change repeated so it compares the last round with every earlier round, and run show(run("Make the failing test pass.", model=model_alternates)) again. Then run the retries and never_answers steps and check that their output is the same as before. Doing this once shows you how small the difference is between a check that catches one runaway form and one that catches two, and how easy it is to test.

A good result: the alternating model stops after round 3 with repeated_state, one round earlier than before, and the other two runs print what they printed in the lesson. What would a model have to do to get past this version of the check, and which rule on this page would catch it then?

Stretch: Add a passing check as a stop rule: give run a done parameter, a function of the message list, and stop with the reason done when it returns True. Make a fake model whose second fix makes the tests pass, and check that the run ends with done before the model gets to say so.

Recap

1. A harness combines stop rules because each one misses something: the model’s answer, a step limit, a token or cost budget, a passing check and a person’s Ctrl-C [2].
2. The API names its own stop reasons, end_turn, tool_use, max_tokens and pause_turn, and your loop adds the ones only it can know [1].
3. A loop runs in circles by repeating a round or by alternating between two. A check that compares with the previous round ends the first, and one that compares with every earlier round ends both [3].
4. Progress a person can see while the loop runs is a stop rule too, and the harness reports which rule ended every run.

You can now

• Implements the loop with error handling and a stop condition

Mark lesson finishedI know this, skip it

References

1. Anthropic. Stop reasons and fallback. Claude Platform documentation. Reference. Claude docs handling-stop-reasons
2. Addy Osmani, Ivar Soares Urdalen, Leo Simons. Building your first agent: the loop from scratch, then with an SDK. Agent Engineer Course. Course. AEC-13
3. Addy Osmani, Ivar Soares Urdalen, Leo Simons. Planning and reasoning: the loop, plan-then-execute, hierarchy. Agent Engineer Course. Course. AEC-06