Reasoning before each action

In this lesson the fake model writes one line of reasoning before each tool request, and the loop prints that line in the trace. We run one task with the model that writes the line and with one that doesn’t, and compare the traces. After that we look at a task where the line changes which tool the model asks for, and at what the line costs.

The loop is the one from Stopping the loop on purpose, with the same run and the same stop rules. Its show prints one more line, the thought, when a request has one. The complete program is site/examples/building-agents/react/agent.py in the repository. The tools belong to a small shop. get_order returns the item and the days since delivery, get_policy returns the text of a shop policy, and issue_refund refunds an order in full.

One line of reasoning before each call

The pattern has a name, ReAct, from the paper that tested it: the model alternates a short written reasoning step with an action, reads the result, and reasons again [1]. In the paper, the reasoning helps the model keep and update a plan, and handle surprises, and the actions fetch the facts that the reasoning needs [1]. The lesson on agentic design patterns in the Agent Engineer Course presents it as a loop of think, act, observe [2].

In the fixture, the reasoning is one more field in the reply. The fake model model_reasons writes a thought, then asks for the tool that follows from it. Here is its first round:

if len(done) == 0:
    if question.startswith("Refund"):
        thought = "A refund is allowed only inside the return window, so I check the order first."
    else:
        thought = "A return depends on when the order arrived, so I look up the order."
    return {"thought": thought, "tool": "get_order", "args": {"order_id": order_id}}

In the second round it reads the days since delivery from the result and asks for the return policy. After that it compares the two numbers and answers. show prints thought: and the text before the round line of every request that has one.

Example · run it

The trace with thoughts

Run this, and compare what you see with the output below.

show(run("Can I still return order 1042?", model=model_reasons))

Output

user: 'Can I still return order 1042?'
thought: A return depends on when the order arrived, so I look up the order.
round 1: get_order(order_id='1042') -> {'ok': True, 'item': 'kettle', 'days_since_delivery': 12}
thought: It arrived 12 days ago. I need the return window to compare.
round 2: get_policy(topic='returns') -> {'ok': True, 'text': 'Returns and refunds within 30 days of delivery.'}
stop: end_turn
answer: Yes. The kettle arrived 12 days ago, inside the 30-day window.

Output verified in CI from site/examples/building-agents/react/return_reasons.py.

Two rounds, each with the reason written before it. The final reply is an answer with no tool request, so run returns before a third round and the answer has no thought line in front of it.

The same run without the reasoning

model_acts is the kind of model the loop had before this lesson. It replies with a tool request only, and it picks the tool from the words of the question. Run it on the same task.

Example · run it

The plain loop on the same task

Run this, and compare what you see with the output below.

show(run("Can I still return order 1042?", model=model_acts))

Output

user: 'Can I still return order 1042?'
round 1: get_order(order_id='1042') -> {'ok': True, 'item': 'kettle', 'days_since_delivery': 12}
round 2: get_policy(topic='returns') -> {'ok': True, 'text': 'Returns and refunds within 30 days of delivery.'}
stop: end_turn
answer: Yes. The kettle arrived 12 days ago, inside the 30-day window.

Output verified in CI from site/examples/building-agents/react/return_acts.py.

The calls, their order, their results and the answer are all the same. The trace with thoughts adds one thing, a reason you can read for each call. On this task, a person who debugs the run learns why the agent looked up the order, and the agent does nothing different.

When the reasoning changes the call

The second task asks for a refund of order 1077, a lamp delivered 45 days ago. model_acts sees the word “Refund” and asks for the refund tool.

Example · run it

A refund without reasoning

Run this, and compare what you see with the output below.

show(run("Refund order 1077.", model=model_acts))

Output

user: 'Refund order 1077.'
round 1: issue_refund(order_id='1077') -> {'ok': True, 'refunded': '1077'}
stop: end_turn
answer: Done. Order 1077 is refunded.

Output verified in CI from site/examples/building-agents/react/refund_acts.py.

The refund went through for an order outside the 30-day window, and the model didn’t read the policy. model_reasons writes its first thought about what a refund depends on, and that thought leads to a different first call.

Example · run it

A refund with reasoning

Run this, and compare what you see with the output below.

show(run("Refund order 1077.", model=model_reasons))

Output

user: 'Refund order 1077.'
thought: A refund is allowed only inside the return window, so I check the order first.
round 1: get_order(order_id='1077') -> {'ok': True, 'item': 'lamp', 'days_since_delivery': 45}
thought: It arrived 45 days ago. I need the return window to compare.
round 2: get_policy(topic='returns') -> {'ok': True, 'text': 'Returns and refunds within 30 days of delivery.'}
stop: end_turn
answer: The lamp arrived 45 days ago, outside the 30-day window. I did not refund it.

Output verified in CI from site/examples/building-agents/react/refund_reasons.py.

Both fake models are written by hand to show the difference. The fixture proves nothing about real models. The paper measured it. Adding the reasoning steps did better than the same prompts with actions only, on question answering and fact checking, and on the two interactive tasks ReAct beat methods trained by imitation and reinforcement learning [1].

The line has a price. The thoughts stay in the message list, and the loop sends the whole list to the model again in every round. The cost step runs the return task with each model and prints the size of the message list after the last round.

Example · run it

The size of the message list

Run this, and compare what you see with the output below.

for name, model in (("model_acts", model_acts), ("model_reasons", model_reasons)):
    messages = run("Can I still return order 1042?", model=model)["messages"]
    print(f"{name}: {len(str(messages))} characters")

Output

model_acts: 466 characters
model_reasons: 623 characters

Output verified in CI from site/examples/building-agents/react/cost.py.

The calls and the answer are the same, and the list is longer by the two thoughts. A real model writes each line as output tokens, and the loop sends the line back as input in every later round of the run.

The loops in this course already alternate

The loops in this course already alternate a model decision with an action and its result: the model decides on an action, the loop runs it, and the model decides the next action from the result. The pattern adds the written reason between the result and the next decision. Some models write that reason on their own. On Claude, a model that supports interleaved thinking can write its reasoning after each tool result and before the next call. The API returns that reasoning as a summary, or as an empty field when the display setting is omitted, and it bills the full reasoning in both cases [3]. The choice you make as the builder is whether the reasoning appears in a form you can read and keep.

A task gains from the extra tokens when its next step depends on what an earlier result said. The agent then chooses between tools as it goes. The refund is such a task: the order decides whether the refund tool is right. When the calls are the same whatever the results say, as with one export per table every night, the line doesn’t change a call, and a fixed script or a plan fits the job better. People who review or debug the runs have a second reason to keep the line, because it shows them where the agent went wrong.

Checkpoint · choice

Which task gains from the reasoning line?

Which of these tasks gains the most from one line of reasoning before each tool call?

Rename 300 photos by the same date rule, with one tool call per photo.Read a customer's plan and payment history, then choose a refund, a credit or a handover to a person.Translate a product description into French, with no tools at all.Report the stock level of one item from one tool that is always the only call.

CheckHintSkip

Pitfall: Reading the thought as a record of what happened

A reviewer sees “I check the order first” in a trace and concludes that the agent checked the order. The thought is text the model wrote. In the fixture’s run, the loop runs reply["tool"] with reply["args"] and never reads reply["thought"], so a thought and a call can disagree and the loop runs the call. Research on written reasoning found that models leave out factors that changed their answer and write plausible reasons for answers they were pushed toward [4]. On Claude, the thinking you get back is a summary or an empty field, and no display setting returns the raw reasoning [3]. Rule: check what the agent did in the round lines, the calls and their results, and read the thought as the model’s explanation of them.

Exercise

Copy site/examples/building-agents/stopping-the-loop/agent.py from the repository. Add a thought field to the replies of its three fake models, model_never_answers, model_retries_forever and model_alternates, and change its show to print the thought before each round line, the way this lesson’s show does. Run the never_answers, retries and alternates steps, and put each trace next to the one the original file prints. Doing it once on a loop you already know shows how little the pattern changes in the code.

A good result: three pairs of traces with the same rounds and the same stop reasons, where the new trace of each pair has a thought line before every round. Give model_retries_forever the same thought in every round for that. Then put the round number in its thought and run retries again. Which rule ends the run now, and what in repeated did the thought change?

Stretch: Give model_alternates a thought that names the fix it tried last and the result it saw, and check whether a reader of that trace could spot the alternation before the step limit.

Recap

1. ReAct alternates a short reasoning step with an action and reads the result before it reasons again [1].
2. On a task with a fixed path, the reasoning line adds a readable reason for each call and leaves the calls the same.
3. When the next tool depends on an earlier result, the reasoning step can change which tool the model asks for, at the cost of one more line per step that the loop sends back in every round.
4. A tool-using loop already alternates decisions and actions, and on Claude a thinking model can reason between tool calls whether or not the loop shows it [3]. A shown thought can leave out what drove the choice [4], and the round lines are the record of what the loop ran.

You can now

• Picks a design pattern for a task and says why

Mark lesson finishedI know this, skip it

References

1. Shunyu Yao, Jeffrey Zhao, Dian Yu and 4 others. ReAct: Synergizing Reasoning and Acting in Language Models. International Conference on Learning Representations (ICLR 2023), arXiv preprint 2210.03629. Paper. Yao 2022
2. Addy Osmani, Ivar Soares Urdalen, Leo Simons. Agentic design patterns: ReAct, reflection, tool use, planning. Agent Engineer Course. Course. AEC-04
3. Anthropic. Thinking. Claude Platform documentation. Reference. Claude docs thinking
4. Miles Turpin, Julian Michael, Ethan Perez and 1 other. Language Models Don't Always Say What They Think. Advances in Neural Information Processing Systems 36 (NeurIPS 2023). Paper. Turpin 2023