Skip to content
AI Training

Several tool calls in one turn

In this lesson the model asks for three things at once. The question “Compare the weather in Amsterdam, Lisbon and Oslo” needs three lookups that have nothing to do with each other, and a loop that takes one call per turn spends a model round on each. We change the loop so a reply holds a list of calls, and the loop runs the whole list before it returns every result in one message. Then we add a call that needs one of those results and see why it waits for the next round.

Every example runs against a fixture with a fake model, so you can predict the output and check yourself. The complete program is site/examples/building-agents/parallel-tool-calls/agent.py in the repository. Its run function is the loop from When a tool fails with one change: a model reply holds a list under calls, each call has an id, and the tool message holds a list of results that name the id they answer. The execute parameter of run decides how the list runs, and the fake models read the message list to see which results they already have. The fixture prints one line per message, then the round count, the stop reason, and the answer. A round is one call to the model.

One call per turn

Section titled “One call per turn”

The first fake model asks for one city at a time. It reads the weather results it has and asks for the first city that’s missing. Once it has all three, it answers. A real model behaves this way when the request sets disable_parallel_tool_use to true, and the last lesson’s loop only knew how to handle one call per reply anyway.

def model_one_at_a_time(messages):
    known = weather_so_far(messages)
    missing = [city for city in CITIES if city not in known]
    if missing:
        return {"calls": [weather_call(len(known) + 1, missing[0])]}
    return {"answer": summary(known)}

weather_call(1, "Amsterdam") builds {"id": "call_1", "tool": "get_weather", "args": {"city": "Amsterdam"}}. The loop counts its rounds and runs each reply’s calls in the order they appear, with in_order:

def call_tool(call: dict) -> dict:
    tool = TOOLS[call["tool"]]
    return {"id": call["id"], "content": tool["fn"](**call["args"])}


def in_order(calls: list) -> list:
    return [call_tool(call) for call in calls]


def run(question, model, execute=in_order, max_steps=5):
    messages = [{"role": "user", "content": question}]
    rounds = 0
    for _ in range(max_steps):
        rounds += 1
        reply = model(messages)
        if "answer" in reply:
            return outcome("end_turn", reply["answer"], rounds, messages)
        results = execute(reply["calls"])
        messages.append({"role": "assistant", "content": reply})
        messages.append({"role": "tool", "content": results})
    return outcome("max_steps", None, rounds, messages)

The show function prints the user line as the question in repr, the same as the last lesson, an assistant line as call_1 get_weather(city='Amsterdam') and a tool line as call_1 '14°C, rain'. If a message has two or more calls or results, the line lists them all, separated by commas. The weather table has Amsterdam: 14°C, rain, Lisbon: 27°C, sun and Oslo: 6°C, snow, and summary joins them in that order with a period after each.

Checkpoint · predict

Predict the rounds, one call per turn

Section titled “Predict the rounds, one call per turn”

What does this print?

show(run(QUESTION, model=model_one_at_a_time))

Output verified in CI from site/examples/building-agents/parallel-tool-calls/one_at_a_time.py.

The three lookups took four rounds. Each round is a full trip to the model with the whole message list, and those trips cost most of the time and the tokens. If each lookup took a second and each model round took two, the run would take about eleven seconds, and eight of them are spent waiting for the model.

All three in one reply

Section titled “All three in one reply”

The second fake model asks for every city in its first reply. The same loop handles that, because execute already takes a list. The list can run in a different way, though. together hands the calls to a thread pool, and map returns the results in the order of the calls, whichever one finishes first.

from concurrent.futures import ThreadPoolExecutor


def model_all_at_once(messages):
    known = weather_so_far(messages)
    if not known:
        return {"calls": [weather_call(number, city) for number, city in enumerate(CITIES, 1)]}
    return {"answer": summary(known)}


def together(calls: list) -> list:
    with ThreadPoolExecutor() as pool:
        return list(pool.map(call_tool, calls))

The loop appends one assistant message with the three calls and one tool message with the three results.

Checkpoint · predict

Predict the rounds, three calls in one reply

Section titled “Predict the rounds, three calls in one reply”

What does this print?

show(run(QUESTION, model=model_all_at_once, execute=together))

Output verified in CI from site/examples/building-agents/parallel-tool-calls/together.py.

The same answer came back in two rounds instead of four. With the timings from the last section the run takes about five seconds. The three lookups overlap, so they cost one second together, and the model is asked twice. The vendor page says independent reads, such as three files the model wants to see, are usually safe to run together [1]. A model that asks for them in one reply saves the round trips in between [2].

The Claude API works the same way [1]. A reply with stop_reason tool_use can hold several tool_use blocks, each with its own id. Your loop answers with one user message that holds one tool_result block per tool_use block, each naming its tool_use_id, and any text in that message comes after the results [3]. Whether those calls ran one after another or in a thread pool is invisible in that message, so in_order and together both produce a valid reply. When a model stops asking for several tools at once, the docs name one result per user message as the most common cause: a history in that format teaches the model to stop asking for several at once [1]. To turn the batching off, set disable_parallel_tool_use: true inside the tool_choice object. With the default tool_choice type auto the model then makes at most one tool call per reply, and the run looks like the first section.

A call that needs a result

Section titled “A call that needs a result”

The third fake model wants to add packing advice for Amsterdam. The suggest_clothing tool takes a weather report as its argument, so the model can’t ask for it in the first reply: the argument is a result it doesn’t have yet. It asks for the three lookups, then for the advice, then answers.

def model_with_follow_up(messages):
    known = weather_so_far(messages)
    if not known:
        return {"calls": [weather_call(number, city) for number, city in enumerate(CITIES, 1)]}
    requests, results = results_by_call(messages)
    advice = [text for call_id, text in results.items() if requests[call_id]["tool"] == "suggest_clothing"]
    if not advice:
        call = {"id": "call_4", "tool": "suggest_clothing", "args": {"weather": known["Amsterdam"]}}
        return {"calls": [call]}
    return {"answer": f"{summary(known)} Pack {advice[0]} for Amsterdam."}

suggest_clothing returns "a raincoat" for a report that mentions rain.

Checkpoint · predict

Predict the rounds with a dependent call

Section titled “Predict the rounds with a dependent call”

What does this print?

show(run(QUESTION, model=model_with_follow_up, execute=together))

Output verified in CI from site/examples/building-agents/parallel-tool-calls/follow_up.py.

The four calls took three rounds. The batch saved two rounds on the lookups, and the dependent call cost one round of its own, because a round is the only place where a result can flow into an argument. The rule for what goes in one reply is the data flow: calls whose arguments are all known now can share a reply, and a call whose argument is another call’s result waits for the reply after it.

A model doesn’t always get this right, and a dependent call can arrive in the same batch as the call it needs. The docs give two cases. For a call you choose not to run, return a tool_result with is_error set to true and a short note. For a call that ran and failed because the call it needed hadn’t finished, return the error with is_error set to true, and the docs say the model makes the call again in the next turn. They also suggest a line in the system prompt that asks the model to batch only calls that don’t depend on each other [1]. The exercise below builds that case.

Side effects under concurrency

Section titled “Side effects under concurrency”

A weather lookup is safe to run at any time and in any order, because it doesn’t change anything. Tools that write are different. Two appends to the same file end up in either order when they run in two threads, and a write followed by a read of the same record can return the old value if the read runs first. The choice of in_order or together is a property of the tools in the batch [1]. A harness that knows which tools only read can use both strategies in one round: the read-only calls go to the thread pool, and the writes wait their turn. The reply still holds one result per call, whichever way each call ran.

Checkpoint · choice

Which pair can run together?

Section titled “Which pair can run together?”

The model asked for two tool calls in one reply. Which pair is safe to run at the same time?

Exercise

Copy agent.py from the repository. Change model_with_follow_up so its first reply holds four calls, the three lookups and a suggest_clothing call whose weather argument is the string "call_1", the way a model does when it guesses that it can refer to a result it doesn’t have. Run show(run(QUESTION, model=model_with_follow_up, execute=together)) and look at what suggest_clothing returns for that string. Then change call_tool so a call whose argument is another call’s id isn’t run and returns {"id": ..., "content": "not run: depends on call_1", "is_error": True} instead, and make the fake model treat that result as missing advice so it asks again with the real report. Run again and compare both message lists and round counts with the lesson’s follow-up run. Doing this once shows you what the harness can and can’t do about a dependent call: it can refuse to run it, and only the next round can supply the argument.

A good result: the first run answers with advice for the string call_1 in two rounds, which is wrong and fast. The second run shows the error result for call_4 in the first tool message, then a second reply with the real weather argument, and three rounds, the same count as the lesson’s follow-up run. Your fake model asks again by construction. A real model may, and for a call you didn’t run the docs promise no retry, so the harness can’t count on it. What would the harness need to know to tell a dependent call from an independent one without a rule about ids in arguments?

Stretch: Add a third strategy to the fixture, next to in_order and together: run every get_weather call in the thread pool and every other call one after another, in the same round. Check that the follow-up run's output stays the same.

Recap

  1. A model reply can hold several tool calls at once. The loop runs the list and returns one result per call, all in the next message, each under the id of the call it answers [1].
  2. Three independent lookups cost two rounds in a batch and four one at a time. The batch saves model round trips, and running the calls in a thread pool saves the wait for the tools too [2].
  3. A call whose argument is another call’s result waits for the next round. Data flow decides what can share a reply.
  4. Running a batch at the same time is safe for calls that only read. For calls that write, run them in order, and keep the ids on every result either way.

You can now

  • Implements the loop with error handling and a stop condition

References

Section titled “References”
  1. Anthropic. Parallel tool use. Claude Platform documentation. Reference. Claude docs parallel-tool-use
  2. Addy Osmani, Ivar Soares Urdalen, Leo Simons. Tools, giving agents hands: function calling, schema design, the N x M problem. Agent Engineer Course. Course. AEC-03
  3. Anthropic. Handle tool calls. Claude Platform documentation. Reference. Claude docs handle-tool-calls