Orchestration in code or by a model

In this lesson one customer message goes through three fake agents in two arrangements. First a program runs the agents in a fixed order and passes each output to the next. Then a fake manager model decides which agent runs when, and it calls the other agents as tools. After that we name the arrangements, and we break the manager version at each boundary to see what reaches the customer.

The customer is from the small shop in Writing the plan before taking the steps, and the message is about the kettle from order 1042. The agents and the manager are fake models written by hand. The runs show how each arrangement passes work along, and they don’t show how well a real model does it. The complete program is site/examples/building-agents/orchestrating-agents/agent.py in the repository. Copy the orchestrating-agents folder, and each step below is python3 agent.py <step>.

A pipeline written in code

The message is “My kettle from order 1042 stopped working. Please send the replacement to my office address this time.” The agents are lookup, writer and checker. lookup reads the message, looks up the order and writes a brief of one line. writer writes the reply from the brief. checker reads the reply and approves it or rejects it. Here are the first two, each a fake model call with text in and text out.

def lookup(message: str) -> str:
    """Reads the customer's message, looks up the order and writes a one-line brief."""
    order_id = order_id_in(message)
    order = ORDERS[order_id]
    wish = "refund" if "money back" in message else "replacement"
    days = order["days_since_delivery"]
    return f"order {order_id}, {order['item']}, delivered {days} days ago, wants a {wish}"


def writer(brief: str) -> str:
    """Writes the reply to the customer from the brief."""
    parts = brief.split(", ")
    if len(parts) == 4 and parts[0].startswith("order "):
        order, item, wish = parts[0], parts[1], parts[3].replace("wants a ", "")
        return (
            f"Dear customer, we are sorry that your {item} broke. "
            f"A {wish} for {order} is on its way."
        )
    return "Dear customer, we are sorry about the problem. We will get back to you soon."

The order data says the kettle was delivered 12 days ago. The program decides who runs when, and it always runs lookup, then writer, then checker.

def pipeline(message: str) -> list:
    messages = [("customer", "lookup", message)]
    brief = lookup(message)
    messages.append(("lookup", "writer", brief))
    draft = writer(brief)
    messages.append(("writer", "checker", draft))
    verdict = checker(draft)
    messages.append(("checker", "program", verdict))
    if verdict == "approved":
        messages.append(("program", "customer", draft))
    return messages

This is orchestration in code, which the Agent Engineer Course calls deterministic. The flow is written down in advance, so it runs the same way every time and you can step through it like other code. Deciding what comes next costs no model call [1]. The messages from one agent to the next are the handoffs. The handoffs step prints those two, each as sender -> receiver: 'text'.

Checkpoint · predict

Predict the handoffs

What are the two handoff messages? Predict the output, then run it.

python3 agent.py handoffs

What is printed?

Output verified in CI from site/examples/building-agents/orchestrating-agents/handoffs.py.

CheckHintSkip

The brief has no place for the office address, so the address is lost at the first boundary. The checker approves the reply, because it sees only the draft, and the draft reads well. Here is the whole run.

Example · run it

The whole pipeline

Run this, and compare what you see with the output below.

python3 agent.py pipeline

Output

customer -> lookup: 'My kettle from order 1042 stopped working. Please send the replacement to my office address this time.'
lookup -> writer: 'order 1042, kettle, delivered 12 days ago, wants a replacement'
writer -> checker: 'Dear customer, we are sorry that your kettle broke. A replacement for order 1042 is on its way.'
checker -> program: 'approved'
program -> customer: 'Dear customer, we are sorry that your kettle broke. A replacement for order 1042 is on its way.'

Output verified in CI from site/examples/building-agents/orchestrating-agents/pipeline.py.

A fact can be dropped at each boundary between agents. The Agent Engineer Course lists this among the weaknesses of a pipeline, because a mistake early in the line travels forward to the agents after it [2]. One agent that read the message and wrote the reply would have had the address in its context. The pipeline made three model calls and two handoffs, and it lost a fact on the way. Add a second agent when it can do something the first can’t, such as work with a focused prompt, its own small set of tools or other permissions [2]. The next lesson counts what each arrangement costs in tokens.

A manager that delegates to workers

Now a model decides who runs when. A manager model gets the customer’s message, and it can call two workers, lookup and writer, the way an agent calls tools. The loop around the manager runs each worker call the manager asks for, and gives the result back to the manager. This arrangement has no checker, and the manager’s last reply goes to the customer. Here is the fake manager.

def manager(history: list) -> dict:
    last = history[-1]
    if last["from"] == "customer":
        return {"calls": [{"worker": "lookup", "task": last["text"]}]}
    if last["from"] == "lookup":
        # Like a real model, it passes on what it got, an error message included.
        brief = last["text"] if last["ok"] else last["error"]
        return {"calls": [{"worker": "writer", "task": brief}]}
    return {"answer": last["text"]}

Each reply either asks for worker calls or gives the answer. The loop runs a worker the way a harness runs a tool, and turns its result into a dictionary with ok and either text or error.

for _ in range(max_turns):
    reply = model(history)
    problem = check_reply(reply)
    if problem:
        stop(problem)
        return
    if "answer" in reply or not reply.get("calls"):
        show("manager", "customer", reply.get("answer", ""))
        return
    for call in reply["calls"]:
        show("manager", call["worker"], call["task"])
        result = run_worker(workers, call["worker"], call["task"])
        show(call["worker"], "manager", result)
        history.append({"from": call["worker"], **result})
        problem = check_result(call["worker"], result)
        if problem:
            stop(problem)
            return

check_reply and check_result are the checks at the two boundaries, and for now neither of them stops a run. The manager step runs the same message.

Example · run it

The manager and its workers

Run this, and compare what you see with the output below.

python3 agent.py manager

Output

manager -> lookup: 'My kettle from order 1042 stopped working. Please send the replacement to my office address this time.'
lookup -> manager: {'ok': True, 'text': 'order 1042, kettle, delivered 12 days ago, wants a replacement'}
manager -> writer: 'order 1042, kettle, delivered 12 days ago, wants a replacement'
writer -> manager: {'ok': True, 'text': 'Dear customer, we are sorry that your kettle broke. A replacement for order 1042 is on its way.'}
manager -> customer: 'Dear customer, we are sorry that your kettle broke. A replacement for order 1042 is on its way.'

Output verified in CI from site/examples/building-agents/orchestrating-agents/manager.py.

The customer gets the same reply, and the address is still lost. This time a model decided who ran when. Each line that starts with manager -> comes from a model call that picked the next agent, and a real manager could pick another order, skip a worker or call one twice. So orchestration by a model can handle a task whose steps nobody wrote down in advance. In exchange, the same input can take a different path, which is harder to debug, and each decision is one more model call [1].

Claude Code uses this arrangement for its subagents. A subagent that gets a task from the main conversation begins with a new context window of its own. The messages exchanged so far aren’t in it. When it finishes, the main conversation gets back a short summary instead of everything the subagent read [3]. The Academy course on subagents gives this as a reason to delegate: the main conversation holds less text and doesn’t drift from its topic [4]. It also means that a fact from the conversation that the task text doesn’t mention is out of the subagent’s reach. The task text is the handoff, as the brief was in the pipeline.

Checkpoint · choice

Who decides the order?

How should the team coordinate the three agents?

A manager model that decides for each claim which agent runs next, so it can adapt to odd claims.A program that runs the three agents in a fixed order and passes each output to the next one.Three peer agents in one shared conversation that agree among themselves who goes next.A manager model with the three agents as tools and a list of the order in its prompt.

CheckHintSkip

Sequential, hierarchical and collaborative

The pipeline and the manager are two of the common arrangements. The Agent Engineer Course describes them with two more, collaborative and competitive [2], and this lesson covers the collaborative one.

• Sequential. The agents form a pipeline, and each one works on the output of the one before it. The pipeline above is sequential.
• Hierarchical. A manager splits the task, hands the parts to worker agents and puts their results together. The manager above is hierarchical, with two workers.
• Collaborative. Peer agents share a workspace or a conversation, with none of them in charge. They build on each other’s work until they agree on a result.

The coordination gets harder in that order. The course rates it low for a pipeline, medium for a manager and its workers, and high for a group of peers, and it lists endless discussion among the risks of the last [2]. The arrangements also combine, and a step of a pipeline can itself be a manager with workers. The course advises starting with one agent, then a pipeline if the task has stages, and a system of many agents only when one agent has shown that it can’t do the task [1].

Checkpoint · sort

Name the arrangement

Unplaced (drag an item to a bucket, or select it and then click the bucket)

A program sends each scanned letter to an extractor, then a classifier, then a filing agentA lead agent splits a research question and hands each part to a search agentThree reviewer agents post findings in one shared thread and reply to each other until they agreeA coding agent starts one helper for the tests and one for the docs, and merges what they returnAn editing agent always gets the text of the drafting agent and passes its edit to a layout agentA planning agent and a critic agent each edit a shared document whenever they see a need, and neither assigns work to the other

Sequential

Hierarchical

Collaborative

CheckHintSkip

A fault at each boundary

The manager version has two boundaries: the manager’s reply to the loop, and a worker’s result to the manager. We break each one. At the first, the manager’s reply asks for no worker and doesn’t answer either, which is an empty plan. At the second, the order service times out, and the lookup worker returns an error. Here is the empty plan.

Example · run it

An empty plan from the manager

Run this, and compare what you see with the output below.

python3 agent.py empty_plan

Output

manager -> customer: ''

Output verified in CI from site/examples/building-agents/orchestrating-agents/empty_plan.py.

The loop takes a reply with no calls as the end of the run, and the customer gets an empty message. Here is the timeout.

Example · run it

A tool error from a worker

Run this, and compare what you see with the output below.

python3 agent.py tool_error

Output

manager -> lookup: 'My kettle from order 1042 stopped working. Please send the replacement to my office address this time.'
lookup -> manager: {'ok': False, 'error': 'order service timed out'}
manager -> writer: 'order service timed out'
writer -> manager: {'ok': True, 'text': 'Dear customer, we are sorry about the problem. We will get back to you soon.'}
manager -> customer: 'Dear customer, we are sorry about the problem. We will get back to you soon.'

Output verified in CI from site/examples/building-agents/orchestrating-agents/tool_error.py.

The manager passed the error text to the writer as if it were a brief, and the customer got a polite reply that promises nothing. Nobody in the shop knows that the lookup failed. Each run ends with a message to the customer, and neither message is true to what happened. The course asks for a check of each agent’s output before the next agent gets it, so that a bad result doesn’t travel on [1]. The guarded run passes stop_on_error as the check for worker results.

def stop_on_error(worker: str, result: dict) -> Optional[str]:
    """Stops the run when a worker returns an error."""
    if not result["ok"]:
        return f"{worker} failed: {result['error']}"
    return None

When a check returns a text, stop prints it after stopped: and then prints nothing was sent to the customer. The check runs in the loop, after the worker’s result is shown and before the manager’s next turn.

Checkpoint · predict

Predict the report

The lookup worker times out again, and this time the loop checks each worker result. What does the run print? Predict the output, then run it.

python3 agent.py guarded

What is printed?

Output verified in CI from site/examples/building-agents/orchestrating-agents/guarded.py.

CheckHintSkip

The run stops at the boundary where the fault happened, and the report names the worker and the error. A person or a retry policy can act on that. The writer never ran, and the customer got nothing false.

Pitfall: A manager model for steps that never change

A team builds a manager model to run a workflow of four steps that always run in the same order, because a manager with workers looked more capable. The manager makes a model call before each step to choose the step the team already knew, and one day it skips one. The trace shows a choice at every turn, and none of them was needed. Rule: when the steps and their order are known in advance, write the order in code, and keep a model for the parts where the next step depends on what the last one found [1].

The guarded loop has a check for worker results, and its check for the manager’s reply, stop_on_empty_plan, lets every reply through. The exercise step runs both faults with the guarded loop.

Example · run it

Both faults before the exercise

Run this, and compare what you see with the output below.

python3 agent.py exercise

Output

fault: empty plan from the manager
manager -> customer: ''
fault: tool error from the lookup worker
manager -> lookup: 'My kettle from order 1042 stopped working. Please send the replacement to my office address this time.'
lookup -> manager: {'ok': False, 'error': 'order service timed out'}
stopped: lookup failed: order service timed out
nothing was sent to the customer

Output verified in CI from site/examples/building-agents/orchestrating-agents/exercise.py.

Exercise

In your copy of the orchestrating-agents folder, write stop_on_empty_plan so that it returns a text when the manager’s reply has no worker calls and no answer, and None otherwise. Run python3 agent.py exercise. The output is the two traces, one per fault. Writing the check yourself shows that the boundary checks belong to the code around the model, whoever decides the order.

A good result: the empty plan trace ends with a stopped: line that says the manager returned an empty plan, then nothing was sent to the customer, and the tool error trace is as it was. Compare with python3 model_answer.py in the same folder. Which other boundary in this arrangement could pass a bad result on, and what would its check look for? The opening comment of model_answer.py gives one answer.

Stretch: Add a check for the writer's result as well: stop when the draft doesn't name the order the customer wrote about. Pass it as check_result in step_tool_error, run tool_error and see which boundary now catches the fault.

Recap

1. In orchestration by code, a program decides which agent runs when. It runs the same way every time and costs no model call to decide [1].
2. In orchestration by a model, a manager decides and calls the other agents as tools. It can handle steps nobody planned, and the same input can take another path [1].
3. A fact can be lost at each handoff. Add a second agent when it can do something the first can’t [2].
4. Sequential, hierarchical and collaborative arrangements get harder to coordinate in that order [2]. Check the result at each boundary in code, and stop with a report when it is wrong.

You can now

• Justifies the coordination cost of more than one agent
• Composes patterns and names the failure modes of the composition

Mark lesson finishedI know this, skip it

Habit

The next time you hand a task to a subagent or a second agent, read the task text it gets and check that it holds every fact the work depends on.

References

1. Addy Osmani, Ivar Soares Urdalen, Leo Simons. Orchestrators: code- versus model-driven, patterns, anti-patterns. Agent Engineer Course. Course. AEC-18
2. Addy Osmani, Ivar Soares Urdalen, Leo Simons. Multi-agent systems: architectures, roles, the orchestration tax. Agent Engineer Course. Course. AEC-07
3. Anthropic. Create custom subagents. Claude Code documentation. Reference. Claude Code subagents
4. Anthropic. Introduction to subagents. Claude Academy. Course. Academy introduction-to-subagents