Skip to content
AI Training

Chat, agent or automation

In this lesson we pick how to apply a model to a task. Chat, agent, and automation differ in how much the model does between one moment you look and the next, and the lesson gives you a way to pick the level for a task and to write down what would change the pick. You don’t run anything here. The tasks are supplied, and the work is reading them and deciding.

The concepts course gave you a test for telling an assistant from an agent, and the safety course showed where a person has to approve before an agent acts. This lesson puts the two together into one decision you make before the work starts: how much of it the model does on its own.

Chat, agent, and automation

Section titled “Chat, agent, and automation”

The same model can be used in three ways, and the difference is where you are while it works.

Chat. You type, the model answers, you read the answer and type again. Every step is yours, and the model does one step per turn. A difficult email is a chat task. You know what you want to say and the words won’t come. You paste the thread and ask for a first draft. You cut a paragraph and ask for a softer second sentence, and you send the version you end up with. Every turn is a decision about tone, and you are the one making it. Chat fits thinking and one-off drafts, and any other task where you would change your mind halfway through.

Agent. You write a brief, and the agent works through the steps with its tools. It checks in with you at the points the brief names. A folder of two hundred misnamed scans is an agent task. You say what the names should look like and what to do with a file it can’t read. You also say that it reports back before deleting anything. Then it works, and you read the report. The steps are known, but the path through them varies per file, and a person doesn’t need to see each one. An agent fits a bounded task with clear done-criteria, where you want to read the result once instead of steering every step.

Automation. An event starts the run while nobody is watching, the model does its part, and the result goes where it goes. Invoices renamed the moment they arrive in a mailbox are an automation task. The steps and the path are both fixed, the same thing happens every time, and the work is checked by sampling later, or by whoever opens the folder. Automation fits repetitive, low-stakes work that has been tested enough that nobody needs to watch it.

The pattern across the three is one variable moving. In chat the model does one step and waits. In an agent it does many steps and waits at the points you named. In automation it does the whole task and waits for nobody. The word for that variable is autonomy, and each level trades your attention for the model’s independence. A wrong pick either wastes your attention, when you steer step by step through work that has one right path, or removes your attention from the place where it was needed.

Checkpoint · sort

Which level fits each task?

Section titled “Which level fits each task?”

Autonomy rises and the checks rise with it

Section titled “Autonomy rises and the checks rise with it”

Moving up a level gives the model more steps between checks. The trade only holds when the checks on the other side grow with it. In chat, the check is you reading every answer, and you need nothing else. In an agent, you read the result once. The brief has to contain the checks you won’t be there for, such as what the agent must not do and where it stops to ask. In automation, you don’t read anything at the time, and the checks have to exist and be tested before the first run. A sample you look at each week is one. An error the system reports instead of guessing through is another, and so is a way to undo a batch. A step-up in autonomy without the matching step-up in checks is a task that runs unwatched and unchecked, and the failures that follow are the ones the safety course described: the mistake now has hands [1].

The level is also picked for the task’s worst case, and this is the part people get wrong. A support queue looks like an automation task. Most tickets are a password reset, a delivery question, or a refund under the limit, and a well-briefed agent answers those correctly all day. One ticket in fifty is different. It is a complaint that mentions a lawyer, or a customer who was told something wrong by a person last week. The forty-nine routine tickets make the case for automation, and the fiftieth decides the level. If a wrong answer on that one ticket is expensive, and the system can’t tell that ticket from the others with certainty, the queue stays at the agent level, with a person approving the result. Call that “agent with approval”: the agent drafts every reply and a person sends it, or the agent sends the routine ones and routes anything it isn’t sure about to a person. It is the agent level with the check that level needs, and the count of levels stays three. The level is set by the ticket you can’t afford to get wrong, and the routine tickets only decide how little attention that level should cost.

So the question for a task is never “what does this task need on most days?” It is “what does this task need on its worst day, and can the system tell that day from the others?” A task with a worst case that’s cheap to fix, or that the system reliably detects and hands over, can run at the level its routine case suggests. A task whose worst case is expensive and looks like the routine case from the outside is kept one level down.

Checkpoint · choice

Which level does the queue get?

Section titled “Which level does the queue get?”

Which level does the support queue get, and why?

The moving rule

Section titled “The moving rule”

A pick made this way has a reason, and a reason can be written down. The useful form is two sentences per task, one for what would move the task up a level and one for what would move it back down. “The invoice renaming goes to automation once a month of agent runs hasn’t needed a correction and the folder is backed up nightly. It comes back to agent if a run renames a file we can’t match to an invoice.” A rule like that makes the pick reviewable by someone who wasn’t there when it was made, and it makes the next change a decision that someone takes on purpose.

What moves a task up is evidence, and what moves it back is a new worst case. Evidence is a run of results without corrections, a test on the cases that used to fail, a backup or an undo that turns the worst case into a cheap one, or a check that finds the odd case and routes it to a person. A new worst case is a task that starts to receive a kind of input it didn’t before, or a wrong result that got more expensive because the folder is now shared with a client. A correction that nobody caught for a week is a new worst case too. So is a changed rule, for example a new customer contract that says a person answers every message. When the level is written with its moving rules, an improvement in the tool is one more piece of evidence and a change in the task is a reason to look again, and both are decisions someone made on purpose [2].

Writing the rule also stops the two silent moves. In the slide up, a task at the agent level with approval becomes automation because the person approving stopped reading and started clicking, and nobody decided anything. In the slide down, a task that could safely run unattended stays in chat because it always has, and a person spends an hour a week steering a task with one right path. A written rule names the evidence the first move needs and the worst case that would justify the second, and a task with neither stays where it is.

Checkpoint · scenario

Six months of approvals

Section titled “Six months of approvals”

The operations lead has approved the agent’s monthly report draft for six months, and admits that since about March the approval has been a click without a read. Nothing in the drafts has been wrong. What do you do?

Exercise

For each of the five tasks below, pick chat, agent or automation for the task’s worst case, and write one sentence for what would move the task up a level and one for what would move it back. Fifteen lines, and fifteen minutes is enough.

1. Answering the questions new colleagues ask in the team channel
   about where things are and how the tools work.
2. Turning the notes of a customer meeting into a follow-up email
   to the customer.
3. Sorting the receipts that arrive in the expenses mailbox into
   the month and the category they belong to.
4. Writing the first version of a project plan for something the
   team has never done before.
5. Checking each new supplier contract against the company's
   standard terms and listing the differences.

A good answer has the plan in chat, because the thinking is the task and you would change course halfway. It has no up move while the plan is being thought out, and once the plan exists, filling in the sections from notes is a different task that an agent can do. It has no down move, because chat is the lowest level. The follow-up email is at agent with you sending it, because the meeting is in your memory and the customer receives the result. It moves up to automation only when the notes are complete enough to draft from without you and a wrong email costs little, and it moves back to chat when a meeting was difficult and the wording is the work. The receipts are at automation with a monthly sample check and a way to move a wrong file. They moved up on a month of agent runs without a correction, and they come back to agent if a receipt turns out to be a contract or an invoice above a set amount. The team-channel questions are at agent. The agent answers from the team’s own pages and says when it can’t find an answer. It moves up to automation once a month of answers has been checked and none was about access or a password, and it moves back to chat if a wrong answer about access or a password shows up. The contract check is at agent with a person reading each list, because a missed clause is the tenth time. It moves up for the standard contracts only, once a test on last year’s contracts has found every difference a lawyer found, and it moves back to agent for all contracts if a supplier sends a form the test never saw. Where your answer differs, the sentence that matters is the one about the worst case, so check that one first. Which of your five picks would slide up on its own if nobody wrote the rule?

Stretch: Take a task from your own work that a model already does or could do, write its level with the two moving sentences, and check the pick against the task's worst day of the last year.

Recap

  1. Chat, agent and automation are three ways to apply one model, and the difference is how many steps it does before you look.
  2. Autonomy rises across the three. The checks rise with it, from your reading in chat to the brief’s stop points for an agent to a tested sample and undo for automation [1].
  3. The level is picked for the task’s worst case. A worst case that is expensive and can’t be told from the routine case keeps the task one level down.
  4. Each pick gets a moving rule: the evidence that moves the task up, and the new worst case that moves it back [2].
  5. The written rule stops the two silent moves, the approval that becomes a click and the task that stays in chat because it always has.

You can now

  • Picks chat, agent or automation for a task and says why

References

Section titled “References”
  1. Anthropic. Building effective human-agent teams. Claude Academy. Course. Academy building-effective-human-agent-teams
  2. Anthropic. AI Fluency for builders. Claude Academy. Course. Academy ai-fluency-for-builders