Skip to content
AI Training

The prompt before the conversation

In the last lesson every part of the prompt went into one message: the instruction, the material, the reader, and the format. Some of those parts are the same for every message you send. The reader is always your team, the answers should always be short, and there are topics the assistant should never touch. In this lesson we look at the system prompt, the text that holds those parts and that the model reads before the conversation starts. We compare two system prompts and their answers, and then we see who writes one and where a rule has to go when it should hold every time.

The text the model reads first

Section titled “The text the model reads first”

Everything the model works from is text in its context. The conversation you see is part of that text. Before it, there is a block that whoever set the assistant up wrote, and in most products you do not see it. That block is the role and system prompt, in most products called the system prompt. It says who the model is in this conversation and how it should sound. It also lists the rules that hold for every answer and what the model must never do [1] [2].

A short one covers four things. The role says what the assistant is for, such as the help desk of a bicycle rental shop. Tone is how it speaks: plain, formal, brief, patient. Standing rules are the parts of a prompt from the last lesson that would otherwise repeat in every message, such as the audience and the format. The never-do list names the topics or actions that are off limits, and often what to do instead.

Because the system prompt comes before every turn, the model treats it as the frame for the whole conversation. It is still tokens in the context, like everything else. A rule in it competes with everything said afterwards, which the lesson on how models work called instruction dilution. A long conversation can drift away from it, and a rule in a system prompt is a strong default and never a guarantee. Within one short conversation, though, the system prompt decides more about the answers than one user message does [1].

Same question, different system prompt

Section titled “Same question, different system prompt”

Below are fictional system prompts for two assistants at the same small company, each followed by the same user message. The responses are illustrative: the author wrote them to show the pattern, and each one is one possible answer among many. Your model gives a different text, and the comparison to make is between the two pairs.

Prompt (illustrative, not a recorded transcript)

System prompt: You are the assistant for the internal help desk of a small company. Answer in plain English, in at most three sentences. Do not give steps the user has not asked for. If a question is about pay, contracts or leave, do not answer it. Say that the human resources (HR) team handles it and give the HR mailbox, hr@example.com.

User: My laptop won’t connect to the office Wi-Fi. What do I do?

Response

Turn Wi-Fi off and on again, and pick the network named “Office” rather than “Office-Guest”. If it still fails, open a ticket with the help desk and include your laptop’s asset tag. They usually respond within an hour.

Prompt (illustrative, not a recorded transcript)

System prompt: You are the onboarding guide for employees in their first month at the company. Give the steps in order and say why each one matters, because the user does not know the office yet. When a request is unclear, ask one question back before you answer. Otherwise, end with one question that checks where the user got stuck, and name one page from the onboarding handbook they could read next.

User: My laptop won’t connect to the office Wi-Fi. What do I do?

Response

Welcome, and don’t worry, this happens to most people in their first week. First, check which network your laptop is trying to join. The office has two, one for staff and one for guests, and the staff one needs your company login. Second, turn Wi-Fi off and on, which makes the laptop ask for the network again. Third, if you see a login window, use the same username and password as for your email. Which of these steps did you get to before it failed? That tells us where to look next. The handbook page “Getting online on your first day” has screenshots of each step.

The user typed the same sentence. The first answer is three sentences of plain instructions, and the second is a walkthrough with a question back and a reading tip. Nothing in the user message asked for either style. The difference comes from the system prompt.

Now imagine the same two assistants get a second message: “Can I carry over unused vacation days to next year?” The first one has a rule for this. It says the question is for HR and gives the mailbox, and it does not try to answer. The second one has no such rule, so it gives a general explanation of how vacation carry-over works and asks which contract the user has. One of those answers is what the company wanted, and the difference is one sentence in the system prompt.

Who writes it

Section titled “Who writes it”

In a chat product, the vendor writes the system prompt, and in most products you do not see it. It names the assistant and sets its tone, and it often gives the model facts such as the current date [3]. When two products built on the same model behave differently, much of the difference is in those prompts. Many products let you add standing text of your own: account-wide instructions on a settings page, or a project with instructions that apply to every chat in it [4] [5]. That text applies to every conversation you start there.

In an agent or a custom application, the developer writes it. The programming interface of every major model has a field for it, separate from the user messages, and the application fills that field on every request [2]. That is where recurring behavior gets fixed. When an assistant keeps answering in the wrong language, or keeps writing five paragraphs where three bullets were wanted, the fix goes into the system prompt once, and every later conversation starts from it [1]. The lesson on iterating on a prompt shows how to make those changes one at a time.

Checkpoint · sort

System prompt or this message?

Section titled “System prompt or this message?”

Place each line where it belongs: in the system prompt the team writes once, or in the message an employee types for one task.

Checkpoint · choice

Where does the fix go?

Section titled “Where does the fix go?”

A team built a support assistant on a model’s programming interface. In every conversation it offers discounts it may not offer, and someone corrects it in the chat each time. Where does the fix go?

Exercise

Copy the two system prompts from this lesson into a text file, one above the other. Under them, write three user messages of your own, and choose ones that differ from the two in this lesson: one that is unclear, such as “it doesn’t work”, one that neither prompt has a rule for, such as a farewell message for a colleague, and one that only the never-do list covers, such as a question about a contract. For each message, write one line per system prompt that predicts how the answer differs: its length, whether it asks a question back, and whether it answers at all. Six short lines is enough, and it takes about five minutes. Predicting before you run is what turns a system prompt from a block of text into a set of rules you can reason about.

If you have a chat product that lets you set custom instructions, or a project with its own instructions, paste one system prompt there, send the three messages, and compare with your predictions. A good result is one where each difference you predicted traces back to one sentence in one of the two system prompts. Which prediction was wrong, and which sentence did you misjudge?

Stretch: Write a third system prompt that combines the brevity of the first with the question-back rule of the second, and predict where the two rules conflict.

Recap

  1. The system prompt is text the model reads before the conversation. It sets a role, a tone, the standing rules and a never-do list, and it frames every later turn [2].
  2. Two assistants on the same model with different system prompts give different answers to the same message, in length and in what they refuse.
  3. In a chat product the vendor writes it, and you can add standing instructions of your own. In an agent or a custom application the developer writes it, and that is where recurring behavior gets fixed [1].
  4. A rule typed into the chat lasts one conversation. A rule that should hold every time goes where the conversation starts.
  5. The instruction files that coding agents read at the start of a session, AGENTS.md or CLAUDE.md, do the same job for a project: standing rules, loaded before your first message. The lesson on project instructions shows how to write one.

You can now

  • Turns a vague request into instruction, context, example and format

References

Section titled “References”
  1. Addy Osmani, Ivar Soares Urdalen, Leo Simons. How agents think: tokens and context, reasoning strategies, model choice, system prompts. Agent Engineer Course. Course. AEC-02
  2. Anthropic. Building with the Claude API. Claude Academy. Course. Academy building-with-the-claude-api
  3. Anthropic. System prompts. Claude Platform documentation. Reference. Claude docs system-prompts
  4. Anthropic. Understanding Claude's personalization features. Claude Help Center. Reference. Claude support personalization
  5. Anthropic. How can I create and manage projects?. Claude Help Center. Reference. Claude support projects