Show one example
In the previous lesson you stated the format in words and in numbers, and that worked because “three bullets, each under 15 words” is a format words can state. Some formats resist that. In this lesson we take one of those, a team’s house style for meeting notes, and watch a paragraph of rules fail to produce it. Then we replace the rules with one worked example and run both prompts.
You need any chat assistant you have access to. Every response shown here is illustrative. The author wrote it to show the pattern, and the component labels it that way. Your model gives a different text, and each response here is one possible answer. The comparison that matters is between your own two runs.
A format that words describe badly
Section titled “A format that words describe badly”Here is a note in the house style of a fictional team. They have written notes this way for years and nobody has ever written the rules down.
Thu 4 Sep, office move (MK, JV, SR)DECIDED Move on Sat 27 Sep, not Fri 26. -> MK books movers by TueDECIDED Old desks go to the charity shop. -> SROPEN Who packs the server room? -> JV asks ITNOTED Parking permits arrive after the move; no action.Try to describe this format in words. The first line has the weekday and
date, a short subject, and the attendees’ initials in brackets. Each later
line starts with a tag in capitals, padded so the items line up. An item is
a fragment rather than a sentence. An arrow at the end gives the owner’s
initials, with a deadline where there is one, and a NOTED line has no
arrow. The note has no heading, and it has no bullets or blank lines. You
already have a paragraph, and it still leaves out the column widths and
what counts as a fragment. It doesn’t say whether the deadline is a day or
a date.
Here are the rough notes from a second meeting, as someone typed them during the call. We want them in the house style.
18 sept, meeting about the Varrowmere delivery, present: Mia K, Jonas V, Sara R- varrowmere moved order 4471 from tue 14 oct to fri 17 oct morning- we said yes to friday, mia will confirm to them before thursday- the tuesday unloading shift can be cancelled, sara does it- not clear who takes the two replacement chairs when they arrive, jonas will ask facilities- reason is a port strike, nothing for us to doSend the rules paragraph with the rough notes, and see how close the model gets.
Rewrite the rough meeting notes between the triple quotes in our house style. The first line has the weekday and the date, a short subject, and the attendees’ initials in brackets. Lines after the first start with a tag in capitals: DECIDED, OPEN or NOTED. The tags are padded so the items start in the same column. Each item is a short fragment. At the end of a DECIDED or OPEN line, an arrow and the owner’s initials, with a deadline if there is one. NOTED lines have no arrow. Use no heading, bullets or blank lines.
"""[the rough notes, pasted in full]"""**Meeting notes, 18 September: Varrowmere delivery**Attendees: MK, JV, SR
- DECIDED: Accept the new delivery date of Friday 17 October (morning). -> MK to confirm with Varrowmere before Thursday.- DECIDED: Cancel the Tuesday unloading shift. -> SR- OPEN: Who will receive the two replacement chairs on arrival? -> JV to ask Facilities.- NOTED: The delay is caused by a port strike. No action required for us.Every rule is in the prompt and the answer breaks most of them. There is a heading, there are bullets, the tags have a colon, the items are full sentences, and the weekday is missing. In the lesson on how models work you saw that the model predicts the most likely next token given the whole prompt. The phrase “meeting notes” in your prompt pulls toward the millions of meeting notes in its training text, and those have headings and bullets. Your paragraph of rules is a weak pull against that, because each rule is one sentence, and the model has to turn a sentence into a pattern before it can follow it.
One worked pair instead
Section titled “One worked pair instead”Now give the model the pattern directly. Delete the rules and put the office-move note in their place, together with the rough notes it was written from. The model then has an input and an output side by side, and your instruction becomes “do the same for this one”.
Rewrite rough meeting notes in our house style. Here is one example of rough notes and the finished note.
Rough notes:"""4 sep, office move planning, Mia K, Jonas V, Sara R- decided to move saturday 27th instead of friday 26th, mia books the movers, needs to happen by tuesday- old desks to the charity shop, sara arranges- nobody knows yet who packs the server room, jonas will ask IT- parking permits only arrive after the move, nothing we can do"""
Finished note:"""Thu 4 Sep, office move (MK, JV, SR)DECIDED Move on Sat 27 Sep, not Fri 26. -> MK books movers by TueDECIDED Old desks go to the charity shop. -> SROPEN Who packs the server room? -> JV asks ITNOTED Parking permits arrive after the move; no action."""
Now write the finished note for these rough notes:"""[the Varrowmere rough notes, pasted in full]"""Thu 18 Sep, Varrowmere delivery (MK, JV, SR)DECIDED Accept Fri 17 Oct morning slot. -> MK confirms by ThuDECIDED Cancel Tue unloading shift. -> SROPEN Who receives the two replacement chairs? -> JV asks facilitiesNOTED Delay caused by port strike; no action.Put this next to the first run. The rules paragraph is gone and the format
is right: the first line, the padded tags, the fragments, the arrows, the
missing arrow on NOTED. You never wrote “pad the tags” or “write
fragments”. The model saw one input turned into one output and
continued the pattern, the same way it continues any text. The pair you
gave it is a few-shot example, a term made
common by the paper that showed how much a large model picks up from a
few worked examples in the prompt, with no training involved
[1]. “Shot” means one example, so a prompt with one pair is
one-shot and a prompt with none is zero-shot, which is what the rules
paragraph was.
The example also did something the rules didn’t try to. The rough notes
in the example say “nothing we can do”, and the finished note turns that
into no action with no arrow. Your second input had “nothing for us to
do”, and the model treated it the same way. One example covered an edge
case, and the rules paragraph would have needed another sentence for it.
When to show an example instead of describing
Section titled “When to show an example instead of describing”Show an example when the thing you want is easier to recognize than to state. That happens for a format, for a tone and for an edge case.
A format with layout in it: columns, padding, where the line breaks go, what a short item looks like next to a long one. Words for layout are long and still ambiguous, and one example settles every one of those questions at once.
A tone. “Friendly but brief” means something different to every reader. Two sentences written in the tone you want mean one thing, and the model matches them more closely than it matches an adjective.
An edge case. What to do with a missing owner, an item that is neither a decision nor a question, a line that says nothing needs doing. A rule for each case makes the prompt long, and the model may still not connect the rule to the case when it meets it. An example that contains the case shows the answer in place [2].
Describe in words when the requirement is one number or one word. “Under 100 words”, “in Dutch”, “three bullets” need no example, and an example would add tokens and a topic the model might copy along with the format. The lesson on structured output shows the case where both are used at once: the field names described and one filled-in example of the format.
How many examples? The paper that made the term common found the biggest jump from no example to one, with smaller gains from each example after that [1]. This course’s advice for a chat prompt follows from that. One example is often enough for a format, as you just saw. Add a second or a third when one example can’t show the variety you need, for example a short note next to a long one, or a meeting with no decisions. Pick each one to show something the others don’t, because every example costs tokens that the model has to read on each call. The order of a prompt engineering course for developers follows the same path this lesson does, a clear instruction first and examples where the instruction isn’t enough [3].
Describe it or show it?
Section titled “Describe it or show it?”The lesson showed a paragraph of formatting rules fail to produce a team's house style for meeting notes, and one worked input and output pair succeed.
For each request, decide whether you would describe the format in words or show one worked example.
Ask of each request: can you state the requirement in one number or one word, or would a reader recognize it faster than you could write it out?
The output repeats the slip
Section titled “The output repeats the slip”A prompt contains one worked example of a team's meeting-note format. The example has a slip on one line, and the model's output repeats the slip.
You gave the model one example of the house style, and one OPEN line in
it has no arrow by mistake. The output leaves the arrow off its OPEN line
too. What is the right fix?
Which part of the prompt is the model copying most closely, and what is the smallest change that makes that part say what you want?
Show it or say it?
Section titled “Show it or say it?”The lesson shows that one worked input and output pair gets a format that a paragraph of rules couldn't, and says when to describe a requirement in words instead.
Does the requirement have a layout, a tone or an edge case in it, or is it one number or word?
The example had a slip
Section titled “The example had a slip”A prompt shows the model one worked example of a team's format for a list of open tickets. In the example, one ticket number was typed without its leading zero.
The output drops the leading zero from some ticket numbers, the same way your one example does by mistake. What do you do?
What does the model copy most closely: the rules or the example?
Exercise
Pick a format from your own work that you know by sight, such as the way your team writes a status update or a shift handover. Find one finished piece in that format and the rough material it was made from, or write both in five minutes. In a fresh chat, describe the format in a paragraph of rules and ask the model to convert a second piece of rough material. In another fresh chat, replace the paragraph with the finished piece and its rough material, fenced and labeled as in this lesson, and ask for the same conversion. Fifteen minutes is enough. Doing this once on a format you know by heart shows you which of your requirements were never in your rules.
A good result is a second output that a colleague would accept as written by the team, and a first output that you can mark up with the rules it broke. Which rule did the model break in round one that you had not thought to write down at all?
Stretch: Add a deliberate mistake to your example, such as a wrong date format or a missing field, and run again. Did the output copy it? Then fix the example and confirm the copy is gone.
Recap
- A paragraph of rules is one sentence per requirement, and the model has to turn each sentence into a pattern before it can follow it. The default from its training text often pulls harder.
- One worked input and output pair shows the pattern directly, and the model continues it, layout and edge cases included. The pair is a few-shot example, a term the paper on the effect made common [1].
- Show an example for a format with layout in it, a tone, or an edge case. Describe in words when the requirement is a number or a word.
- The biggest gain is from no example to one [1]. One example is often enough for a format, and this course’s advice is two or three chosen to show variety before a page of rules. Each one costs tokens.
- The model often copies the example more closely than it follows a rule, so it copies a mistake in the example too. Read every example as a spec, and fix the example rather than writing a rule against it.
You can now
- Turns a vague request into instruction, context, example and format
References
Section titled “References”- Tom B. Brown, Benjamin Mann, Nick Ryder and 28 others. Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Paper.
Brown 2020 - Anthropic. Building with the Claude API. Claude Academy. Course.
Academy building-with-the-claude-api - DeepLearning.AI. ChatGPT Prompt Engineering for Developers. DeepLearning.AI. Course.
DLAI-1