Loading only what the skill needs

The previous lesson ended with a release skill that ran from a new session in a clean clone. Suppose the team then had a release go wrong, and someone added a section to the skill on how to undo one. The section is useful, and it is needed in one release out of many. This lesson measures what that section costs on every other release, moves it into a separate file the skill points to, and measures again.

The measuring is done by a short Python script that only reads files, and the course checks its output on every build. It is in the course repository at site/examples/customizing-agents/progressive-disclosure/, next to two versions of the release skill. Run the commands in this lesson from that directory. In the cd line, replace <course-repo> with the path to your copy of the course repository.

cd <course-repo>/site/examples/customizing-agents/progressive-disclosure

What the agent reads, and when

An agent that supports the open skill format doesn’t read a whole skill at once. It reads it in stages, and the specification names three [1]. At startup the agent loads the name and description of every skill it can see. When a task matches a description, it loads that skill’s whole SKILL.md. Other files in the skill’s directory are loaded only when a step in SKILL.md sends the agent to them. Anthropic’s overview of skills describes the same three stages for Claude [2]. Designing a skill for this is called progressive disclosure: each stage holds only what the agent needs at that point [3].

Each stage has its own cost. The name and description are in context on every turn of every session, whether the skill runs or not. Both documents put that stage at about 100 tokens per skill. The specification recommends keeping the body under 5000 tokens, and Anthropic’s overview gives the same figure for the body [1] [2]. In Claude Code, the body stays in the conversation on the turns after the skill runs [4]. As How a language model works showed, the model reads everything so far each time it predicts a token, so a body that’s still in context is read again on each of those turns. A file the agent never opens doesn’t cost any tokens [2].

Checkpoint · sort

When does the agent load it?

Unplaced (drag an item to a bucket, or select it and then click the bucket)

The description in the front matter of SKILL.mdThe numbered release steps in the body of SKILL.mdThe line in the body that says when to read rollback.mdThe undo steps in rollback.md

At startup, for every skill

When a task matches the skill

Only when a step sends the agent to it

CheckHintSkip

Measure the skill as it stands

The directory grown/release/ holds the release skill from the previous lesson with the new section added. It is called “When a release goes wrong” and has five numbered steps. A release that nobody has pushed yet is undone with git tag -d and a reset. For a release that was pushed, the agent leaves the remote tag alone and makes a new fix release. The section is between the release steps and “Done when”. The agent reads it on every release.

The script measure.py prints one line per stage. For the first stage it counts the name and description lines of the front matter. The second stage is the whole SKILL.md, and each other Markdown file in the directory gets a line of its own. Then it prints two totals, one for a run that reads only SKILL.md and one for a run that reads every file.

The counts are estimates. The script uses the same rule as the counter in What the model can see, about four characters per token. A word of up to six characters counts as one token and a longer word as one token per four characters. Each punctuation mark counts as a token of its own. Markdown has a lot of punctuation, and the rule gives a higher count for a skill than for plain prose of the same length. The exact count for a model comes from the vendor’s own tokenizer. The rule is the same for both versions of the skill, so the difference between them is what matters here.

Before you run it, guess the second line. The file is about two pages of Markdown, and the counter from the concepts lesson gave 115 tokens for half a page of notes.

Example · run it

Measure the skill as it stands

Run this, and compare with the output below.

python3 measure.py grown/release

Output

grown/release
  always loaded, name and description: 39
  when the skill runs, SKILL.md: 1256
  when a step reads it: no other files
  total, SKILL.md read: 1295
  total, every file read: 1295

Output verified in CI from site/examples/customizing-agents/progressive-disclosure/before.py.

The first line is small, and it should be. A description of two short sentences is far below the figure of about 100 tokens. The second line is the one to read. A release loads all 1256 tokens of SKILL.md, and the rollback steps are part of them even when the release goes right.

Move the rollback into a reference file

The fix is to keep the common path in SKILL.md and move the rare case into its own file. The specification suggests keeping SKILL.md short and moving detailed material into separate files, referenced by a relative path and one level deep from SKILL.md [1]. Anthropic’s guide explains the one level. The agent may read only part of a file it reaches through another reference file [5].

Make a copy of the grown skill to work on. The first line deletes ~/skill-copy, so the commands also work on a second attempt. The copy’s directory is named release, because a skill’s directory name must equal its name.

rm -rf ~/skill-copy
mkdir ~/skill-copy
cp -R grown/release ~/skill-copy/release

In ~/skill-copy/release/, create rollback.md with the heading and the opening line below. Move the text of the “When a release goes wrong” section into it, from its first sentence to the end of step 5. Change “Use this section only when” to “Use these steps only when”, because the text is now a file of its own.

# Undo a release of the notes package

The release skill sends you here. Use these steps only when the user
asks you to undo a release, or when a check after the tag shows that the
release is wrong. Undo nothing else.

1. Run `git describe --tags`, `git log --format='%h %s' -n 3` and ...

Then leave the heading in SKILL.md and replace the text under it with a pointer that says when to read the file:

## When a release goes wrong

If the user asks you to undo a release, or a check after the tag shows
that the release is wrong, read `rollback.md` in this skill's directory
and follow it. Don't undo anything before you have read it.

The pointer names the situation, and that is what makes it work. A line that says only “see rollback.md” gives the agent no way to tell whether this run needs the file, so it may read it every time, or never. Anthropic’s guide says to watch for this when you test a skill: an agent that never opens a bundled file may be missing a clear enough signal in SKILL.md [5]. The agent reads the pointer only in a run where the skill has loaded. An undo asked for in a new session loads the skill only if its description names that request.

The course repository has the result in split/release/. Measure it, and then measure your copy with python3 measure.py ~/skill-copy/release. If you moved the text as shown, your copy gives the same numbers. Its first line is your path.

Checkpoint · choice

Which line drops?

Before you run the script on the split skill, predict which line drops compared with the grown skill.

The line for the name and description, since the skill now has less text in it.The line for SKILL.md, since the undo steps are no longer in that file.The total when every file is read, since the undo steps are now read only on demand.

CheckHintSkip

Example · run it

Measure the split skill

Run this, and compare with the output below.

python3 measure.py split/release

Output

split/release
  always loaded, name and description: 39
  when the skill runs, SKILL.md: 813
  when a step reads it, rollback.md: 516
  total, SKILL.md read: 852
  total, every file read: 1368

Output verified in CI from site/examples/customizing-agents/progressive-disclosure/after.py.

The description didn’t change, so the first line is still 39. A release that goes right now loads 852 tokens where it loaded 1295. A rollback reads both files and loads 1368, a little more than before, because the agent reads the pointer and the new file’s heading as well as the steps. The split makes the common run cheaper, and the rare run pays a small amount extra.

Checkpoint · choice

Where does the table go?

Where should the error-code table go?

In the description, so the agent knows about every error code before it picks the skill.In the body of SKILL.md, under a heading that says to use it only when a deploy fails.In errors.md, with a line in SKILL.md that says to read it when the deploy tool prints an error code.In errors.md, reached from a troubleshooting.md file that SKILL.md points to for every kind of failure.

CheckHintSkip

Pitfall: The skill that grew one section at a time

A team’s release skill starts at forty lines. Over six months people add a note on urgent fixes and a table of past incidents. Later come the rollback steps and a paragraph on the old release tool. Each change is small enough that nobody objects in review. The body ends up three times its first size, and a release loads all of it, although most runs need only the first forty lines. Someone also adds trigger words for each new case to the description, which every session pays for. Each change looked cheap, and nobody measured the sum. Measure the skill each time you add to it, and keep in the body only what every run needs.

Exercise

If you followed the walkthrough, your ~/skill-copy/release/ is the exercise. Go to the result below. Otherwise make the copy with the commands in the last section. Measure it with measure.py, move the “When a release goes wrong” section into rollback.md with a pointer that says when to read it, and measure again. The script only reads files, and you change only the copy. Afterwards you can check any skill you write for a rare case that every run pays for.

For the stretch, these commands put your split skill into the clean clone from the last lesson and commit it, then make one change for the release to list. The skill edits files and makes commits and tags, so run it only in that clone, which has no remote.

rm -rf ~/notes-clean/.claude/skills/release
cp -R ~/skill-copy/release ~/notes-clean/.claude/skills/release
cd ~/notes-clean
git add .claude/skills/release
git commit -q -m "Split the release skill"
echo 'Run `python3 notes.py list` to see your notes.' >> README.md
git commit -q -a -m "Document the list command"

A good result: the first line didn’t change, the second dropped from 1256 to 813 if you moved the text as shown, and rollback.md appears as its own line. The pointer in SKILL.md names the requests that need the file. Read the body that is left. Is there another part that only some runs need?

Stretch: After the commands above, type /release 0.4.1 in a new session in ~/notes-clean, and check in the transcript whether the agent opened rollback.md.

Recap

1. An agent loads a skill in stages: the name and description of every skill at startup, the whole SKILL.md when a task matches it, and other files only when a step points to them [1] [2].
2. The description is paid for on every turn of every session, and the body on every run of the skill. In Claude Code the body stays in the conversation for the turns after it runs [4].
3. Keep the common path in SKILL.md. Move a rare case or a long reference into a file one level from SKILL.md, and leave a pointer that names when to read it [5].
4. Measure the skill with the same rule before and after a change. An estimate is enough to compare two versions. The exact count for a model comes from the vendor’s tokenizer.
5. Skills grow a little at a time, so measure after each addition.

You can now

• Structures a skill so the agent loads only what it needs

Mark lesson finishedI know this, skip it

Habit

The next time you add a section to a skill, measure the skill before and after the change. Then ask whether every run needs the new section.

References

1. Anthropic. Agent Skills specification. agentskills.io. Reference. Agent Skills spec
2. Anthropic. Agent Skills. Claude Platform documentation. Reference. Claude docs agent-skills-overview
3. Addy Osmani, Ivar Soares Urdalen, Leo Simons. Agent skills: skills versus tools, the spec, progressive disclosure. Agent Engineer Course. Course. AEC-17
4. Anthropic. Extend Claude with skills. Claude Code documentation. Reference. Claude Code skills
5. Anthropic. Skill authoring best practices. Claude Platform documentation. Reference. Claude docs best-practices