Professional, lesson 6

Prompt layers and caching

Stable layers first, then cache the prefix

A timestamp sits on the first line of the Lumen Supply system prompt, so the prefix changes on every call and the cache never hits. The 7,100-token prefix is billed in full on each call, and every team's tenant text is mixed into it. This lesson orders the layers so the second call reads the prefix from the cache. The prompt has three stable layers and one volatile turn. The policy layer is the shop's rules, shared by every team. The tenant layer is one team's tone, and the guardrail layer holds the limits on every reply. The turn is the question and the time it was sent.

I doWatch the failure and its fix.

One prefix, many calls

The model reads all the layers, the API caches the stable ones, and your code decides their order. A cache hit needs the start of the prompt to be identical, byte for byte, up to a marker you place.

Three things go wrong, in this order. A timestamp at the top changes the prefix on every call. There is no cache marker, so nothing is reused even when the prefix is stable. And tenant text is mixed into the policy, so one edit changes the start of the prompt.

The second call reads from the cache

Two calls ask different questions with the same prefix. The first call writes 7,113 tokens to the cache and reads none. The second reads all 7,113 back and sends only 33 fresh tokens.

Cached tokens still fill the context window. A cache lowers the price of the prefix, not its size, so the prefix, the history and the reply must still fit together.

Open a card to see the usage fields behind each call. A cache write costs 1.25 times a fresh input token, and a read costs a tenth of one. Ten thousand calls like the second cost $9.89 on Haiku. The prices are this lab's figures.

buildlayers: policy, tenant, guardrail, turn
Show the result
```
{
  "tenant": "Acme"
}
```

Call 1wrote 9,779 tokens, read 0 tokens, fresh 39 tokens

Show the result

{
  "sent at": "2026-06-01 09:00:00",
  "ask": "Can I return a monitor after 20 days?",
  "cache written": 9779,
  "cache read": 0,
  "fresh input": 39,
  "output tokens": 120,
  "cost of 10,000 calls like this": "$12.86"
}

Call 2wrote 0 tokens, read 9,779 tokens, fresh 37 tokens

Show the result

{
  "sent at": "2026-06-01 09:01:37",
  "ask": "How long is the warranty on the dock?",
  "cache written": 0,
  "cache read": 9779,
  "fresh input": 37,
  "output tokens": 120,
  "cost of 10,000 calls like this": "$1.61"
}

HIT: the second call read 9,779 tokens from the cache, and 10,000 calls like it cost $1.61.

The layer order, written as code

build takes the three layers and the turn, and returns the system prompt and the messages. Stable text goes first, the turn goes last, and one cache marker follows the guardrail.

The marker cannot sit inside text that changes. A conversation's earlier messages change on every turn, so they go in the messages after the marker, never in the system layers. The prefix must also reach a minimum size before the API will cache it, which is 4,096 tokens for this model. Below that, a marker is ignored without an error.

starter/layers.py

1"""EDIT THIS FILE. It has two planted defects.21. build(): the time is the first line of the system prompt, so the3 prefix changes on every call and the cache never hits.42. build(): the layers are one block with no cache marker, so nothing is5 cached even when the prefix does not change.63. SKILL.md, in this same folder, is graded by the checks too: give it a7 description that says when to use it, and keep its body to stable rules.89Run `./workshop check prompt-layers-caching` until it is green, then10`./workshop watch prompt-layers-caching`.11"""1213from kit.api import Json141516def build(policy: str, tenant: str, guard: str, turn: Json) -> Json:17 """Return the system prompt and the messages for one turn.18 `turn` holds the customer's `text` and the time it was sent, `at`.19 """20 system = f"Time now: {turn['at']}\n{tenant}\n{policy}\n{guard}"2122 return {23 "system": system,24 "messages": [{"role": "user", "content": turn["text"]}],25 }

Does a changing timestamp at the top break the cache?

The time of each request is useful for audits. It seems harmless on the first line, because the rules that follow it never change.

Predict: Does a changing timestamp at the top of the prompt break the cache?

ANo. Only the last lines matter.
BYes. The cached prefix must match exactly.

Show the answer

B. A cache hit needs an identical prefix up to the marker. Anything that changes at the top invalidates all of it.

A timestamp at the top breaks the cache

This is a real Haiku 5.5 run, recorded once and replayed here. The starting build puts the time on the first line and sets no marker. Both calls send about 7,150 fresh tokens, and the cache writes and reads nothing.

At that rate, 10,000 calls cost $73.92. The prefix is the same rules every time, yet the API bills it in full on every call.

buildlayers: time, tenant, policy, guardrail, turn
Show the result
```
{
  "tenant": "Acme"
}
```

Call 1wrote 0 tokens, read 0 tokens, fresh 9,819 tokens

Show the result

{
  "sent at": "2026-06-01 09:00:00",
  "ask": "Can I return a monitor after 20 days?",
  "cache written": 0,
  "cache read": 0,
  "fresh input": 9819,
  "output tokens": 120,
  "cost of 10,000 calls like this": "$10.42"
}

Call 2wrote 0 tokens, read 0 tokens, fresh 9,817 tokens

Show the result

{
  "sent at": "2026-06-01 09:01:37",
  "ask": "How long is the warranty on the dock?",
  "cache written": 0,
  "cache read": 0,
  "fresh input": 9817,
  "output tokens": 120,
  "cost of 10,000 calls like this": "$10.42"
}

MISSED: the second call read 0 tokens from the cache, and 10,000 calls like it cost $10.42.

Policy, tenant, guardrail, then the turn

The time moves to the end, inside the turn. The three layers become separate blocks in order, and one marker follows the guardrail. Each tenant's text stays in its own layer, so an edit to it changes that layer and leaves the policy block unchanged.

With one marker, each team's cached prefix includes its own tenant text, so the teams do not share a cache.

The first call now pays a write of 7,113 tokens. Every later call reads them for a tenth of the price, so 10,000 calls cost $9.89 and not $73.92.

layers.pygreen lines are what the fix adds

1"""The solution: stable layers first, one marker, the turn last."""23from kit.api import Json456def build(policy: str, tenant: str, guard: str, turn: Json) -> Json:7 """Return the system prompt and the messages for one turn.8 The layers run policy, tenant, guardrail. One cache marker follows the9 guardrail, so everything before it is the cached prefix. The time rides10 with the turn, after the marker, where a change cannot break the cache.11 """12 system = [13 {"type": "text", "text": policy},14 {"type": "text", "text": tenant},15 {16 "type": "text",17 "text": guard,18 "cache_control": {"type": "ephemeral"},19 },20 ]21 asked = f"{turn['text']}\n\nSent at {turn['at']}."2223 return {24 "system": system,25 "messages": [{"role": "user", "content": asked}],26 }

We doChange one setting at a time and predict.

The same input, two layer orders

Switch between the two orders with the same tenant. The policy text, the question and the model are the same in both runs.

Choose Stable first. The note "This is where the runs split." appears and Call 1 is outlined. It is the first call that wrote the prefix to the cache, and without the fix it wrote nothing.

Only the order and the marker changed. The second call reads 7,113 tokens from the cache with the fix and none without it.

This part is interactive in the lab: run ./workshop watch prompt-layers-caching.

Try it: order and tenant

Change one setting at a time. Every combination is a recorded run, so this works with no internet.

The time-first order misses for every tenant. Choose "Acme, marker added" with the time first to see the trap in a marker alone. Both calls write 7,132 tokens, and neither reads, so 10,000 calls cost $91.75. That is more than the $73.92 with no marker, because a write costs more than a fresh token.

This part is interactive in the lab: run ./workshop watch prompt-layers-caching.

The stable layer as a Skill

A prompt that a whole team pastes drifts, because each copy is edited on its own. A Skill is one file the team loads, so the stable rules live in one place. Its body holds only text that does not change, which is the rule of the cached prefix: no time and no tenant name.

Its description says when to use it. A session loads the Skill by that description, so a vague one means the Skill is skipped.

The Messages API calls above do not load a Skill, so the Skill is a separate file. The checks grade SKILL.md in the same work folder, and Enforce in managed settings and hooks ships a skill to every team.

SKILL.mdgreen lines are what the fix adds

1---2name: lumen-reply-style3description: Use when writing a Lumen Supply customer reply. Holds the guardrails every team shares.4---56Answer in at most three sentences.7Never quote a price that is not in the rules, and never promise a delivery date.8Never ask for a card number or a password.9If the customer is angry, acknowledge it in one clause, then answer.10If the question is not about a Lumen order, say you can only help with Lumen orders.

You doFix the starting code, then test yourself.

You do: order the layers and mark the cache

Copy the starting files and open work/prompt-layers-caching/layers.py and SKILL.md. build() puts the time on the first line and sends the layers as one block with no cache marker.

Terminal

./workshop start prompt-layers-caching
./workshop check prompt-layers-caching
./workshop watch prompt-layers-caching

The checks want policy, tenant and guardrail in that order, one marker after the guardrail and the time with the turn. The Skill needs a description that says when to use it, and a body of stable rules.

Check your understanding

Answer each question, then read why every option is right or wrong. The wrong options are the mistakes engineers make on the job.

1. A support agent sends a 7,000-token system prompt 10,000 times a day, and the usage field cache_read_input_tokens is zero on every call. The prompt has a cache marker after its last rule, the first line holds the current time, and the cost report shows the prefix billed in full. What should the team do first?

ARaise max_tokens so the cache has room to store the prefix between the calls.
BAdd a second cache marker at the top of the prompt, so the time line is cached as well.
CMove the text that changes on every call to the end, then read the usage again.
DWait a day, because a cache needs many calls before it starts to return reads.

Show the answer

Answer: C

A max_tokens limits the reply, not the cache. Raising it changes nothing about the prefix.
B A marker caches the text before it, and that text still changes on every call, so a second marker never matches.
C Correct. A hit needs an identical prefix. The time on the first line changes it every call, so it moves after the marker.
D A cache hit does not depend on call volume. A prefix that differs each call never hits, however long you wait.

2. The system prompt starts with the time of the request, and the compliance team wants it kept for audits. The prompt is 7,000 tokens long, 10,000 calls a day share the rest of it, and the cost report shows no cache reads. Both the audit record and the cost target must be met. What is the right change?

ADelete the time, because audit records are not the prompt's job and cost comes first.
BKeep the time, but send it after the marker with the turn, and log it from your code.
CKeep the time on the first line and add a cache marker right after it, so it is cached.
DRound the time to the nearest hour so the prefix changes less often than it does now.

Show the answer

Answer: B

A The audit need is real and can be met without the prompt. Deleting it drops a requirement.
B Correct. The time stays in the request, after the cached layers, so the prefix is identical while the audit still has the value.
C A marker after the time caches a prefix that starts with a changing line, so it still never matches.
D A prefix that changes every hour still misses on the first call of each hour, and it blurs the audit time.

3. Three teams share a 7,000-token policy and differ only in a short tone paragraph, and each team edits its tone about once a week. The tone paragraph now sits first in one block, above the policy. The layout must keep each edit inside that team's own text and leave the shared policy text untouched. How should the prompt be laid out?

AGive each team its own full copy of the prompt, so one team's edit cannot reach another team.
BKeep one block and cache each team's prompt for longer, so a weekly edit is covered by the cache lifetime.
CKeep one block with the tone paragraph moved to the end and no marker, so the edit sits last.
DPolicy, tenant and guardrail as separate layers, in that order, so an edit changes only the tenant layer.

Show the answer

Answer: D

A Copies drift, and each copy is still one block with the tenant text mixed into the policy.
B An edit makes a new prefix whatever the cache lifetime is, and the policy is still mixed with the tone paragraph.
C Without a marker nothing is cached, and the policy and the tone paragraph still share one block.
D Correct. Each layer has one owner and one rate of change, so a tone edit changes the tenant layer and the policy block stays as it was.

4. A review of a cost report asks what must be true for a cached prompt to be read back. The system prompt is 7,000 tokens and the marker is placed after its last stable layer. The model has a minimum cacheable size. Select two facts that the team should rely on.

AA longer prompt caches better, so padding the prefix raises the hit rate.
BOnce a marker is set, every later call reads from the cache, whatever the prefix holds.
CThe prefix up to the marker must match the earlier call exactly.
DThe prefix must reach the model's minimum size, or the marker is ignored.

Show the answer

Answer: C, D

A Length helps only up to the point where the prefix is cacheable. Padding adds cost and does not change whether text matches.
B A read depends on a matching prefix and on the entry still being live. A marker alone guarantees nothing.
C Correct. One changed character before the marker is a different prefix and a miss.
D Correct. Below the minimum, the API ignores the marker and no error is raised.

5. Eight engineers each paste the same 20-line review prompt into their sessions, and the copies have drifted over a month. Reviews now vary by who ran them, and nobody knows which copy is current. The prompt must stay editable by the team and be versioned. What should the team do?

APut it in one Skill the team loads, with a description that says when to use it.
BAsk each engineer to re-paste the newest copy every Monday from the chat history.
CCache the pasted prompt so every session sees the same text for the whole week.
DMake the prompt longer so it covers every case and no engineer needs to edit it.

Show the answer

Answer: A

A Correct. One file is the home of the rule, the whole team loads it, and a change is made once.
B Re-pasting relies on memory and still leaves eight copies that can differ by Tuesday.
C A cache saves cost on a repeated prefix. It does not decide which text a session starts with.
D A longer prompt is a longer copy to drift, and it still needs edits.

6. A team keeps its 12 reply rules in a Skill and again in its agent's system prompt. Last week someone edited the Skill, and the agent still follows the old rules, which was found only after a customer complaint. The fix must stop a stale copy from existing and keep the Skill for the other teams that load it.

AEdit both copies each time and add a reminder to the review checklist.
BMove the rules into the system prompt only and delete the Skill, since that text is read first.
CKeep one home for each rule and let the other place refer to it, so an edit is made once.
DRaise the cache lifetime so the old rules are dropped sooner and the new ones load.

Show the answer

Answer: C

A A checklist relies on memory, and two copies will drift again.
B That drops the reuse the Skill gave other teams, and it moves the problem without removing a copy.
C Correct. One home per rule means a change cannot leave a stale copy behind.
D Cached text is the prefix as it was sent. A lifetime does not decide which rules are current.

7. A reusable support prompt serves two teams, and each has its own house style. The shared rules are 90 percent of the text and change quarterly, while the style notes differ and change monthly. The design must keep one copy of the shared rules and let each team edit only its own style. Select two parts that the design should include.

AA shared stable part that both teams load unchanged.
BA small per-team layer for the house style, kept apart from the shared part.
COne prompt for each team that repeats the shared rules in full.
DThe current time at the top, so each team can see when it ran.

Show the answer

Answer: A, B

A Correct. The shared part has one owner and one rate of change.
B Correct. A team's edit touches only its own small layer.
C Repeating the rules makes two copies that drift, and each team edits its own.
D A changing time at the top would make the shared part differ on every run.

8. A platform team reviews a reusable prompt before it is published to 40 engineers. The prompt will be read thousands of times a day and edited about once a month, and the team wants every call to reuse the same cached prefix. Select two properties that a reviewer should require.

AIt is versioned, so a change has an author and can be rolled back.
BIt puts the per-request details first, so the model sees the task early.
CIt is as short as possible, with every rule cut to a single word.
DIts stable part comes first and its per-request part comes last.

Show the answer

Answer: A, D

A Correct. A version gives an owner and a way back when a change goes wrong.
B Per-request text first changes the start of the prompt on every call, so nothing is reused.
C Cutting rules to a word loses the detail the model needs. Size is not the property to require.
D Correct. Stable first and volatile last keeps the prefix identical, which is what makes reuse and caching work.

9. A team marks a 150,000-token document as a cached prefix on a 200,000-token window and expects the cache to free the space. Each call then adds 40,000 tokens of history and asks for a 20,000-token reply, and long cases fail with a length error. The cost target is already met. What explains the error?

ACached tokens still fill the window, so 150,000 plus 40,000 plus 20,000 is over 200,000 and the input must shrink.
BThe cache read is slower than a fresh read, so the call times out and reports a length error.
CThe reply budget is too low, so raising max_tokens to 40,000 will leave room for the reply.
DA second marker after the document would stop its tokens from counting against the window.

Show the answer

Answer: A

A Correct. A cache lowers the price of the prefix and not its size. Input and reply share the window, and 210,000 does not fit in 200,000.
B A cache read is faster than a fresh read, and a slow call does not produce a length error.
C A larger max_tokens asks for a bigger reply, which leaves less room for the input and makes the error more likely.
D A marker changes how the prefix is billed. Every token of it is still in the window.

10. A support agent's history reaches 9,500 tokens after 20 turns, and the team appends each new exchange to the system prompt, between the guardrail and the cache marker. The 7,000-token prefix read from the cache on turn 1 and has read nothing since. The reply budget is 1,500 tokens. Where should the history go?

ATrim the cached prefix to make room, since it is the biggest block in the prompt.
BIn the messages after the marker, with the turn, so the prefix stays identical.
CAt the top of the system prompt, so the model reads the history before the rules.
DRaise max_tokens, so the cache can keep the longer history from one call to the next.

Show the answer

Answer: B

A Trimming the prefix changes the text that matches, so the cache misses and the savings are lost.
B Correct. History changes on every turn, so it rides with the turn after the marker and the prefix stays the same.
C Text at the top that grows each turn changes the start of the prompt, so the prefix never matches.
D max_tokens limits the reply. It does not decide what the cache keeps.

11. A cost report shows a cache hit rate of 12 percent, and spend is four times the forecast. The prompt has a marker, and the team has already tried making the prefix longer without effect. The first line of the system prompt holds a session id that changes on every call. What is the right next step?

AMake the prefix longer still, since a longer prompt caches better.
BAdd a second marker at the top, so the session id line is cached as well.
CRaise the forecast to match the spend and revisit it next quarter.
DMove the session id after the marker, then measure the hit rate again.

Show the answer

Answer: D

A Length does not help a prefix that changes at the top, and it makes each write dearer.
B A marker caches the text before it, and the session id still changes on every call, so it never matches.
C A forecast change hides the cost and fixes nothing.
D Correct. The cause is the changing line inside the prefix, so the fix is the order, and the hit rate proves it.

12. A team adds a cache marker to a prompt whose first line is the request time. The cost of 10,000 calls rises from $73.92 to $91.75, and cache_read_input_tokens is zero on every call. The cost target must still be met, and the time line cannot be dropped. What does the rise show, and what is the next step?

AA cache read costs more than a fresh token, so the marker comes off and the prefix stays uncached.
BEvery call paid the write price of 1.25 times a fresh token and nothing read it back, so fix the order.
CThe marker doubled the input tokens, so the team halves the policy text to offset it.
DThe extra $17.83 is output cost, so the team lowers max_tokens and measures again.

Show the answer

Answer: B

A A read costs a tenth of a fresh token. This run never read, so the premium came from the writes.
B Correct. A write costs more than a fresh token, and a prefix that changes on every call is written and never read.
C A marker adds no tokens. It only marks where the cached prefix ends.
D The marker changes how the input is billed. The reply is the same, so max_tokens is not the lever.

13. A team's review Skill exists, but reviewers say it is rarely picked up, and each of the 14 reviewers still pastes a prompt of their own. The Skill's description reads "Lumen helper", and about 30 reviews a week should use it. The fix must make the Skill load when a review starts, without a reviewer naming it. What should change?

ACache the Skill so every reviewer's session loads it faster than a pasted prompt.
BMake the body longer so it covers more cases and matches more of the requests.
CRewrite the description to say when to use it, such as when reviewing a Lumen pull request.
DAsk each reviewer to paste the Skill text into their own prompt as a backup.

Show the answer

Answer: C

A A cache lowers the cost of a repeated prefix. It does not change whether a session loads the Skill.
B A session decides by the description, so a longer body does not make the Skill load more often.
C Correct. The description says when the Skill applies, so a review that matches it loads the same rules for everyone.
D Pasting is the source of the drift, and it brings back 14 copies.

14. A team's reply Skill body opens with "Today is 3 March" and names the customer, Acme, on its second line. Both are edited by hand before each use, and nine copies of the file now exist with different values. The Skill must be one file with one owner. What should the body hold?

AOnly text that does not change, with the date and customer supplied in the turn.
BThe date and customer as placeholders in the body, which each developer fills in by hand.
COne copy of the file per customer, each with its own date, so that the values stay correct.
DA link to a wiki page that holds the rules, with the date and customer kept in the body.

Show the answer

Answer: A

A Correct. The body is the stable layer, so the changing values ride with the turn and one file serves every team.
B Filling placeholders by hand is the edit that created nine copies.
C A copy per customer is more copies, and each date goes stale.
D The rules would live on a page that nobody versions with the Skill, and the values still change in the body.

Play: predict the output

Each worker process builds its tools from a set of names.

python

NAMES = {"get_order", "get_policy", "process_refund"}
tools = [TOOLS[name] for name in NAMES]

Predict: Eight worker processes send the same system prompt with a marker after it. Do they share one cached prefix?

ANot reliably. Each process may list the tools in a different order, and tools come first in the prefix.
BYes. The system prompt is the same, so it is read from the cache.

Show the answer

A. Python seeds string hashing per process, so a set of names can iterate in a different order in each one. Sort the names, so every process sends the same bytes.

References

Prompt caching: the tools, system, messages order and what breaks a prefix.
System prompts: what belongs in the system prompt and what rides with the turn.
Agent Skills: how a Skill's description decides when it is loaded.
Python command line: PYTHONHASHSEED and why set order changes between runs.

Conclusions

Put stable layers first and the volatile turn last, so the prefix is the same on every call.
Place one cache marker after the last stable layer, and keep the prefix above the minimum size.
Read cache_read_input_tokens on the second call to prove the cache hit, and keep shared rules in one Skill.

You can now order a prompt into layers, place a cache marker, and read the usage fields to prove a hit.

Prefix

The start of the prompt, up to the cache marker, which must match exactly to hit

Cache marker

The cache_control field that ends the cached prefix

Cache write

The first call that stores the prefix, billed at 1.25 times a fresh input token

Cache read

A later call that reuses the prefix, billed at a tenth of a fresh token

Layer

One block of the system prompt with one owner and one rate of change

Skill

One file of reusable instructions that a team loads instead of pasting