Professional, lesson 8

The context budget

Window minus output minus thinking is your room

A Lumen Supply agent fails with a context error on a 60-turn refund case, so a developer raises max_tokens. The error comes back with a larger overage, because output and thinking take their room from the same window as the history. This lesson budgets the room, trims on a rule, and keeps the case facts in a state object. The case is 60 turns long and holds 26,982 tokens of history. The window is fixed, and input, output and thinking all share it. You will count the tokens before the call, see what a raised output limit costs, and write the two functions that fix it.

I doWatch the failure and its fix.

One 60-turn case and a fixed window

The model reads only what fits in its window. Your code decides what goes in, so your code does the budget: it counts the input, reserves the output and thinking, and trims the rest.

Three things go wrong, in this order. The output limit is raised to fix a context error, which leaves less room. The history is never trimmed, so long cases always overflow. And the case facts live only in the history, so a trim deletes them.

A case that fits

The medium window holds 40,000 tokens. The output limit is 1,024 and 2,000 are reserved for thinking, so the room for input is 36,976. The history and the state object come to 27,040 tokens, so all 60 turns are sent.

Open the count card to see the number the API returned before the call was made. The reply names the order and the refund, because both are in the history and in the state object.

Case60 turns, 31,644 tokens of history
Budgetroom 36,976 = window 40,000, output 1,024, thinking 2,000
Show the result
```
{
  "window": 40000,
  "output": 1024,
  "thinking": 2000
}
```

count_tokens56 turns, 29,605 tokens in

Show the result

{
  "state": {
    "order": "LS-48213",
    "refund": "$186.40",
    "approver": "Dana Ruiz",
    "deadline": "2026-07-14",
    "carrier": "TrackNova",
    "tier": "gold"
  }
}

ReplyThe case concerns order LS-48213, with a refund of $186.40 recorded in the case facts, but the case logs show no completed refund action.

COMPACTED: 56 of 60 turns sent, $0.003 of input for this call.

The room formula, written as code

The room for input is the window minus the output limit minus the thinking budget. Thinking tokens count toward the window, so a thinking budget is not free. The API spends them from the output limit. This lab reserves them as a line of their own on top, which can only reserve more than a call uses.

compact then keeps the newest turns that fit in 80 percent of the room. The other 20 percent is margin, because a count is exact only for the request you counted.

A turn that states a case fact carries it in a facts key, such as the order number. compact merges those keys into the state object.

starter/budget.py

1"""EDIT THIS FILE. It has two planted defects.21. room(): the window is returned whole, so output and thinking are ignored3 and raising max_tokens looks free.42. compact(): nothing is trimmed and no state object keeps the facts, so a5 long case always overflows.67Run `./workshop check context-budget` until it is green, then8`./workshop watch context-budget`.9"""1011from collections.abc import Callable1213from kit.api import Json141516def room(window: int, out: int, thinking: int) -> int:17 """Return the tokens left for input.18 `out` is the output limit and `thinking` the thinking budget.19 """20 return window212223def compact(24 turns: list[Json], space: int, size: Callable[[Json], int]25) -> Json:26 """Return the turns to send and a state object of the case facts.27 `space` is the room for input, and `size` gives a turn's tokens.28 A turn that states a fact holds it in a `facts` dict, such as29 {"order": "LS-48213"}.30 """31 return {"turns": turns, "state": {}}

Does raising max_tokens add room?

A context error occurs on a long case. More tokens seem like more space.

Predict: Does raising max_tokens add room?

AYes. More tokens means more space.
BNo. Output shares the window, so it takes room from the input.

Show the answer

B. The window is split among input, output and thinking. A larger output limit leaves less for input.

Raising max_tokens makes it worse

This is a real Haiku 5.5 count, recorded once and replayed here. The starting room returns the whole window, so it ignores the 16,000 output tokens that were added. The code sends all 60 turns and does not trim.

The request needs 44,982 tokens: 26,982 of input, 16,000 of output and 2,000 of thinking. The window is 24,000, a scaled-down stand-in, so the request is 20,982 over. A real window refuses a request that goes over. The raised limit took 14,976 tokens of room from the input.

Case60 turns, 31,644 tokens of history
Budgetroom 24,000 = window 24,000, output 16,000, thinking 2,000
Show the result
```
{
  "window": 24000,
  "output": 16000,
  "thinking": 2000
}
```
count_tokens60 turns, 31,644 tokens in
Show the result
```
{
  "state": "none"
}
```
Context error49,644 tokens needed, 25,644 over the window

OVERFLOWED: 60 of 60 turns kept, 25,644 tokens over the window, so nothing is sent.

Budget, compact and keep a state object

room subtracts the output limit and the thinking budget from the window. compact keeps the newest turns that fit in 80 percent of that room. It also collects the facts from every turn into a state object.

The state object is sent in the system prompt, outside the part that is trimmed. A trim can then drop any turn without losing a fact. Sending a 27,000-token history on each of 60 turns costs $1.62 of input on Haiku, and the compacted 16,237 tokens cost $0.97.

budget.pygreen lines are what the fix adds

1"""Budget the window: the room for input, and a compaction that keeps facts."""23from collections.abc import Callable45from kit.api import Json67SHARE = 0.88910def room(window: int, out: int, thinking: int) -> int:11 """Return the tokens left for input.12 Input, output and thinking share one window, so a larger output limit13 or thinking budget takes room from the input.14 """15 return window - out - thinking161718def compact(19 turns: list[Json], space: int, size: Callable[[Json], int]20) -> Json:21 """Return the newest turns that fit 80 percent of `space`, and a state.22 The state object holds every fact the dropped turns stated, so a trim23 never deletes a fact the case still needs.24 """25 state: Json = {}26 for turn in turns:27 state.update(turn.get("facts", {}))2829 limit = int(SHARE * space)30 kept: list[Json] = []31 used = 032 for turn in reversed(turns):33 if used + size(turn) > limit:34 break3536 kept.append(turn)37 used += size(turn)3839 kept.reverse()40 while kept and kept[0]["role"] != "user":41 kept.pop(0)4243 return {"turns": kept, "state": state}

We doChange one setting at a time and predict.

The same input now ends well

Switch between the two fixes in the small window. The history is the same in both runs. Choose Trim and state. The note "This is where the runs split." appears and the Budget card is outlined: a room of 24,000 against a room of 20,976.

Trim and state keeps 36 of the 60 turns, counts 16,237 tokens, and the reply still names the order and the refund. The 24 oldest turns were trimmed, and they held all six facts. The state object carried them.

This part is interactive in the lab: run ./workshop watch context-budget.

Try it: fix and window

Change one setting at a time. All six combinations are recorded runs, so this works with no internet.

Raising the output limit overflows in all three windows. The large window, at 44,000 tokens, is only 982 over, so a larger window delays the error and does not remove it. Trim and state completes in all three: the small window is compacted, and the others send every turn.

This part is interactive in the lab: run ./workshop watch context-budget.

Count before you call

The token count is a real API call that returns the input size before you send. The lab counts the full history once and uses that number to size each turn. It then counts the request it plans to send, and sends it only if input, output and thinking fit the window.

The same budget covers what enters the history. Rank retrieved passages and send the top few, and cut a tool result to the fields the next step needs before it is stored. Cache the unchanging prefix. Each of these cuts input tokens, which are billed on every call.

starter/budget.py

Case60 turns, 31,644 tokens of history
Budgetroom 36,976 = window 40,000, output 1,024, thinking 2,000
Show the result
```
{
  "window": 40000,
  "output": 1024,
  "thinking": 2000
}
```

count_tokens56 turns, 29,605 tokens in

Show the result

{
  "state": {
    "order": "LS-48213",
    "refund": "$186.40",
    "approver": "Dana Ruiz",
    "deadline": "2026-07-14",
    "carrier": "TrackNova",
    "tier": "gold"
  }
}

ReplyThe case concerns order LS-48213, with a refund of $186.40 recorded in the case facts, but the case logs show no completed refund action.

COMPACTED: 56 of 60 turns sent, $0.003 of input for this call.

You doFix the starting code, then test yourself.

You do: budget the window and keep the facts

Copy the starting code and open work/context-budget/budget.py. Its room() returns the whole window, so output and thinking look free. compact() trims nothing and keeps no state, so a long case overflows.

Terminal

./workshop start context-budget
./workshop check context-budget
./workshop watch context-budget

The checks want room that shrinks as output and thinking grow, and a trim at 80 percent of the room. The kept history starts on a user turn and fits, and the state keeps every fact.

Check your understanding

Answer each question, then read why every option is right or wrong. The wrong options are the mistakes engineers make on the job.

1. A refund agent fails with a context error on 60-turn cases. The window is 40,000 tokens, the history is 27,000 tokens, and 2,000 tokens are reserved for thinking. A developer raised max_tokens from 1,024 to 16,000 to stop the error, and the next run failed with a larger overage. Which change addresses the cause?

ARaise max_tokens again to 32,000, so replies are not cut off and the error cannot return.
BTrim the history so input, output and thinking fit the window, and keep max_tokens modest.
CRemove the thinking budget and keep max_tokens at 16,000, since thinking is the only part that shares the window.
DMove to a larger window tier and leave the history growing, since the window then cannot overflow.

Show the answer

Answer: B

A Output shares the window with input. A larger limit takes room from the history, so the error comes sooner.
B Correct. The room is the window minus output minus thinking, so the history has to fit inside what is left.
C Thinking tokens count toward the window like any others. Removing the budget frees room, but the raised output limit still leaves too little.
D A larger window moves the limit and the history keeps growing each turn, so the error returns later.

2. A planning agent runs in a window of 200,000 tokens. It reserves 8,000 tokens for its answer and 16,000 tokens for thinking, and neither can be lowered. The case history is 190,000 tokens, and a teammate says it fits because the window is larger than the history. What is the room for input, and what follows?

AThe room is 200,000 tokens, so the history fits with 10,000 to spare.
BThe room is 192,000 tokens, so it fits with 2,000 to spare after the answer reserve.
CThe room is 176,000 tokens, so the history is 14,000 over and must be trimmed.
DThe room is 184,000 tokens, so the history is 6,000 over and only the answer reserve needs lowering.

Show the answer

Answer: C

A The window is the total, not the room. The answer and thinking reserves are taken from it first.
B That subtracts the answer reserve and forgets the thinking reserve, which is also taken from the window.
C Correct. 200,000 minus 8,000 minus 16,000 is 176,000, and the history is 14,000 over that.
D That subtracts only the thinking reserve and forgets the answer reserve. The room is 176,000, and the trim is larger.

3. A 60-turn refund case is trimmed to its newest 36 turns to fit the window. The refund amount was agreed in turn 3, so it was trimmed away, and the closing answer now asks the customer for it again. The fix must keep the amount and must not send more tokens than the room allows. Which design keeps the amount?

AKeep the facts in a state object sent with every request, outside the part that is trimmed.
BTrim the newest turns first, so the early turns and their facts survive the trim.
CAdd a summary of the dropped turns as one more turn in the history, and let the next trim treat it like any other.
DTrim at 100 percent of the room, so there is no margin and the early turns are more likely to stay.

Show the answer

Answer: A

A Correct. The state object carries the facts, so a trim can drop any turn without losing them.
B Dropping the newest turns loses the latest instructions, and the closing answer depends on those.
C A fact kept only in the history is deleted by the next trim. The state object must live outside it.
D Compaction keeps the newest turns that fit, so the oldest go first at any share. A trim at 100 percent only removes the margin.

4. A review of a failing agent lists everything that shares its 40,000-token window. The agent sends a 27,000-token history, a short system prompt and one closing question, and the review must say what else reduces the room left for input. Select two items that take room from the input.

AThe price per million input tokens, which is set by the model.
BThe output limit set with max_tokens, which is reserved before the call.
CThe temperature setting, which changes how varied the replies are.
DThe thinking budget reserved for the reply, which counts toward the window.

Show the answer

Answer: B, D

A Price is a cost, not a share of the window. It does not change how many tokens fit.
B Correct. Output tokens are reserved from the same window, so a larger limit leaves less for input.
C Temperature changes how the reply varies, not how much of the window it takes.
D Correct. Thinking tokens count toward the window and the output limit, so they take room too.

5. A retrieval step returns 40 passages of about 700 tokens each, which is 28,000 tokens. The window is 32,000 tokens with 4,000 reserved for output and 2,000 for thinking, and neither the window nor the cost per call can grow. The step must send its input inside 80 percent of the room. Which change fits the budget?

ASend the first 20 passages in the order they were retrieved, and drop the other 20 without a ranking.
BRaise max_tokens so the answer can cite more of the 40 passages in its reply.
CSend all 40 passages and ask the model to ignore the ones it does not need.
DRank the passages by relevance and send the top ten ahead of the question.

Show the answer

Answer: D

A Fourteen thousand tokens fit, but the order retrieved is not the order of relevance, so the best passages may be dropped.
B A larger output limit takes room from the input, so fewer passages fit.
C Ignored passages still take room and cost, and the request overflows at 28,000 tokens against a room of 26,000.
D Correct. Ten passages are about 7,000 tokens, which is inside 80 percent of the 26,000-token room, and they are the relevant ones.

6. A team chose a window five times larger so that it never has to trim a case history. The history now grows to 100,000 tokens, each call re-sends all of it, and the cost per call rose fivefold while the accuracy bar did not move. The budget is fixed. Which change is right?

ATrim old turns and keep the facts in a state object, so each call sends only what the step needs.
BKeep the large window and ask for shorter answers, since output is where the cost sits in this setup.
CRaise the output limit to use the extra window, so each call does more work for the same price.
DCompact only when the history reaches 100 percent of the room, so fewer calls pay for compaction.

Show the answer

Answer: A

A Correct. Input tokens are billed on every call, so sending less is the lever. A larger window only moves the overflow.
B The cost grew because the input grew. Shorter answers cut the small part of the bill.
C A larger output limit takes room from the input and adds spend. It does not fix the re-sent history.
D Compacting at 100 percent leaves no margin and the request fails first. A trim at about 80 percent keeps the call safe.

7. An agent calls a lookup tool that returns a 20 KB record, about 5,700 tokens, on every call. After six calls the 40,000-token window is full, and the next step needs only the order status and the carrier from each record. Which change keeps the loop running?

ARaise max_tokens so the later calls have more room to finish their replies.
BMove to a larger window and keep every full record in the history for later calls.
CCut each tool result to the fields the next step needs before it is stored.
DKeep every full record and drop the oldest turns only when the context error appears.

Show the answer

Answer: C

A A larger output limit takes room from the input, so the window fills sooner.
B A larger window only delays the overflow, and each full record is billed on every later call.
C Correct. Two fields are a few dozen tokens, so six calls use almost none of the window.
D Waiting for the error drops turns after the failed call, and the dropped turns may hold facts the case needs.

8. A support agent re-sends its whole history on every turn, and the input bill grows with each turn of a 60-turn case. The agent must keep the case facts and must not lower its accuracy. Select two changes that cut the cost of the re-sent history.

ACompact old turns into a short block, and keep the facts in a state object.
BRe-send the whole history on every turn, so the model never misses a fact.
CCache the stable prefix, such as the system prompt and the tool list, so it bills at the cached rate.
DPick a window ten times larger, so the history is billed once and not on every turn.

Show the answer

Answer: A, C

A Correct. Fewer tokens per call is the direct cut, and the state object keeps the facts.
B Re-sending everything is the cost. Facts belong in a state object, not in a growing history.
C Correct. The unchanging prefix is the same on every call, so caching it cuts its price.
D Every call still bills its input tokens, whatever the window size. A larger window does not bill once.

Play: predict the output

This trim keeps the newest 36 turns and drops the rest.

python

def trim(turns: list[Json]) -> list[Json]:
    return turns[-36:]

Predict: The cut falls between an assistant turn that calls a tool and the user turn that holds its result. What does the API do with the trimmed history?

AIt rejects the request, because a tool result has no tool call before it.
BIt reads the result as plain text and answers.

Show the answer

A. A tool result must answer a tool call in the turn before it. Cut only where a user turn starts a new exchange, never at a turn that carries tool results.

References

Context windows: how input, output and thinking share one window.
Token counting: measure a request before you send it, at no cost.
Context editing: clearing old tool results on the server instead of in your code.
Effective context engineering: why the smallest set of tokens that does the job wins.

Conclusions

Room is the window minus the output limit minus the thinking budget, and the history must fit inside it.
Count the tokens before the call, trim at about 80 percent of the room, and never raise max_tokens to fix a context error.
Keep the case facts in a state object outside the trimmed history, so a trim never deletes one.

You can now compute the room, compact a long history on a rule, and prove with a count that the request fits.

Context window

The fixed number of tokens that input, output and thinking share

Room

The tokens left for input: window minus output minus thinking

count_tokens

An API call that returns the input size of a request before it is sent

Compaction

Dropping the oldest turns until the history fits a share of the room

State object

The case facts, kept outside the history and sent with every request

Thinking budget

The tokens reserved for the model to think, which count toward the window