Foundations, lesson 5

Structured tool errors

A tool error should say what to do next

Your refund tool fails with the single word Error. The agent cannot tell a timeout from a refusal, so it retries a refusal and then guesses at a reason for the customer. This is the agent at Lumen Supply, an online electronics shop. You will watch it fail, then fix it with two small functions.

I doWatch the failure and its fix.

Every failure has a category and a flag

Three parties share the work. The tool reports the failure. The harness, the code around Claude that runs each tool, decides whether to retry. Claude explains the result to the customer.

A refund can fail four ways: transient, validation, permission or business. A timeout is transient and a wrong order id is validation. Another customer's order is permission, and a closed refund window is business.

Each failure needs two answers. What kind is it, and can a retry help? The tool reports both, so the harness never guesses from a sentence.

A refusal is explained, not retried

The order is 239 days old. The harness stops after one attempt, and Claude tells the customer why.

Open the error card. The tool result carries isError, which marks it as a failure. MCP, the standard protocol that connects tools to Claude, spells it isError, and the API and the Python SDK spell the same flag is_error. Inside are the errorCategory (business), isRetryable (false) and a message the customer can read.

CustomerPlease refund order 0871, the laptop stand is wobbly.
Show the result
```
Please refund order 0871, the laptop stand is wobbly.
```

Clauderefunds order 0871

Show the result

{
  "input": {
    "order_id": "0871"
  },
  "output": {
    "isError": true,
    "error": {
      "errorCategory": "business",
      "isRetryable": false,
      "message": "Refunds close after 30 days. This order is 239 days old."
    }
  }
}

Attempt 1Refunds close after 30 days. This order is 239 days old.

Show the result

{
  "errorCategory": "business",
  "isRetryable": false,
  "message": "Refunds close after 30 days. This order is 239 days old."
}

ClaudeI'm sorry, but I couldn't refund order 0871 because refunds are only available within 30 days of purchase, and this…

Show the result

I'm sorry, but I couldn't refund order 0871 because refunds are only available within 30 days of purchase, and this order is 239 days old. I can't process this refund through our system, so please contact our support team if you'd like to discuss other options for the wobbly laptop stand.

Tried once, then explained the result.

Two functions as written

fail builds every error the tool returns. The starting code returns the word Error and nothing else. should_retry decides whether the harness tries again, and the starting code always says yes.

A retry of a refusal is not free. Three attempts are three tool calls for one answer that never changes.

starter/errors.py

1"""EDIT THIS FILE. Every tool failure is built here, and retries follow it.2Run `./workshop check structured-errors` until it is green, then3`./workshop watch structured-errors`.4"""56from kit.api import Json789def fail(category: str, message: str) -> Json:10 """The result a tool returns when it fails."""11 # TODO(workshop): set isError and carry errorCategory, isRetryable and12 # the message. A category of "empty" is not a failure: return the empty13 # list with the message and no error.14 return {"error": {"message": "Error"}}151617def should_retry(error: Json) -> bool:18 """Whether the harness should try the same call again."""19 # TODO(workshop): retry a transient failure and nothing else.20 return True

What happens when the order is too old?

Take the ticket for a laptop stand bought 239 days ago. The refund window is 30 days. The starting code retries on every failure.

Predict: How many times is the refund tried?

AThree times. The starting code retries every failure.
BOnce. The agent sees a refusal and stops.

Show the answer

A. The harness tried three times and got Error each time. Then Claude wrote a reply about a system error that was never the cause.

Three tries at a refusal, then a wrong reply

This is the same ticket with the starting code. Three identical attempts, each returning Error. The result card is red because a retry could not have helped.

This is a real Claude Haiku 5.5 run, recorded once and replayed here. The reply blames a system error because the tool gave Claude nothing else.

CustomerPlease refund order 0871, the laptop stand is wobbly.
Show the result
```
Please refund order 0871, the laptop stand is wobbly.
```

Clauderefunds order 0871

Show the result

{
  "input": {
    "order_id": "0871"
  },
  "output": {
    "error": {
      "message": "Error"
    }
  }
}

Attempt 1Error
Show the result
```
{
  "message": "Error"
}
```

Clauderefunds order 0871

Show the result

{
  "input": {
    "order_id": "0871"
  },
  "output": {
    "error": {
      "message": "Error"
    }
  }
}

Attempt 2Error
Show the result
```
{
  "message": "Error"
}
```

Clauderefunds order 0871

Show the result

{
  "input": {
    "order_id": "0871"
  },
  "output": {
    "error": {
      "message": "Error"
    }
  }
}

Attempt 3Error
Show the result
```
{
  "message": "Error"
}
```

ClaudeI wasn't able to refund order 0871 because the system returned a generic error without saying why. The refund hasn't…

Show the result

I wasn't able to refund order 0871 because the system returned a generic error without saying why. The refund hasn't been processed, so please contact us again to retry, or double-check the order number if it doesn't look right.

Tried 3 times. A retry could not have helped.

A category and a flag replace the word Error

fail now sets isError and carries errorCategory, isRetryable and the message. Only a transient failure is retryable. should_retry reads the flag, so a timeout is retried and a refusal is not.

The harness bounds the retries and waits a short random time between them. Any transient fault gets the same bounded retry.

errors.pygreen lines are what the fix adds

1"""A failure names its category, says if a retry can help, and explains."""23from kit.api import Json45RETRYABLE = {"transient"}678def fail(category: str, message: str) -> Json:9 """The result a tool returns when it fails.10 Nothing found is a valid answer, so "empty" carries no error.11 """12 if category == "empty":13 return {"orders": [], "message": message}1415 return {16 "isError": True,17 "error": {18 "errorCategory": category,19 "isRetryable": category in RETRYABLE,20 "message": message,21 },22 }232425def should_retry(error: Json) -> bool:26 """Retry only a failure that says a retry can help."""27 return bool(error.get("isRetryable"))

Nothing found is not a failure

A customer with no orders asks for a list. The lookup finds nothing, and that is a valid answer. With structured errors the result has no error flag, and Claude says there are no orders.

The starting code reports the empty list as a failure and retries it. Tell an access failure from a valid empty result: only the first carries an error.

This part is interactive in the lab: run ./workshop watch structured-errors.

We doChange one setting at a time and predict.

The same ticket now stops after one try

Use the switch to run the ticket with the starting code, then the structured errors. On the second button, the card marked "This is where the runs split" is where the harness stops retrying.

Try the flaky payment ticket too. Both versions retry it once and succeed, because the starting code retries everything.

This part is interactive in the lab: run ./workshop watch structured-errors.

Try it: four kinds of failure and an empty result

Change one setting at a time. Pick the wrong order id: the fixed code stops and Claude asks for a corrected id. Pick another customer's order: it stops and refuses plainly. Pick the empty ticket: it is answered once.

Every combination is a recorded run, so this works with no internet.

This part is interactive in the lab: run ./workshop watch structured-errors.

The same tool runs in the Agent SDK

The Claude Agent SDK serves the refund and the lookup from an in-process MCP server. The Python SDK spells the failure flag is_error, as the API does.

Here Claude decides whether to retry, so the error shape is all it has. That is why the prompt asks for one retry, a line the harness above never needs. This is a real Claude Haiku 5.5 run. With the starting code the reply blames a system error, and with structured errors it explains the refund window.

A subagent is a second agent that a coordinator starts for one job. It should retry a transient fault locally and pass up only what it cannot fix, with the partial results. Two mistakes to avoid: returning an empty result as if it worked, and aborting all four subagents because one failed. This is the report a coordinator receives:

text

failed:  lookup for order 1050, timed out twice
tried:   retried once, locally
partial: orders 1042 and 1043 found, 1050 not checked
next:    the customer can wait for a call back or ask again

app/sdk_agent.pyfile: 109 lines

1"""The refund and the lookup served as MCP tools that flag a failure.23The MCP result flag is `isError`; the Python SDK spells it `is_error`. Here4Claude decides whether to retry, so the shape of the error is all it has.5"""67import json8from types import ModuleType910from claude_agent_sdk import McpSdkServerConfig, create_sdk_mcp_server, tool1112from kit import sdk13from kit.api import Json14from .prompt import SYSTEM15from .tickets import Ticket16from .tools import LOOKUP, REFUND, lookup, refund1718SERVER = "support"19PREFIX = f"mcp__{SERVER}__"20MAX_TURNS = 62122···9 lines not shown3233 @tool(REFUND["name"], REFUND["description"], REFUND["input_schema"])34 async def process_refund(args: Json) -> Json:35 tries["n"] += 136 order_id = str(args.get("order_id", ""))3738 return result(refund(order_id, tries["n"], ticket.customer, errors))3940 @tool(LOOKUP["name"], LOOKUP["description"], LOOKUP["input_schema"])41 async def lookup_orders(args: Json) -> Json:42 return result(lookup(ticket.customer, errors))4344 return create_sdk_mcp_server(SERVER, tools=[process_refund, lookup_orders])454647def request_for(ticket: Ticket, tag: str) -> Json:48 """The plain request a recording is keyed by."""49 return sdk.build_request(50 ticket.text,51 SYSTEM,52 tools=[PREFIX + REFUND["name"], PREFIX + LOOKUP["name"]],53 max_turns=MAX_TURNS,54 tag=tag,55 )56···53 lines not shown

CustomerPlease refund order 0871, the laptop stand is wobbly.
Show the result
```
Please refund order 0871, the laptop stand is wobbly.
```

Clauderefunds order 0871

Show the result

{
  "input": {
    "order_id": "0871"
  },
  "output": {
    "isError": true,
    "error": {
      "errorCategory": "business",
      "isRetryable": false,
      "message": "Refunds close after 30 days. This order is 239 days old."
    }
  }
}

Attempt 1Refunds close after 30 days. This order is 239 days old.

Show the result

{
  "errorCategory": "business",
  "isRetryable": false,
  "message": "Refunds close after 30 days. This order is 239 days old."
}

ClaudeI couldn't refund order 0871 because refunds close 30 days after purchase, and this order is 239 days old. If you meant…

Show the result

I couldn't refund order 0871 because refunds close 30 days after purchase, and this order is 239 days old. If you meant a different order, please send its four-digit order ID and I'll process that one for you.

Tried once, then explained the result.

You doFix the starting code, then test yourself.

You do: build a useful failure

Copy the starting code and open work/structured-errors/errors.py. fail() returns a bare "Error", should_retry() retries everything, and an empty lookup counts as a failure.

Terminal

./workshop start structured-errors
./workshop check structured-errors
./workshop watch structured-errors

The checks want an error flag, a category, a retry hint and the message on every failure. Only a transient failure is retried, and an empty result is an empty list with no error.

Check your understanding

Answer each question, then read why every option is right or wrong. The wrong options are the mistakes engineers make on the job.

1. A refund tool returns the text "Error" for every failure. The agent retries an order that is past the refund window and then guesses at a reason. What is the best fix?

AAdd a long prompt paragraph that lists every reason a refund can fail and what to say for each.
BRaise the retry limit so the agent has more chances to find the cause.
CReturn an error with a category, a retryable flag and a message the agent can read.
DLog the failure on the server and return an empty result so the agent moves on.

Show the answer

Answer: C

A A prompt cannot know which reason applied to this call. The tool knows.
B More retries repeat a refusal that no retry can change.
C Correct. The category and the flag tell the harness whether a retry can help, and the message tells Claude what to say.
D An empty result hides the failure, and the agent has nothing to explain.

2. Which failure should the harness retry without asking Claude?

AA refund request for an order that is 239 days old.
BA refund request for an order id written with a letter in it.
CA refund request for an order that belongs to another customer.
DA payment service that timed out once.

Show the answer

Answer: D

A This is a business refusal. Trying again gives the same answer.
B This is a validation error. The input has to change first.
C This is a permission error. A retry does not change the owner.
D Correct. A timeout is transient, so the same call may succeed a moment later.

3. Which fields make a tool error useful to both the harness and Claude? Select two.

AA stack trace from the server that raised it.
BAn error category such as transient, validation, permission or business.
CThe HTTP status code alone, such as 500, used as the whole category.
DA retryable flag the harness can read without parsing text.

Show the answer

Answer: B, D

A A stack trace is operator detail. A customer never needs it.
B Correct. The category says what kind of failure it is, so each kind gets the right response.
C A status code says where a call failed, not what to do next. A 500 can be a timeout or a bug.
D Correct. The harness decides on the flag, not on a guess from a sentence.

4. A tool fails because the customer gave an order id that matches no order. Which category fits, and what follows?

ATransient, so the harness retries the same call again.
BValidation, so Claude asks for a corrected id.
CPermission, so Claude asks the customer for a manager.
DBusiness, so Claude explains the refund window to them.

Show the answer

Answer: B

A The same id fails again. Only a transient failure earns a retry.
B Correct. The input is wrong, so the next step is a corrected input.
C Nothing was denied. The order was not found.
D The refund window is not the problem. The id never matched an order.

5. A lookup returns an empty list for a customer with no orders. The agent calls it three times and then says the lookup failed. What is the right handling?

AReturn the empty list, no error flag, and say nothing was found.
BRaise the retry limit to five, in case the orders load on a later try.
CReturn an error flag on an empty list, so the agent stops asking.
DDrop the result and ask the customer to try again later.

Show the answer

Answer: A

A Correct. Nothing found is a valid answer. An access failure is different, and it carries an error.
B The list is empty because the customer has no orders, so five tries return the same list.
C An error flag says the call failed. This one worked and found nothing.
D The customer learns nothing, and the real answer is lost.

6. Where should retries of a transient failure live?

AIn the system prompt, as a rule for Claude to follow.
BIn a background job that replays every failed call until one succeeds.
CIn code around the tool call, bounded by the retryable flag.
DIn the tool description, as a sentence that tells Claude to try the call twice.

Show the answer

Answer: C

A A prompt rule depends on Claude counting attempts. Code counts exactly.
B Replays with no bound hammer a tool that refuses. A retry needs a limit and a flag.
C Correct. Code can bound the attempts and stop on a refusal.
D A description says what a tool does. It cannot enforce a limit.

7. A refund fails with a permission error because the order belongs to another customer. Which reply is best?

AApologise, say something went wrong, and ask them to try again later.
BSay the refund cannot be done on that order, and offer to look up their own orders.
CSay who owns the order, so they can contact that person and sort it out between them.
DClose the ticket at once, because the first failed call ends the conversation.

Show the answer

Answer: B

A This hides the reason, and a later try gets the same refusal.
B Correct. It refuses plainly and offers a next step that stays within their account.
C Naming the owner leaks another customer's data.
D One refusal does not end the ticket. The customer still needs a reason and a next step.

8. A coordinator runs four subagents and one fails. Select two mistakes in how the failure is handled.

AThe subagent swallows a timeout and returns an empty result as if it had worked.
BThe reply names what failed, what was tried, and what the customer can do next.
CThe subagent retries a timeout once, and passes up only what it could not fix.
DThe coordinator aborts the whole run because one of four subagents failed.

Show the answer

Answer: A, D

A Correct. The coordinator now trusts a result that is not real.
B This is what a good failure report contains.
C A local retry of a transient fault is good practice. Only what it cannot fix goes up.
D Correct. Three good results are lost, and the failed one could have been reported as partial.

Play: spot the bug

The retry rule is right, but the harness hands it the wrong thing.

python

def should_retry(error: Json) -> bool:
    return bool(error.get("isRetryable"))


result = lookup_order("1042")  # times out
if result.get("isError") and should_retry(result):
    result = lookup_order("1042")

Predict: The lookup times out, and its error says isRetryable is true. Is it retried?

AYes. The error says a retry can help.
BNo. The flag sits inside result["error"], so the rule reads nothing and says no.

Show the answer

B. The rule reads isRetryable from the top of what it gets, and the flag is one level down. Pass result["error"], and test the harness with a real failure, not only the rule alone.

References

Handle tool calls: how to return is_error so the model knows a call failed.
MCP tools specification: the isError field an MCP server returns on a failed tool.
Tool use overview: where a tool result sits in the conversation.
Agent SDK custom tools: how an SDK tool reports an error to the model.

Conclusions

A tool failure carries a category, a retryable flag and a readable message.
Only a transient failure is retried, in code, with a bound.
An empty result is a valid answer, not an error.
Claude explains a refusal in plain words and leaves out operator detail.

You can now build tool errors that tell the harness whether to retry and tell Claude what to say.

Error category

The kind of failure: transient, validation, permission or business

Retryable flag

A field that says whether the same call may succeed later

Transient error

A temporary fault, such as a timeout, that a retry can clear

Empty result

Nothing found, reported with no error flag

Business error

A refusal by a rule, such as a closed refund window