Tracing

1. A customer reports a wrong answer from the support agent and gives the time and the order number. The log holds 12,000 prompt lines from that hour, and none carries an id that ties a run's calls together. The on-call engineer has 20 minutes before the next escalation. What should the team add first?

1. AA run id on every span, made once per run and passed to each call.
2. BA longer retention period for the prompt lines, so nothing is lost.
3. CSearch by order number for now, and add run ids when the log grows.
4. DA second copy of each prompt line in another store.

2. An agent run writes one giant log line at the end, holding the prompt, every tool result and the reply. A slow run took 9 seconds, and finding which call was slow meant reading a 40 KB line by hand. The team wants the slow call to show up at once. What should it change?

1. AWrite the line sooner in the run, so the log is fresher.
2. BPrint the same 40 KB line with line breaks for readability.
3. CWrite one span per model or tool call, each with its own timing.
4. DAdd a flag to the line that says the run was slow.

3. A log store keeps every prompt for two years. A privacy review finds customer emails and phone numbers in 3,400 of the lines. The team may keep the logs for debugging but must stop holding those values in clear text. What is the right change?

1. ARestrict the log store to two engineers and keep writing the same lines.
2. BDelete the logs older than one year and keep the newer ones as they are.
3. CAdd a banner to the log viewer that warns readers about personal data.
4. DRedact the personal fields before a span is written, with a stable hash.

4. A design review lists the fields every span in the order status agent must carry. The team wants any span to be found in one query and read without opening the prompt. Select two fields that every span needs.

1. AThe run id that ties it to its run.
2. BThe full prompt, in case a reader needs it.
3. CThe name of the call and its duration.
4. DThe customer's email, to find their runs.

5. An alert pages the on-call engineer on every tool error, and the agent logs about 60 of them a day, mostly short carrier timeouts that retry and succeed. The team now ignores the pages. The goal is fewer pages that still catch a real outage. What fits?

1. AMute the alert at night, when fewer customers are online.
2. BSend each error to a chat channel as well as the page.
3. CPage only when the error rate over a 10-minute window crosses a limit.
4. DRaise the retry count so fewer errors are logged.

6. After a release, the mean latency of the agent is flat at 1.1 seconds, yet customers say some answers take 10 seconds. The spans show 5 percent of tool calls now take over 8 seconds. The team needs an alert that would have caught this. Which metric should it read from the spans?

1. AThe p95 latency of the spans, with a limit.
2. BThe count of runs per hour.
3. CThe mean latency, with a tighter limit.
4. DThe size of the log files.

7. Cost per run rose 40 percent this week, from 0.005 to 0.007 dollars, while traffic stayed flat at 2,000 runs a day. Nothing in the error alerts changed. The team wants to know which step got more expensive. What should it do?

1. AAsk the model provider for last week's invoice.
2. BCut the answer length in the prompt and watch the total.
3. CSwitch to a cheaper model for every call.
4. DDerive cost per run from the model spans and compare steps week on week.

8. An on-call setup for the agent needs alerts that read from the spans and say something about users, not only about servers. The team has room for two. Select two metrics that come from the spans.

1. ACPU use of the host.
2. Bp95 latency per step.
3. CThe number of engineers on call.
4. DCost per run.

9. An on-call engineer receives a report that the agent told a customer the wrong thing at 14:05. There is no runbook, and the engineer starts by reading raw log text for the whole hour. Twenty minutes pass with no cause found. What order of checks should the team write down?

1. AEdit the prompt, then test, then read the log.
2. BFind the run by its id, then read the tool spans, then the last model call.
3. CRestart the agent, then read the log if the fault returns.
4. DRead the whole hour of logs from the first line.

10. A wrong answer is reported, and the engineer on call starts to rewrite the system prompt. The run's spans show the get_shipping tool returned a carrier timeout, and the model then guessed a status from the order alone. The engineer has spent 30 minutes on the prompt. What should have come first?

1. AAsk the customer for a screenshot of the answer.
2. BSwitch the agent to a larger model before changing anything else.
3. CFind the failing span and fix or handle that tool error.
4. DAdd a line to the prompt that says to be accurate.

11. Two engineers debug the same incident on Monday and each finds half the cause. Their notes are in two chat threads, and on Tuesday a third engineer repeats both checks. The team wants the next incident to start from shared steps. What should it keep?

1. AA runbook with a first check, the next checks and an owner for each step.
2. BA longer chat history, pinned for everyone to read.
3. CA rota that puts the most senior engineer on every incident.
4. DA weekly meeting to go over last week's incidents.

12. A platform review checks a triage runbook for the agent before it goes to the whole on-call rota. The runbook must work for an engineer who has never seen the agent. Select two parts the runbook must have.

1. AThe first check to run, with what a good result looks like.
2. BThe customer's email, so the engineer can call them.
3. CA rule that hashing the customer's fields is optional when in a hurry.
4. DThe owner of each step, so an engineer knows who to ask.

13. A team chose a fixed workflow for an order status feature, then found that the next step depends on what each tool returns. The spans of 50 runs show the path branches in four different ways. The team wants to decide whether an agent fits better. What should it use?

1. AA vote among the engineers on which design feels cleaner.
2. BThe prompt length as a measure of how complex the task is.
3. CThe number of tools the feature holds as a proxy for branching.
4. DThe spans, to count how often and where the path branches.

14. An agent chooses its own tools, so the path it takes cannot be predicted from the code. A customer disputes an answer, and the team must show what the agent did step by step. Only a trace can answer that. What must be true of the trace?

1. AIt was switched on after the dispute, for the next run.
2. BIt holds only the final reply, to keep the log small.
3. CEvery model and tool call is a span under the run id, so the path can be replayed.
4. DIt holds the prompts in clear text, so the wording is exact.

15. The order status agent has run in production for six months, and its spans are stored and never read. Last week a slow tool went unnoticed for 9 days until a customer complained. The team wants to catch drift sooner. What should it schedule?

1. AA regular review of the span metrics, such as p95 latency and cost per run.
2. BA yearly rewrite of the system prompt from scratch.
3. CA larger log store, so spans can be kept for longer.
4. DA rule that any engineer may mute alerts they find noisy.

16. A feature that used the agent was retired in March, yet its code still writes 15,000 spans a day. The log store grows by 2 GB a week, and nobody reads those spans. The team wants to stop paying for them. What is the right move?

1. ACompress the old spans and leave the writer running.
2. BRaise the alert limit so the feature stops showing up.
3. CSample one span in ten and keep writing the rest.
4. DRemove the span writing together with the feature, and delete its old spans.