CCDV-F 4.1 — Debugging and error handling

Three layers a failure can live in

check from the top down

TransportHTTP status, never reached the model
ProtocolHTTP 200, but the turn is incomplete
Contenta complete answer that is wrong

Find the layer before choosing the fix. The top two are your integration; only the bottom one is the model's output.

Identifying the error type

Start with the status code. The API returns a JSON body of {"type": "error", "error": {"type": …, "message": …}} together with a request_id, and the error type names the category precisely.

Status	Type	Retry?
400	`invalid_request_error` — malformed or invalid request	No — fix the request
401 / 403	`authentication_error` / `permission_error` — credentials or scope	No — fix the credential
404	`not_found_error`	No
413	`request_too_large`	No — send less
429	`rate_limit_error` — rate or spend limit	Yes, with backoff; honour `retry-after`
500 / 504	`api_error` / `timeout_error`	Yes, with backoff
529	`overloaded_error` — the API is temporarily overloaded	Yes, with backoff

The official SDKs already retry the transient set — connection errors, 429, 500, 504 and 529 — with exponential backoff, two attempts by default and configurable, and they respect a retry-after header when one is present. A hand-rolled client has to do the same. What no amount of retrying fixes is the 4xx family: a 400, 401, 403, 404 or 413 will return exactly the same result next time.

When the call succeeded but the turn did not

The second layer is the protocol. stop_reason tells you why generation stopped, and each value implies a different recovery.

`stop_reason`	What happened	Recovery
`end_turn`	Finished naturally	Use the response
`tool_use`	Waiting on a tool	Run it, append the results, continue the loop
`max_tokens`	Hit your output limit mid-answer	Raise `max_tokens` or continue — never treat the partial text as complete
`stop_sequence`	One of your stop sequences fired	Read `stop_sequence` to see which
`pause_turn`	A server-side tool loop hit its iteration limit	Send the assistant content back so it can continue
`refusal`	The model declined	Read `stop_details`; escalate or try a fallback — do not loop
`model_context_window_exceeded`	The response filled the window	Treat it as truncated and trim context

Tool failures are a third kind of signal again, and the mistake is to let them escape as exceptions. A tool that fails should return its error to the model as a tool_result with is_error set, so the model can try a different argument or report honestly that it could not proceed. Raising instead ends the run for a problem the agent might well have recovered from.

Triage in one placepython

try:
    resp = client.messages.create(**kwargs)
except RateLimitError:            # 429 — transient
    sleep(backoff()); return retry()
except APIStatusError as e:
    if e.status_code in (500, 504, 529):
        sleep(backoff()); return retry()
    raise                         # 4xx: retrying changes nothing

if resp.stop_reason == "tool_use":
    return run_tools_and_continue(resp)
if resp.stop_reason in ("max_tokens", "model_context_window_exceeded"):
    return mark_truncated(resp)   # not a finished answer
if resp.stop_reason == "refusal":
    return escalate(resp)         # looping will not help

Choosing a recovery

What kind of failure is it?

429, 5xx, timeout
Retry with backoffhonour retry-after
400, 401, 413
Fix and resendsame request always fails
Truncated or paused
Continue the turnraise the limit, send content back
Wrong but complete
Validate and re-prompta content problem

Four families, four responses. Nothing here is a judgement call once you know which family the failure is in.

Isolating the integration layer from the model

This is the heart of the objective. A symptom that looks like a bad model answer is very often your own plumbing, and the fastest way to tell is to reproduce the exact request the model actually received. If the same messages, tools and system prompt produce a good answer in isolation, the fault is in what your code assembled or how it read the reply.

Same symptom, two causes

Integration layer

A tool_result missing its tool_use_id
The assistant turn never appended to history
A truncated reply read as a finished answer
Stale tool output the model had to trust

Model output

A complete answer that is wrong or invented
Output that ignores a stated constraint
Valid-looking JSON with a missing field
A refusal, reported in stop_reason

Both show up in a bug report as “the model gave a bad answer”. The left column is all fixable in your code, and none of it is prompt work.

Four questions settle the split quickly. Did the request contain what you think it did? Did the response carry stop_reason of end_turn? Did every tool call get a matching result? And did the tools return what they were supposed to? Only when all four hold is the answer genuinely the model's, and then you are in prompt, schema and validation territory — covered by the output-handling objective in Domain 6.

Worked example · “The model keeps hallucinating order numbers”

A retailer's support agent sometimes quotes an order number that does not exist. The team files it as a hallucination and starts rewriting the system prompt.

Capture one bad run. The trace shows two tool_use blocks in one assistant turn and only one tool_result in the following user turn.
Reproduce. Replaying the same messages with both results present produces the correct order number every time.
Diagnose. The harness built results in a loop that overwrote its accumulator, so the second call's answer never came back. The model filled the gap with something plausible.
Fix the layer that was broken. Every tool_use block now gets a tool_result with the matching tool_use_id, and an assertion fails the run when the counts differ.
Keep one prompt change. The system prompt now tells the agent to say it could not retrieve an order rather than guess — useful, but a second line of defence, not the fix.

Takeaway: Invented content is a symptom, not a diagnosis. A missing tool result, a dropped assistant turn and a truncated reply all present as hallucination, and all three live in your code.

Trace analysis

Agents are non-deterministic between runs even with identical prompts, so debugging depends on what you recorded rather than on reproducing the failure on demand. Anthropic's own account of running agents in production stresses full tracing of decision patterns and interaction structures, and notes that agents are stateful: a small failure part-way through compounds, which is why durable execution and resuming from a checkpoint matter more than a clean restart.

What a usable trace records

Passes: request_id for every API callQuote it to support; it is in the response header and error body
Passes: stop_reason on every responseSeparates truncation from completion
Passes: Session identifier and turn numberLets you resume rather than restart
Passes: Tool name, input and output, per callWhere most “model” bugs actually are
Check: Retry events with attempt and statusA retry storm is invisible without them
Check: Token usage and cost per runExplains a run that stopped at a budget cap
Fails: Only the final answerTells you it broke, never where

Record these on every run, not only on failures — you cannot go back and add them to the run that broke.

The harnesses give you much of this for free. A Claude Code run in stream-json mode emits an event for each API retry, carrying the attempt number, the delay before the next one and an error category such as rate_limit or overloaded — so a slow run that was really a retry storm is visible rather than mysterious. An Agent SDK run ends with a result carrying the termination subtype, num_turns, cost and session_id; the subtype alone distinguishes a finished task from one that hit a turn or budget cap.

Worked example · A Monday-morning retry storm

A bank's document agent takes nine minutes per file on Monday mornings and four seconds the rest of the week. No errors reach the application logs.

Symptom. Latency, not failure. Nothing is being reported because the SDK is retrying transparently and eventually succeeding.
The trace. Retry events show attempts climbing to five with a rate_limit category, every Monday between 09:00 and 09:30.
The cause. A weekly batch job shares the same API key and consumes the rate limit for the workspace.
Recovery strategy. Keep the backoff, honour retry-after, and move the batch job to the Message Batches API and a separate key so interactive traffic is not competing with it.
The guardrail. An alert on retry count per run, because a silent retry is a cost and a latency problem even when it succeeds.

Takeaway: “No errors in the logs” is not the same as no failures. Transparent retries hide a 429 problem until you record the attempts.

Traps the wrong answers are built from

Tempting but wrong	Do this instead
Retrying a 400 or 401 with backoff	Fix the request or the credential; only 429 and 5xx are worth retrying.
Treating a `max_tokens` stop as a finished answer	Mark it truncated and continue or raise the limit.
Letting a tool exception escape the loop	Return the failure as a `tool_result` with `is_error` so the model can recover.
Calling a wrong answer a hallucination before checking the trace	Verify the request, the stop reason and the tool results first.
Logging only the final output	Record request ids, stop reasons, tool inputs and outputs, retries and cost on every run.

You should now be able to

Name the common API error types and say which are worth retrying.
Distinguish an HTTP error, a stop reason and a tool error, and handle each appropriately.
Choose a recovery strategy — retry, repair, continue, escalate — from the failure's category.
Isolate whether a bad answer came from the integration layer or from the model.
Specify what a trace must capture for an agent failure to be diagnosable after the fact.

Debugging and error handling