CCDV-F 6.3 — Output handling

Everything between the model and your database

Constrainschema or tool defines the shape
Parse defensivelycheck stop_reason, then decode
Validateranges, references, business rules
Act or escalatecommit, retry, or hand to a human

invalid · retry once with the error, then escalate

Most production bugs in this area come from deleting one of the middle two steps because the happy path worked in testing.

Structured output patterns

There are three ways to get machine-readable output from Claude, and they are not interchangeable. Knowing which one a scenario calls for is most of this task statement.

Ask and parse. You describe the format in the prompt — ideally with examples, as 6.2 covers — and parse whatever comes back. It works everywhere, needs no special parameters, and it is the only option when the shape is loose. It is also the one that occasionally returns “Here's the JSON you asked for:” in front of the JSON, which is why the other two exist.

Structured outputs. You pass a JSON Schema and the API constrains generation so the final response text conforms to it. The current parameter is output_config with a format object of type json_schema carrying your schema. It was previously a top-level output_format parameter, and the old beta header is still accepted during transition, but new code uses output_config.format. This is the pattern for a final answer that your code consumes — extraction, classification, anything that lands in a database.

Strict tool use. You set strict: true on a tool definition, and the tool's input_schema is enforced the same way. This is for the arguments of a function you are about to run, rather than for the answer to the user. The two can be combined in one request, and they solve different halves of the problem: one makes Claude's reply parseable, the other makes its function calls safe to execute. Tool design proper is 8.1.

A schema-constrained extractionpython

schema = {
    "type": "object",
    "properties": {
        "invoice_number": {"type": "string"},
        "total": {"type": "number"},
        "currency": {"type": "string", "enum": ["GBP", "EUR", "USD"]},
        "due_date": {"type": "string", "format": "date"},
    },
    "required": ["invoice_number", "total", "currency"],
    "additionalProperties": False,   # no surprise keys
}

resp = client.messages.create(
    model=MODEL_ID,
    max_tokens=1024,
    messages=[{"role": "user", "content": invoice_text}],
    output_config={"format": {"type": "json_schema", "schema": schema}},
)

# Shape is guaranteed. Truth is not — see validation below.
data = json.loads(resp.content[0].text)

The supported schema vocabulary is a deliberate subset, and the gaps are where teams get hurt. Types, properties, required, additionalProperties: false, items, minItems and maxItems, enums and consts, anyOf and oneOf, nested objects, and string formats such as date, date-time, email, uri and uuid are supported. minimum, maximum, minLength and maxLength are not enforced — the SDKs turn them into description text, which is advice to the model rather than a constraint. Neither are pattern, $ref, allOf, or recursive schemas. There are also complexity limits, including a cap on strict tools per request, and an over-complex schema is rejected with a 400 rather than silently downgraded.

What the schema buys you, and what it does not

Guaranteed by the schema

Valid JSON, no preamble or trailing prose
Required keys present, no unexpected keys
Each value of the declared type
Enum values drawn from your list

Still your job

Numeric ranges — minimum is not enforced
Patterns and lengths — pattern is unsupported
Cross-field logic: does the total match the lines?
Whether the facts are actually in the source

Read the right-hand column as your validation to-do list. Everything there is your code's job, whatever the schema says.

Defensive parsing

Defensive parsing starts before the parse. A response is not a string: content is a list of blocks, and a turn can carry thinking, text and a tool request together, so indexing content[0] and assuming text is a bug waiting for the day thinking is switched on. And stop_reason decides whether the body is worth parsing at all. Two values in particular invalidate a structured response even though the request returned 200: max_tokens means the output was cut off mid-object, and refusal means the model declined and the reply does not follow your schema. In both cases tokens were charged and a naive parse throws — or worse, half-succeeds.

What arrives	Why	Handle it by
Truncated JSON	`stop_reason` is `max_tokens`	Retry with a higher limit; never parse the fragment
Prose, not schema	`stop_reason` is `refusal`	Route to a human; do not retry identically
`"Fraud"` where you expected `"fraud"`	Enum casing can vary	Compare case-insensitively
Valid JSON, impossible value	Ranges are not enforced	Range-check after parsing
Well-formed JSON, unknown id	The model filled a required field	Check references against your own data
A partial object mid-stream	Streaming deltas are fragments	Accumulate, then parse at the end (2.3)

Then the parse itself. Decode inside a try, treat a decode failure as an expected outcome rather than an exception to log and forget, and validate the decoded object against your own model — a Pydantic model, a Zod schema, a hand-written check — before anything downstream touches it. The retry worth having is a narrow one: send the response back with the specific validation error and ask for a correction, once. An unbounded retry loop against a model that keeps making the same mistake is how a cheap request becomes an expensive one.

The parse that survives contact with productionpython

if resp.stop_reason == "max_tokens":
    raise Truncated("output was cut off; raise max_tokens and retry")
if resp.stop_reason == "refusal":
    return escalate_to_human(resp)

text = "".join(b.text for b in resp.content if b.type == "text")

try:
    data = Invoice.model_validate_json(text)   # types + your own rules
except ValidationError as e:
    return retry_once_with_error(text, e)      # one correction, then stop

# Rules a schema cannot express, checked here and only here:
assert data.total >= 0
assert data.currency.upper() in ALLOWED_CURRENCIES
assert supplier_exists(data.supplier_id)

Worked example · A logistics firm extracts 4,000 invoices a night

A freight company reads supplier invoices into its ledger. The first version prompted for JSON and called json.loads on content[0].text. It ran clean for three weeks, then began failing in ways that were individually rare and collectively constant.

Failure one: the preamble. Roughly one invoice in four hundred came back with a sentence before the JSON. Moving to a schema with output_config removed this class entirely.
Failure two: the long invoice. A 40-page consolidated invoice hit the output limit; the JSON was cut off mid-array and the parse threw at 3am. Now stop_reason of max_tokens is checked first and the job retries with a higher ceiling.
Failure three: the negative total. A credit note produced total: -1450. The schema declared "minimum": 0, which is not enforced, so it parsed cleanly and posted a negative payable. A range check in code now catches it and routes credit notes down a different path.
Failure four: the confident supplier. For a scanned invoice with a smudged header the model returned a plausible supplier id that belonged to a different company. Nothing in the JSON was malformed. A lookup against the supplier table now rejects unknown ids.
Failure five: the casing. "GBP" was occasionally "gbp". Comparison is now case-insensitive, as the documentation advises for enums.
What shipped. Schema-constrained output, a stop_reason gate, a typed model, four business-rule checks, and a review queue for anything that fails them — about forty lines in total.

Takeaway: The schema removed one failure mode out of five. The other four were only ever going to be caught by code that treated the response as untrusted input.

Scepticism toward confident output

The phrase the exam guide uses is worth taking literally. A language model's fluency is unrelated to its accuracy: it produces the same measured, well-organised prose whether the document said what it claims or not. There is no tremor in the voice. That means confidence conveys no information, and any design that uses tone as a quality signal — including a human reviewer skimming because it reads well — has no quality signal at all.

The documentation's own hallucination-reduction techniques are the practical response, and each of them is a design decision rather than a wording trick. Allow the model to say it does not know, explicitly, so that admitting uncertainty is a permitted answer rather than a failure it must avoid. Ground answers in direct quotes: for long documents, have it extract the relevant passages word for word before doing the task. Require citations so each claim can be traced, and instruct it to retract any claim it cannot support with a quote. Restrict it to the provided material, stating that its general knowledge is not to be used. For higher stakes, chain-of-thought verification exposes the reasoning, and best-of-N runs the same prompt several times — disagreement between runs is a usable signal that something is being invented. The page ends with the caveat that matters most: these techniques significantly reduce hallucinations but do not eliminate them, so critical information must still be validated.

Checking a summary before it goes to the customer

Passes: Output parses and matches the schemaautomatic
Passes: Every figure appears in the source documentquote check
Fails: Policy numbers exist in our recordsone is invented
Check: Model was allowed to answer “unknown”prompt forces a value
Missing: Reviewer sees the cited passagesUI shows the summary only

Three of these are automatic and two need a person. The design question is not “is the model trustworthy” but “which checks are cheap enough to run every time”.

Design scepticism into the product, not just the prompt. Surface the citations next to the claim so a reviewer can check in two seconds rather than two minutes. Make “unknown” a first-class value your schema and your UI both accept. Route low-confidence or check-failing cases to a person instead of to a database. And keep the reviewer's job small: a human asked to verify a hundred fluent paragraphs a day will approve them, whatever they were told.

Worked example · A hospital trust summarises referral letters

A trust pilots an assistant that summarises GP referral letters into a triage note. Clinicians rate the summaries as excellent. In audit, two summaries contain a medication the letter never mentioned — a drug commonly co-prescribed with one that was mentioned.

Recognise the failure type. Nothing malformed, nothing implausible, no error anywhere in the logs. A plausible completion is the hardest failure to catch precisely because it looks like the good output.
Restrict the source. The prompt now states that only the referral letter may be used and that general medical knowledge must not fill gaps.
Quote before summarising. The model extracts verbatim passages for medications, history and the referral reason, then writes the note from those passages only.
Cite in the interface. Each line of the triage note links to the quoted passage; clicking a medication shows the sentence it came from. The invented drug has no passage to show.
Permit “not stated”. The schema allows not_stated for every clinical field, and the prompt says explicitly that this is the correct answer when the letter is silent.
Keep the clinician accountable. The note is a draft that a clinician confirms; the audit now samples confirmed notes weekly rather than trusting the pilot rating.

Takeaway: Every fix was structural: restrict the sources, force quotation, show provenance, allow “unknown”, keep a human decision. None of it depended on the model being more careful.

Traps the wrong answers are built from

Tempting but wrong	Do this instead
Calling `json.loads` on `content[0].text` and catching nothing	Check `stop_reason` first, collect text blocks explicitly, and treat a decode failure as an expected branch.
Trusting `minimum`, `maximum` or `pattern` in a schema to enforce a rule	Those keywords are not enforced; re-check ranges, lengths and formats in your own code after parsing.
Treating a 200 response as a successful answer	A refusal or a `max_tokens` truncation returns 200 and is charged; both need handling before the body is used.
Retrying a failed parse in an unbounded loop	Retry once with the specific validation error attached, then escalate to a human or a fallback path.
Forcing a value for every field so the output is always complete	Make “unknown” or “not stated” a legal value, so the model can decline instead of inventing.

You should now be able to

Choose between prompt-and-parse, schema-constrained output and strict tool use for a given requirement.
Write a schema within the supported subset and name the keywords that are not enforced.
Gate parsing on stop_reason and handle refusal, truncation and enum casing.
Layer business-rule validation on top of schema conformance, and design the escalation path.
Apply grounding techniques — quotes, citations, source restriction, permitted uncertainty — to reduce invented content.
Explain why fluent, confident output is not evidence of correctness, and design the review step accordingly.

Output handling