CCAO-F 7.1 — Diagnosing poor outputs

The troubleshooting loop

Name the gapWhat exactly is wrong with this output?
Classify the causeMissing facts, ambiguity, shape, size or drift
Apply one fixThe cheapest one that matches
CompareIs this version better than the last?

Still wrong? Back to the top with what you learned

The loop is short on purpose. Change one thing, look at the result, and keep whichever version was better — guessing at three changes at once tells you nothing.

Five causes, five different fixes

Nearly every underperforming prompt traces back to one of five causes. They look similar from the outside — the answer is bad — and they respond to completely different treatments, which is why the sympathetic-sounding advice to “add more detail” fails so often.

Symptom	Likely cause	The fix
Confident facts that turn out to be wrong	Claude never had the information	Attach the document; ask it to quote before it concludes
Generic, could-be-about-anyone output	No audience, purpose or context given	Say who it is for, why, and what good looks like
Right content, unusable shape	Format was never specified	State the format you want, not the one you don’t
Some instructions followed, others dropped	Too many requirements in one request	Split into steps; check each before moving on
Quality falls away late in a long chat	The thread has drifted or grown unwieldy	Start a fresh conversation with a clean brief

The first cause is the one to internalise. Anthropic’s help documentation is direct about it: Claude can hallucinate, can produce fabricated quotations that read as authoritative, and can be out of date on recent events because of its training cutoff. No amount of rephrasing fixes an answer about a document Claude has never seen. The fix is not linguistic; it is to put the source in the conversation.

Which fix does this failure call for?

What is wrong with the output?

Facts are invented
Supply the sourceAttach it; ask for quotes before conclusions
Bland or off-target
Add contextAudience, purpose, constraints, two examples
Wrong shape or length
Specify the formatSay what to produce, positively stated
Instructions dropped
Split the taskOne output per step, checked in turn

Ask what kind of wrong it is before you touch the prompt. The commonest mistake is applying the second branch’s fix to the first branch’s problem.

The cheap fixes, in the order to try them

Be explicit, and test it on a colleague. The documentation offers a memorable check: show your prompt to someone with minimal context on the task and ask them to follow it — if they would be confused, so will Claude. Where the order or completeness of steps matters, give them as a numbered list. And explain why a rule matters rather than just stating it; the documented example contrasts a bare “never use ellipses” with the same instruction plus the reason that the text will be read aloud by a speech engine that cannot pronounce them.

Say what you want, not what you don’t. “Do not use markdown” performs worse than describing the output you actually want — smoothly flowing prose paragraphs, say. You can also name the format with a tag, and the style of the prompt itself pulls the answer along: a prompt written in plain paragraphs tends to produce plain paragraphs.

Show examples. Three to five examples, wrapped in tags so they are unmistakably examples, is the documented guidance. They should be relevant to your real case, diverse enough to cover edge cases, and consistently structured. For anything where “good” is a matter of house style rather than of instruction — tone of voice, how a case note is written, what a decent summary looks like here — two or three real past examples beat a paragraph of adjectives.

Give the work structure. Wrapping distinct parts of a long prompt in descriptive tags — instructions, context, the input itself — helps Claude tell them apart when the prompt mixes all three. And with a long document, put the document above the question rather than below it; the documentation reports this ordering can improve performance substantially on long inputs.

Diagnosed, then rewritten

Vague — will disappointtext

Write a summary of our
Q3 performance for the
board.

Diagnosed and rewrittentext

Attached: the Q3
management accounts.

Audience: nine
non-executive directors,
not finance people.

Write 400 words in four
short paragraphs:
result, what drove it,
what worsened, what we
are doing next.

Every number must come
from the attachment,
with the line quoted. If
a figure is not there,
write "not in pack".

Four changes, each answering a named failure: no audience, no source, no format, no length. The rewrite is longer because each line replaces a guess.

Worked example · Blog copy that could be about anybody

A marketing lead at a B2B logistics software company asks for a blog post on reducing warehouse costs. What comes back is fluent, plausible and completely generic — the sort of post any of their competitors could have published.

Name the gap. It is not wrong. It is unattributable: nothing in it could only have come from this company.
Classify. This is the second cause — no audience, no purpose, no house voice, no evidence. Rephrasing will produce a different generic post.
Fix one: context. Who it is for (operations directors at mid-sized third-party logistics providers), what it must do (support a specific campaign), what it must not do (make claims the product team has not approved).
Fix two: examples. She pastes three of their best-performing past posts as examples of house voice, wrapped so Claude can tell them apart from the instructions.
Fix three: evidence. She attaches two anonymised customer results and requires every quantitative claim in the post to trace back to them.
Compare. She reads the new draft against the old one on one question — could a competitor have published this? — and keeps the winner.

Takeaway: Generic output is almost never a model problem. It is the honest consequence of a request that contained nothing specific to answer with.

When the problem is invented facts

Hallucination has its own toolkit, and it is worth knowing by name because the exam asks about it. Give Claude explicit permission to admit uncertainty — a line telling it to say it does not have enough information to assess something confidently, rather than guessing. For a long document, ask it to extract exact quotes relevant to the question first, and to say so plainly when no relevant quotes exist, before it does anything else with them. Require citations: for each claim, a direct supporting quote, with any claim that cannot be supported removed.

Beyond that there are three heavier techniques. Ask for step-by-step reasoning before the answer, which exposes faulty logic you can see and challenge. Run the same prompt more than once and compare — inconsistency between runs is a strong signal something is being invented. And feed an output back as input to be verified or expanded, which catches contradictions. You can also restrict Claude explicitly to the documents provided rather than its general knowledge. The documentation closes with the caution that matters most: these techniques significantly reduce hallucination but do not eliminate it, so critical information still gets validated, especially for high-stakes decisions.

Worked example · A figure that was never in the report

A finance analyst attaches a 60-page supplier performance report and asks for the top five risks. The summary states that on-time delivery fell to 82%. His director asks where that came from. It is nowhere in the report.

Name the gap. One specific fabricated number inside an otherwise sound summary — the most dangerous failure mode, because everything around it is right.
Classify. Not ambiguity, not format. The summary was produced without being anchored to the text.
Fix: quote first. He re-runs it as two steps. Step one: extract the exact sentences in the report bearing on delivery performance, and say “no relevant quotes found” if there are none. Step two: write the risk summary using only those extracts.
Fix: cite each claim. Every risk in the final summary carries the quoted sentence it rests on, so the director can check it in seconds.
Verify anyway. He spot-checks three quotes against the PDF, because the documentation is explicit that these techniques reduce rather than remove the risk.

Takeaway: Quote-then-answer turns an unverifiable summary into one anybody can audit in a minute. Deciding which outputs deserve that level of checking is covered in 2.4.

Traps the wrong answers are built from

Tempting but wrong	Do this instead
Retyping the same request more forcefully	Name the failure first, then apply the one fix that matches it.
Adding examples to fix a factual error	Attach the source and require quoted evidence for each claim.
Writing “do not do X” instructions	Describe the output you want; positive instructions outperform prohibitions.
Changing four things at once and declaring victory	Change one element, compare with the previous output, keep the better one.
Trusting an anti-hallucination technique as a guarantee	Treat it as risk reduction and still validate critical information.

You should now be able to

State precisely what is wrong with an output before changing anything.
Map a symptom to one of five causes and choose the matching fix.
Apply the colleague test and positive format instructions to a weak prompt.
Use quote-first and citation techniques against fabricated content.
Iterate by changing one element at a time and comparing versions.

Diagnosing poor outputs