CCAR-P 6.5 — Supporting the solution lifecycle

The lifecycle of a Claude solution

Discoveryproblem, stakeholders, success criteria
Designoptions, decisions, test set
Handoffdocs, training, support window
MonitoringSLIs, eval runs, feedback
Iterationchange, re-test, release

Evidence from monitoring → re-enter design for the next change

The evaluation set born in discovery travels through every phase. Monitoring feeds iteration, which loops back into design — never straight into production.

What each phase must produce

Each phase has an exit criterion — something written and agreed that lets the next phase start safely. Earlier lessons in this domain cover the craft of individual phases: structured discovery (6.1), communicating decisions (6.2), feedback loops and SLAs (6.3) and documentation (6.4). This lesson is about the sequence: knowing where you are, what is owed before moving on, and what to do when a later phase reveals a problem from an earlier one.

Phase	Architect’s job	Exit criterion	Common failure
Discovery	Frame problem, map stakeholders, gather requirements	Signed success criteria and an initial test set	Jumping to a demo
Design	Compare options, decide, record ADRs	Chosen design meets criteria on the test set	Choosing the most complex option
Handoff	Docs, training, rehearsal, support plan	Client team changes and deploys it unaided	A document dump on the last day
Monitoring	Dashboards, eval runs, review cadence	Agreed SLOs tracked and reviewed	Watching uptime but not quality
Iteration	Triage evidence, change, re-test	Every change passes the regression set	Editing prompts live in production

The thread that runs through all five is the evaluation set. Anthropic’s evaluation guidance recommends task-specific tests that mirror real use, include edge cases and can be graded automatically where possible. Started in discovery, it decides between options in design, defines “done” at handoff, is re-run on a schedule in monitoring, and guards every change in iteration. An engagement without one has no way to prove any phase is finished.

Handoff done properly

Handoff is the phase most often squeezed. A clean one has four parts: the documentation pack (6.4), training for the people who will operate and change the system, a rehearsal in which the client team makes a real change end to end, and a defined support window with named contacts and an exit date. Ownership should be explicit — who approves prompt changes, who watches the dashboard, who talks to Anthropic about capacity or model changes.

A handoff week, message by message

Partner architect

Client engineers

CI & eval suite

Client operations

Walk through docs, ADRs, runbook

Change a prompt, run evals

Pass: 95% vs 93% target

Deploy with rollback plan

Confirm alerts and dashboard work

Sign handoff; start support window

The client team proves it can change and ship the system before the partner steps back into a support role.

Worked example · Handing a hospital’s letter-drafting tool to its digital team

A partner firm built a discharge-letter drafting tool for a hospital. The hospital’s five-person digital team will own it after a go-live in March.

Agree the exit criterion early. In design, the hospital and partner agree handoff is complete when the digital team can ship a prompt change through the eval suite without help.
Train by role. Clinicians learn how to review and correct drafts; the digital team learns the architecture, the eval suite and the runbook; the privacy officer reviews data flows.
Rehearse. The digital team adds a new letter template, runs the 150-case suite, and deploys to a pilot ward while the partner watches.
Set up monitoring before leaving. Dashboards track latency percentiles, draft acceptance rate and clinician edit rate; a monthly eval run is scheduled.
Define support. Six weeks of partner support with named contacts, then a quarterly review for the first year.

Takeaway: The handoff was a planned phase with its own success test, not a last-day document dump. The hospital could change the system the week the partner left.

Monitoring and iteration in production

Monitoring a Claude system watches two kinds of signal. Operational ones — latency percentiles, error and rate-limit rates, cost per task — follow the SLO practice described in 6.3. Quality ones — scheduled eval runs, user ratings, human-review override rates, escalations — tell you whether the answers are still good. Iteration then turns that evidence into changes: a prompt revision, new retrieval content, a tool change, a different model. Anthropic’s agent guidance stresses measuring performance and iterating, and adding complexity only when it demonstrably improves outcomes; iteration is not an excuse to grow the system.

Some iteration is forced on you. Claude models move through documented lifecycle stages — active, legacy, deprecated and retired — and requests to a retired model fail. Anthropic’s deprecations page says publicly released models get at least 60 days’ notice before retirement, notifies customers with active deployments by email, and recommends auditing usage in the Console and testing the replacement well before the retirement date. Partner platforms such as Amazon Bedrock and Google Cloud set their own schedules. A mature lifecycle plan treats model upgrades as routine iteration with an owner, not an emergency.

A model deprecation notice arrives

Does the replacement pass our eval suite?

Passes all agreed targets
Plan migrationstaged rollout, monitor closely
Passes with prompt changes
Iterate, re-testrecord the change in an ADR
Misses a target
Escalate earlyoptions + trade-offs to owner
No eval suite exists
Build one nowyou cannot judge the switch

With at least 60 days’ notice the suite decides the path. Without a suite, the notice becomes a guessing game.

Worked example · A retailer’s search assistant after a deprecation email

A retailer’s product-search assistant runs on a Claude model that has just been announced as deprecated, with a retirement date about three months away.

Audit. The operations lead exports usage from the Console and finds the model is also used by an internal reporting script nobody had listed.
Test. The team runs the 400-query eval suite on the recommended replacement. Relevance improves; answer length grows, pushing cost per query up 12%.
Iterate. A prompt change asking for concise answers brings cost back within budget while keeping relevance above target. The change is recorded in an ADR.
Roll out. The team migrates 10% of traffic, watches latency, cost and ratings for a week, then completes the switch a month before retirement — including the reporting script.

Takeaway: Because the eval suite and ownership existed, a forced model change became a two-week piece of planned work.

Traps the wrong answers are built from

Tempting but wrong	Do this instead
Treating go-live as the end of the engagement.	Plan monitoring, a review cadence and iteration ownership as phases in their own right.
Handing over documents on the last day and leaving.	Train by role, rehearse a real change, and run a defined support window.
Monitoring uptime and latency but not answer quality.	Schedule eval runs and track quality signals such as override and escalation rates.
Making prompt or model changes directly in production.	Route every change back through design and the regression set before release.
Reacting to model deprecation notices as emergencies.	Audit usage, test the replacement against the eval suite early, and migrate in stages.

You should now be able to

Name the five lifecycle phases and the exit criterion each must meet.
Keep one evaluation set in use from discovery through iteration.
Plan a handoff with documentation, role-based training, rehearsal and a support window.
Define monitoring that covers both operational and quality signals.
Handle a model deprecation as routine iteration using the documented lifecycle stages and notice period.

Supporting the solution lifecycle