What you need to know
An AI solution is not delivered once. It moves through phases, and the architect's job is to make sure what was learned in one phase shapes the next.
The five phases
| Phase | Main work | Key outputs |
|---|---|---|
| Discovery | Understand goals; translate them into requirements and assumptions | Translation table, requirements, assumption register, success criteria |
| Design | Choose patterns, models, routes; weigh trade-offs | Architecture, decision log, evaluation plan, control register |
| Handoff | Transfer the design and its reasons to builders and operators | Implementation guidance, runbooks, evaluation set |
| Monitoring | Watch signals, SLAs and governance checkpoints | Dashboards, governance table, reviews |
| Iteration | Improve, fix, extend, or re-architect | Changes gated by evaluation; updated documents; outcome measurement |
The phases connect. A residency requirement found in discovery leads to a regional route in design, a recorded rejection of the global endpoint at handoff, a scheduled residency check in monitoring, and constraints on any performance fix during iteration.
Iterate or re-architect?
| Situation | Response |
|---|---|
| Inconsistent output format | Iterate on the prompt |
| Retrieval quality slipping | Tune chunking, filters or index |
| Cost over budget from bloated context | Optimize context and caching |
| Small latency variation within SLA | Monitor |
| A new mandatory approval step changes the workflow | Re-architect that workflow |
| The chosen route cannot meet a new residency obligation | Revisit the deployment architecture |
| A model version is being retired | Plan a migration, re-run the evaluation suite, update prompts if needed |
Diagnose first (lesson 4.4); re-architect only when the cause is structural.
The outcome document
Technical success is not business success. Before expanding a deployment, document the outcome:
| Field | Captures |
|---|---|
| Use case | The business problem and its scope boundary |
| Metric before | The baseline, measured before deployment |
| Metric after | The same metric, measured the same way, after a long enough period |
| Auditable control | The evidence that makes the result trustworthy (for example, approval logs) |
| Measurement owner | The named person responsible for the measurement |
| Reuse potential | Where the solution or its patterns could apply next |
Model and platform lifecycle
Models are versioned and eventually retired. Pin versions in production, track deprecation notices, and keep the evaluation suite ready so a migration is a planned release, not an emergency. The regression suite is what makes this routine.
In the exam
Lifecycle questions ask which phase an activity belongs to, whether a problem calls for iteration or re-architecture, or whether a deployment is ready to expand. Carry constraints through every phase, diagnose before re-architecting, and gate expansion on an outcome document with a real before-and-after measurement, an auditable control and a named owner.
Check your understanding
Four weeks after launch, a sponsor wants to expand a deployment. The "after" metric has two weeks of data and the control logs have not been checked. What should the architect do?
Show answer
Answer: B. Expansion is gated on evidence. Verify the control, assign ownership and complete the measurement before deciding, without inventing results or abandoning the work.
A new regulation requires a person to approve every outbound letter the system sends, which it currently sends automatically. What is the right response?
Show answer
Answer: A. A new mandatory approval changes the workflow itself. That is structural, so it calls for re-architecting that part of the system, not iteration.
Which TWO belong in an outcome document?
Show answer
Answer: B, C. An outcome document ties business metrics before and after to an owner and an auditable control. Token counts and benchmarks are not business outcomes.
Apply it: Write St. Aldric's outcome document
St. Aldric piloted the discharge-summary drafter on two medical wards for eight weeks. The CMIO wants to roll it out hospital-wide next month. Pilot data: average time per signed summary fell from 35 to 14 minutes on the pilot wards (six weeks of clean data); coding returns on pilot summaries fell from 12% to 5% (four weeks of data, because coding lags); 98% of drafts were signed with minor or no edits; no medication discrepancies were found.
- Write the outcome document with all six fields.
- Decide what can be approved now, and what must wait.
Done when: nothing you approve depends on extrapolated data. - List what must be true before each new ward goes live, carrying constraints from earlier phases.
- Name one event that would trigger re-architecture rather than iteration.
Show model answer
1. Outcome document
| Field | Content |
|---|---|
| Use case | Drafting discharge summaries from EHR notes on two medical wards, with physician sign-off; out of scope: surgical, paediatric and psychiatric discharges |
| Metric before | 35 minutes per signed summary; 12% coding returns |
| Metric after | 14 minutes per signed summary (six weeks); 5% coding returns (four weeks, still maturing) |
| Auditable control | EHR signature log linking every sent summary to a signing physician and a timestamp; medication check logs |
| Measurement owner | Clinical operations lead (time); coding team lead (returns) |
| Reuse potential | Other medical wards first; patterns (source-linked drafts, code-inserted medications) for transfer letters and clinic letters |
2. Decision. Approve expansion to the other medical wards, where the case mix matches the pilot and the time saving is well evidenced. Hold surgical, paediatric and psychiatric wards: their documentation differs and was not in the pilot or the evaluation set. Report coding returns again after eight weeks of data before claiming the billing benefit.
3. Before each new ward
- The ward's note types are in the evaluation set, and the release thresholds from lesson 4.2 are met for that slice.
- The BAA-covered route and US-only data flows are unchanged (control register re-checked).
- Physicians on the ward are trained on the review screen, and the signature control is confirmed working for the ward.
- The ward's outcome baseline (minutes per summary, coding returns) is measured before go-live.
4. Re-architecture trigger. A requirement that summaries be sent to primary care within an hour of discharge, which would need a different drafting and review flow, or a change of cloud provider that the current BAA does not cover.