What you need to know
The architect's job is not to pick the most powerful model. It is to pick the model that meets the application's quality requirement within its cost and latency budgets, and to prove it.
Three tiers, three trade-offs
Claude models come in tiers that trade capability against speed and price.
| Tier | Strength | Typical work |
|---|---|---|
| Smallest and fastest (Haiku family) | Lowest latency and cost per token | Classification, routing, simple extraction, high-volume short tasks |
| Balanced (Sonnet family) | Strong reasoning at moderate cost | Most enterprise work: drafting, analysis, coding, tool use |
| Most capable (Opus family) | Deepest reasoning on hard problems | Complex analysis, difficult multi-step agents, tasks where the balanced tier measurably falls short |
These are starting points, not rules. The right answer for a workload is the one the evaluation supports.
Four questions decide the tier
- How hard is the reasoning? Sorting tickets into five categories is not the same as weighing contract clauses against each other.
- How costly is a mistake? Nuanced judgment may justify a stronger model. Factual or numerical precision usually does not: no tier replaces an authoritative data source or a calculation in code.
- What is the latency budget? A user waiting in a chat window has a hard ceiling; an overnight batch does not.
- What does it cost at volume? A small difference per request becomes large at hundreds of thousands of requests a month.
Routing by difficulty
When a workload mixes easy and hard requests, one model for everything wastes money on the easy ones or quality on the hard ones. A model-routing design classifies each request first (usually with a small, fast model) and sends it to the tier that suits it. Routing only works when the classifier is reliable and the difference in difficulty is real; otherwise it adds a failure point for no gain.
Thinking is a separate dial
Extended thinking lets a model reason before answering, with an effort level or token budget you control. It can raise quality on multi-step problems without changing tier, but it adds latency and output tokens. If latency is the tightest constraint, turning thinking up may break the budget; if quality is the gap, more thinking on the balanced tier may be cheaper than moving to the top tier.
A model change is a release
Swapping models is a one-line code change and a whole-application behaviour change. Prompts tuned for one model can behave differently on another, even a newer one. Before any switch you need:
- a curated test set of representative inputs with known-good outcomes,
- a grading method (code checks, a rubric scored by a model, or human review),
- an acceptance threshold agreed before the run, plus a rollback plan.
Keep a regression suite for behaviour that already works, so an upgrade that improves one area cannot silently break another.
In the exam
Model questions usually give a workload, a volume, a latency target and a quality concern, then ask what to choose or what to do before switching. Look for answers that tie the tier to a stated constraint, route mixed workloads by difficulty, and gate any change on an evaluation with a threshold set in advance. Distractors switch models straight away to save money, assume a newer model needs no testing, or reach for the largest model without a reason.
Check your understanding
A team wants to move a production summarizer to a cheaper tier next week. What must exist before the switch?
Show answer
Answer: B. A model change is a release. It needs evidence on the team's own workload, judged against a bar set before the results are known, not general benchmarks or a live trial on customers.
A chat assistant meets its quality bar but breaks its 2-second latency target on long questions. Which change addresses the binding constraint first?
Show answer
Answer: A. Latency is the constraint that fails. Thinking and larger tiers both add time, and more examples add input tokens. Reducing thinking or routing to a faster tier targets the failing budget directly.
When does routing requests between model tiers make sense? Select TWO.
Show answer
Answer: B, C. Routing pays off when difficulty genuinely varies and the classifier can be trusted. With uniform requests it adds a failure point, and it never removes the need to evaluate each route.
Apply it: Choose tiers for Corvane's routes
Corvane's routing design from lesson 1.3 has a classifier and five paths. Finance wants every route on the cheapest tier; the support director wants the best model everywhere. Settle it with a plan you could defend.
- Pick a starting tier for each step: the classifier, order-status phrasing, product questions with citations, returns conversations, and the complaint hand-off summary.
Done when: each choice has a one-line reason tied to difficulty, volume or latency. - Design the evaluation that would let product questions move from the balanced to the small tier.
Why: this is the route with the most volume after order status, so it is where a cheaper tier saves the most.
Done when: you have a test set, a grader and a threshold. - Name the rollback trigger you would watch in production after the switch.
Show model answer
1. Starting tiers
| Step | Tier | Reason |
|---|---|---|
| Classifier (all 95,000 contacts) | Small | Five fixed labels, highest volume, must be fast |
| Order status phrasing | Small | The facts come from the order API; Claude only phrases them |
| Product questions with citations | Balanced | Reading manuals and choosing the right passage needs judgment |
| Returns conversations | Balanced | Multi-turn, policy-sensitive, money involved |
| Complaint hand-off summary | Balanced | Must capture the issue accurately for the agent; low volume |
2. Evaluation for moving product questions to the small tier
- Test set: 400 real product questions from the last quarter, stratified by product category, including 60 hard ones (multi-part questions, products with revised manuals).
- Grader: code checks that a citation is present and points to the right manual; a rubric scored by a model for correctness and completeness; a weekly human sample of 50 to calibrate the rubric.
- Threshold, agreed with the support director before running: correctness within 2 points of the balanced tier overall, no drop on the hard subset, citation accuracy at least 98%.
3. Rollback trigger. Escalation rate on product questions rising more than 3 points over the two-week baseline, or CSAT on those contacts falling more than 2 points. Either one returns the route to the balanced tier while the cause is investigated.