What you need to know
Fairness is not something an organization can hand to its model provider. Even a carefully trained model sits inside a system of data, prompts, examples and routing rules, and unequal treatment can enter at any of them.
Where unequal outcomes enter
| Injection point | How bias gets in | Example |
|---|---|---|
| Data and retrieval | The sources over- or under-represent groups, or encode past unequal decisions | A hiring assistant retrieves past "successful employee" profiles from a period of skewed hiring |
| Prompt framing | Wording carries assumptions | A prompt describing "typical reliable applicants" in ways that correlate with background |
| Examples | Few-shot examples come from one group or reflect historical bias | Every example of an approved application has the same profile |
| Inputs and processing | Some groups' inputs are handled worse | Handwritten or non-native-language submissions extracted less accurately, then treated as incomplete |
| Routing and thresholds | Different groups are sent down different paths | One group's cases escalated for extra scrutiny more often for the same facts |
Measuring only the final answer misses most of these. Fairness has to be checked wherever the system retrieves, interprets, classifies or routes.
Measure it
- Outcome rates by group, where collecting or inferring group data is lawful and done with suitable protections: approval rates, escalation rates, time to decision, error rates.
- Counterfactual tests: run the same case with only a group-related detail changed (a name, an age, a neighbourhood) and check that the outcome does not change when it should not.
- Slice the evaluation set (lesson 4.1) by the groups and input types that matter, and treat a gap as a regression.
Anthropic's own research on discrimination in model decisions uses this kind of approach: systematic variations of the same scenario to reveal differences in treatment.
Transparency and decision logging
If a decision cannot be reconstructed, it cannot be explained or investigated. For each decision, keep:
- the original inputs,
- the retrieved context that influenced it,
- the model's output and the configuration (model, prompt version),
- the routing decision and any human action,
- an identifier linking all of these.
Different audiences need different explanations:
| Audience | Needs |
|---|---|
| The affected person | Why this decision was made about them, in plain language, and how to contest it |
| Regulator or auditor | Evidence that the required process and controls were followed |
| Engineers | Inputs, context, configuration and path, to reproduce and fix |
Discernment
Reviewers and operators should judge outputs rather than accept them: is this decision justified by the evidence, is the same rule applied consistently, does anything suggest the outcome turned on an irrelevant characteristic? This habit, called discernment in Anthropic's AI Fluency framework, is what turns logging into accountability.
In the exam
Fairness and transparency questions describe a decision system and an observed or potential disparity, or a request to explain a decision. Locate the injection point, choose measurement by group or counterfactual testing, and make sure decisions are logged well enough to reconstruct. Distractors rely on the provider's training, a disclaimer, or overall averages.
Check your understanding
A loan pre-screening assistant uses ten few-shot examples of approved applicants, all from the same postcode area. What is the main risk?
Show answer
Answer: D. Examples shape judgment. A set drawn from one group can encode that group as the pattern of success; examples must be representative.
An applicant asks why their application was refused. Logs contain the model's recommendation but not the documents it was given or the rule that routed it. What is missing?
Show answer
Answer: C. Without the inputs, context and routing, the decision cannot be reconstructed, so it cannot be explained or checked for fairness.
Which TWO methods detect unequal treatment?
Show answer
Answer: A, D. Group-level outcome comparison and counterfactual tests reveal differences that averages and self-assessment hide.
Apply it: Check Larkfield's claim prioritization for fairness
Larkfield wants a queue score so the most urgent claims reach adjusters first (see the case). The proposal: Claude reads each claim and scores urgency from 1 to 5 using the description, the attachments and the customer's claim history; the highest scores are assessed first.
- Find the injection points where comparable customers could be treated differently.
Done when: you have at least four, including one in processing and one in routing. - Redesign the scoring to reduce them.
- Define the measurements that would show the score is fair.
- Define the decision log that lets Larkfield explain a customer's queue position.
Show model answer
1. Injection points
- Inputs and processing: handwritten and scanned claims, and claims written in non-native English, are extracted less accurately, so injuries or uninhabitable homes may be missed.
- Prompt framing: "urgency" left undefined lets the model reward articulate, emotional descriptions over terse ones.
- Data: claim history as a factor penalizes customers who have claimed before, unrelated to current urgency.
- Routing: a low score means waiting longer, which compounds any of the above.
- Examples: if examples of urgent claims come from one channel (web forms), others look less urgent.
2. Redesign. Define urgency as explicit criteria (injury, uninhabitable home, vulnerable occupant, vehicle needed for medical travel), extracted as structured yes/no fields with the supporting text quoted. Code computes the score from those fields, not the model. Claim history is excluded. Low-confidence extractions are flagged for a quick human check rather than scored low. Examples cover every channel.
3. Measurements. Time to first adjuster contact and score distributions by channel (email, web, post) and by language of submission; counterfactual tests rewriting the same claim tersely, emotionally, handwritten and typed, which should get the same score; and the share of truly urgent claims (judged by adjusters afterwards) that were scored urgent, by channel.
4. Decision log. Claim ID, inputs and attachments references, extracted urgency fields with quoted evidence, extraction confidence, the score and the rule version that computed it, queue position over time, and any human change with its reason. Customer-facing explanation: which urgency criteria applied and how to tell Larkfield if something urgent was missed.