What you need to know
Why self-review is weak
A session that generated code still holds the reasoning that led to it. Asked to review, it tends to agree with itself: the same assumptions that produced a bug make the bug look correct. Adding "review critically" or turning on extended thinking in the same session helps less than you would hope.
An independent instance, a fresh session that sees the code but not the generator's reasoning, catches more subtle issues.
| Design | Catches subtle issues? |
|---|---|
| Same session, "now review your code" | Rarely |
| Same session with more thinking | Somewhat |
| Independent instance with the diff and the standards | Best of the three |
Multi-pass review for large changes
A single pass over a 25-file change spreads attention thin: some files get skimmed, and findings can contradict each other. Split the work:
- Per-file local passes: one file at a time, for local bugs, error handling and security issues.
- Integration pass: across files, for data flow, interface mismatches and assumptions one file makes about another.
Confidence for routing
Ask the reviewer to report a confidence with each finding in a verification pass. High-confidence findings can post automatically; low-confidence ones can go to a person or a second check. Calibrate the threshold against findings you have already labelled (lesson 5.5).
In the exam
Look for a generator approving its own bug, a big review with inconsistent findings, or a need to route findings by reliability.
Check your understanding
The session that wrote a fix also reviews it and approves an off-by-one error an independent reviewer catches. Why?
Show answer
Answer: A. The generating session carries its own assumptions into the review. An independent instance does not.
Which design catches the most subtle issues in generated code?
Show answer
Answer: C. An independent instance has no access to the generator's reasoning, so it judges the code on its own merits.
A 25-file review is inconsistent and misses issues in some files. What helps? Select TWO.
Show answer
Answer: A, C. Per-file passes give every file full attention, and a dedicated integration pass handles the issues that span files.
Apply it: Keystone's review architecture
Design Keystone's review architecture for pull requests of any size.
- Separate generation from review. Describe which instance does what.
- Define the passes for a pull request touching 18 files across three services.
- Add confidence routing: what posts automatically and what goes to a person.
Show model answer
1. Instances. The test and fix generator runs in one job. Review runs in a separate claude -p instance that receives the diff, the CLAUDE.md standards and prior findings, never the generator's conversation.
2. Passes
- One local pass per changed file (18 passes, run in parallel), each returning findings in the shared schema.
- One integration pass per service boundary crossed (3 passes): interfaces, data shapes and error handling between services, given the per-file summaries.
- A merge step removes duplicates and resolves contradictions between passes.
3. Routing. Each finding carries a confidence. Security findings post whatever their confidence, flagged for the security engineer if low. Other findings post automatically above a threshold calibrated on 200 labelled past findings; those below it go to a weekly triage instead of the PR.