What you need to know
Prompt engineering is about the instructions. Context engineering is about everything else the model sees: which documents, tool results and history are in the window, when they arrive, and how long they stay. It drives quality, cost and latency at the same time.
What fills the window
Every call carries the system prompt, tool definitions, the conversation so far, retrieved documents, tool results and the space needed for the answer (plus thinking, if enabled). Anything outside the window does not exist for that call.
A bigger window is not automatically better. More tokens cost more and take longer, and long, cluttered context makes the relevant detail harder to find. The goal is the smallest set of high-signal tokens that lets the model do the step in front of it.
Four things that are easy to confuse
| Concept | What it is | Who owns it |
|---|---|---|
| Context window | What the model can see on this call | The request you build |
| Retrieval | Fetching relevant knowledge at the time of the question | Your retrieval layer |
| Application state | The authoritative record: orders, cases, patients | Your systems of record |
| Memory and summaries | Selected facts carried between calls or sessions | Your application (or a memory tool you provide) |
The API does not remember previous calls. If a conversation, a decision or an identifier needs to persist, your application stores it and sends it again.
Four strategies
| Strategy | Idea | Fits | Main risk |
|---|---|---|---|
| Monolithic | Load everything needed up front | Bounded tasks with predictable inputs that need all the material at once | Cost and clutter as content grows |
| Progressive | Load only what the next step needs | Multi-step workflows, agents, long conversations | Dropping something an earlier step needed |
| Retrieval | Fetch relevant pieces at question time | Large or changing knowledge, citations | Retrieval misses the right passage |
| Compaction | Summarize accumulated history and continue | Long-running agents and sessions near the limit | Losing exact identifiers, numbers and decisions |
Real systems combine them. A coding agent might start with a full task description, add files progressively as it explores, retrieve documentation on demand, clear old tool results it no longer needs, and compact its history after many turns.
Two different ways to run out of room
- The request is too large to send. The prompt is over the model's token limit (rejected as an invalid request) or the request body is over the byte limit (rejected as too large). Nothing is generated.
- The request fits, but the answer runs into the ceiling. Generation stops early with a stop reason saying the context window was exceeded, and the output is cut off.
Prevention is mostly measurement: count tokens before sending large requests, read the usage figures on every response, reserve room for the output, and trim what does not earn its place (old tool results, repeated instructions, whole documents where a section would do).
In the exam
Context questions describe long documents, long sessions or growing agents, often with a symptom: the request is rejected, answers are cut off, answers get worse late in a session, or costs climb. Identify which of the two failure modes it is, pick the strategy that fits the data and the task, and look for compaction that preserves identifiers. Be suspicious of options that only enlarge the window or rely on the model's memory.
Check your understanding
A long report generation stops mid-sentence, and the response reports that the context window was exceeded. What happened?
Show answer
Answer: D. The request was accepted, so the input fit. Generation then hit the window ceiling, which is reported as its own stop reason with truncated output.
A support agent forgets the customer's order number after compacting a long conversation. What is the best fix?
Show answer
Answer: C. Exact identifiers must survive. Holding them in structured state, and instructing the compaction step to keep them verbatim, solves the loss without giving up compaction.
Which TWO statements about context are true?
Show answer
Answer: A, D. Your application carries state between calls, and stale tool results can be cleared to keep the window focused. More context is not automatically better, and retrieval fetches knowledge while application state is the authoritative record.
Apply it: Plan context for discharge summaries
St. Aldric's discharge-summary drafter must handle typical stays of 30,000 tokens and long stays of up to 140,000. Design how the drafter fills its context window.
- Classify the inputs: which go in whole, which are fetched as needed, and which come from systems of record rather than notes.
- Design the approach for long stays, so quality does not drop on the patients with the most complex care.
Why: long stays are where summaries matter most and where cluttered context hurts most. - Decide what must be carried verbatim between steps, and where it is stored.
- Add the checks that prevent the two out-of-room failures.
Show model answer
1. Inputs
| Input | Handling |
|---|---|
| Admission note, discharge-day progress note | Loaded whole: always needed |
| Daily progress notes | Progressive: summarized by day, then the day summaries are used |
| Consult and procedure reports | Loaded whole when referenced in the hospital course |
| Discharge medications | From the pharmacy system, inserted by code; never extracted from notes |
| Pending results, follow-up appointments | Queried from the EHR and scheduling system at draft time |
2. Long stays. A two-stage map-reduce: first, one call per day of notes produces a structured day summary (events, decisions, changes in treatment, with note references). Second, the drafter writes the hospital course from the day summaries plus the admission and discharge notes. Every call stays well under the window, and each statement keeps a reference back to its source note for the physician's review.
3. Verbatim facts. Patient identifiers, allergies, procedure names and dates, and the references to source notes are kept in structured state and passed to each call as data; the summaries are told to copy them exactly. Medications never pass through a summary at all.
4. Checks. Count tokens before each call and split further if a day's notes are unusually long; reserve space for the 1,500-token summary; monitor usage and the context-exceeded stop reason; and route any stay that still fails to a physician-written summary rather than a truncated draft.