For most of 2023 and 2024, the skilled part of building with LLMs was assumed to be the prompt. Write the right instructions, add the right examples, and the rest was plumbing. That framing quietly expired. The term that replaced it, "context engineering," now shows up in vendor documentation, conference talks and job descriptions, and it describes a real shift in where applications fail and where engineers actually spend their time.
This is an explainer of how that happened: where the term came from, the research that justified it, the techniques it covers, and what it changes for teams building agents.
The term arrived fast, and from several directions
The word itself is older, but its takeover in AI engineering dates to a specific stretch of weeks in mid-2025:
- On June 19, 2025, Shopify CEO Tobi Lütke wrote on X that he preferred "context engineering" to "prompt engineering", defining it as "the art of providing all the context for the task to be plausibly solvable by the LLM."
- Days later, Andrej Karpathy amplified it with the definition the field mostly uses now: context engineering is "the delicate art and science of filling the context window with just the right information for the next step," listing its components as task descriptions, few-shot examples, RAG, tools, state and history, and compaction (Karpathy on X).
- On July 2, LangChain published a widely cited taxonomy of the discipline around four verbs: write, select, compress and isolate context.
- On July 14, Chroma's Context Rot report gave the field its most uncomfortable empirical result (below).
- On July 18, Manus published lessons from building its agent on what the discipline costs in production.
- In September, Anthropic's Applied AI team published Effective context engineering for AI agents, which functioned as the field's arrival notice: "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference."
Cognition's Walden Yan was arguing the same position a week before the Lütke and Karpathy posts, in a piece telling teams not to build parallel multi-agents precisely because they fragment context. Three independent groups landing on one definition in the same month is a sign the term named something engineers were already feeling, not a coinage in search of a use.
Why the prompt stopped being the hard part
The shift tracks a change in what applications do. Prompt engineering optimized for one-shot calls: a classification, a generation, a question. Agents run in loops. Each turn adds tool calls, tool results, retrieved documents and intermediate reasoning to the message history, so the context a model sees on turn 40 is not the context you wrote. It is whatever accumulated, unless someone curates it.
Anthropic frames the problem as one of scarce resources: models have an "attention budget," and "context, therefore, must be treated as a finite resource with diminishing marginal returns." The architectural reason is the transformer's all-to-all attention, which creates n-squared pairwise relationships among n tokens; as input grows, that structure gets stretched, and training data skews short, so models have less experience with long-range dependencies.
The strongest evidence that bigger windows did not solve this is Chroma's July 2025 study. The team tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, on deliberately simple tasks (word replication, controlled retrieval, conversational question answering) where only the input length varied. Performance "grows increasingly unreliable as input length grows" even there, and degrades faster when the needle matches the question semantically rather than lexically, or when distractors are present. Near-perfect Needle-in-a-Haystack scores had created a false sense that long context was solved; NIAH mostly tests lexical matching, which is the easy case.
The practical consequence: a 200,000-token window is not 200,000 tokens of usable working memory. Cramming more in is a cost and a quality risk, not a fix.
The four moves, and what production adds

LangChain's four verbs organize most of what people actually do (repo):
- Write. Persist state outside the window: scratchpads, files, notes the agent reads back later. Anthropic's post highlights structured note-taking, where an agent keeps a to-do or findings file that survives context resets, and its Claude-plays-Pokemon demo runs multi-hour strategies this way.
- Select. Pull the right tokens in at the right time: retrieval, but increasingly "just in time" loading, where an agent holds lightweight references (file paths, queries, links) and loads data through tools only when needed, the way Claude Code reads databases with targeted queries instead of loading full tables.
- Compress. Shrink what stays: summarizing or pruning the message history, clearing stale tool results, compacting a long conversation into a summary that seeds the next window.
- Isolate. Split contexts so one agent's flood is not another's: sub-agents that explore with their own clean windows and return condensed summaries, often 1,000 to 2,000 tokens from tens of thousands of tokens of work.
Production practice adds constraints the taxonomies tend to skip, and the Manus write-up is the clearest public account. Their claim: the KV-cache hit rate is "the single most important metric for a production-stage AI agent," because agent loops are wildly input-heavy (their average input-to-output ratio is around 100:1) and cached tokens are an order of magnitude cheaper (they cite Claude Sonnet at $0.30 versus $3.00 per million input tokens). That turns context engineering into cache discipline: keep the prompt prefix stable (a timestamp at the top of a system prompt silently invalidates everything after it), treat the context as append-only with deterministic serialization, and mask tools rather than removing them mid-run, because changing tool definitions breaks both the cache and the model's references to earlier actions.
Anthropic has since productized parts of the same idea. Its context-management announcement reports that on a 100-turn web-search evaluation, automatic context editing cut token consumption by 84%, and that memory tool plus context editing improved agent performance 39% over baseline (Claude context management). Same model, same task; the gains came from curating what it saw.
Prompt engineering is a subset, not a casualty
The breathless version of this story is that prompt engineering died in 2025. The people who popularized the replacement term did not say that. Karpathy's own component list includes task descriptions and few-shot examples, which is prompt engineering, verbatim. Anthropic calls context engineering "the natural progression of prompt engineering," not its abolition. The scope grew: writing the instruction is now one row in a larger table that also covers tools, retrieval, memory, compaction and sub-agent topology.
For teams, the translation is concrete. The failure modes worth learning are the ones Drew Breunig catalogued in June 2025: context poisoning (a hallucination enters the window and gets referenced), distraction (too much history overwhelms the training), confusion (irrelevant tools or content skew output) and clash (pieces of the window contradict each other). Auditing an agent means being able to say what is in its window at each step and why, and treating every token as something that must earn its place.
The model is the least controllable part of an AI application. The context is the part you own. That, more than any tweet, is why the discipline took the name it has.
Sources
- Effective context engineering for AI agents, Anthropic
- Context Rot: How Increasing Input Tokens Impacts LLM Performance, Chroma
- Context Engineering for AI Agents: Lessons from Building Manus, Manus
- Andrej Karpathy on context engineering, X
- LangChain context engineering taxonomy and notebooks, GitHub
- Managing context on the Claude Developer Platform, Anthropic
- Don't Build Multi-Agents, Cognition
