Context engineering is eating prompt engineering

In 2024, the hot skill was prompt engineering. You could charge consulting rates to write system instructions. Companies hired "prompt engineers" and paid them well. The entire discipline of working with large language models orbited around a single question: how do I phrase this so the model does what I want?

Eighteen months later, that skill is table stakes. Every intern can write a decent prompt. The models are better at interpreting ambiguous instructions. The frontier moved.

What replaced it is less glamorous and much harder. Context engineering.

What prompt engineering misses

Prompt engineering optimizes the question. You tweak wording, add examples, clarify constraints. You get better at telling the model what to do.

Context engineering optimizes the conditions under which the question is answered. You design the information architecture around the agent. Which data sources it can see. Which knowledge bases are current. How much context fits in a single turn. What gets retrieved and when. How memory works across sessions. Which tools are available and how their outputs are shaped before the model reads them.

The difference shows up fast in production. A well-prompted agent with bad context will confidently hallucinate. A decently-prompted agent with engineered context will just work.

Salesforce put this plainly in their 2026 agent trends report: "Context engineering replaces prompt engineering as the frontier." Sourcegraph published an entire guide on it. The LangChain State of Agent Engineering survey, with 1,300 respondents, flagged context engineering challenges as a top concern for enterprise teams running agents at scale.

This is not a semantic shift. It is a recognition that production AI agents fail for reasons that have nothing to do with the model.

The token budget you keep ignoring

Every inference call has a finite token budget. The model cannot see more than its context window. Even with 200K-token windows becoming standard, the quality of attention degrades as the context grows. Sourcegraph calls this "context rot": larger contexts produce decreased recall accuracy.

The question is not whether you have a good prompt. The question is what you are spending your token budget on.

A coding agent asked to fix a bug in a million-line monorepo fails not because the model cannot reason about Kubernetes. It fails because grep returns 4,000 hits on the symbol name, burning the entire context window on irrelevant matches. The same agent, pointed at a SCIP-backed code intelligence server, gets one definition and three call sites. The task that timed out at two hours finishes in 89 seconds. Score: 0.90 out of 1.0.

Same model. Same prompt. Different context. The result is not incrementally better. It is categorically different.

This pattern repeats everywhere. A customer support agent with access to the full knowledge base but no retrieval strategy drowns in noise. A research agent pointed at the entire web but given no re-ranking step returns surface-level summaries. A sales agent that pulls every CRM field for every contact burns tokens on data it does not need for this call.

The prompt did not fail. The context budget was wasted.

The four pillars that actually matter

Sourcegraph's guide breaks context engineering into four pillars. I have found this framework to be the most useful way to think about agent architecture.

Instructions. The system prompt still matters. But the failure mode is not "vague instructions." It is instructions that contradict the model's trained heuristics for tool use. Too many rules and the agent gets confused. Too few and it drifts. The Goldilocks zone is narrower than most people think.

Retrieval. External data: RAG, SQL queries, file reads, API calls. This is where most agents die. Bad retrieval is the largest single source of hallucination. The fix is not better prompts. It is just-in-time retrieval: pull data only when the agent needs it, using lightweight identifiers. A vector search returning 50 candidates is not a feature. It is a liability. Re-rank to the top five before the model sees them.

Memory. Short-term is the conversation plus tool results. Long-term is user preferences, project conventions, session summaries. As the window fills, you need a compaction step: condense old turns into a summary. Anthropic's pattern of having the model write a scratchpad as a persistent memory file is the cleanest approach I have seen.

Tools. Executable surface area. Every tool definition costs tokens. Every tool result costs more. The most common failure mode, per Sourcegraph, is bloated tool sets that cover too much functionality. If a human engineer cannot definitively say which tool should be used for a given task, the agent cannot either. Ship small, unambiguous tool sets.

What the data says about production

The LangChain survey is the best snapshot we have of where agent engineering actually stands in mid-2026. 57.3 percent of organizations have agents in production, up from 51 percent the year before. Among large enterprises with more than 10,000 employees, that number is 67 percent.

But production does not mean working well. Quality is the number one barrier, cited by 32 percent of respondents. Cost concerns have dropped as model prices fall. Latency is still real. Security emerges as a top concern at scale. But quality is the constant.

The organizations that are succeeding share a pattern. They are not spending their time on prompts. They are spending it on context: observability, evaluation, retrieval quality, memory design.

Eighty-nine percent of teams have some form of observability. Among those in production, 94 percent. Sixty-two percent have detailed tracing that lets them inspect individual reasoning steps. You cannot fix what you cannot see. And what you need to see is not the prompt. It is the full context assembly pipeline that produced the agent's decision.

Stop obsessing over the prompt

I am not saying prompts do not matter. They do. A bad prompt can absolutely break an agent.

But if your agent is failing in production and your first instinct is to rewrite the system message, you are optimizing the wrong variable. The prompt got the agent to the starting line. The context got it across the finish line.

Before you touch the prompt, ask four questions:

What data is the agent actually seeing on each call? Is every retrieved document earning its token cost? Does the memory strategy survive a 50-turn conversation? Are the tools small enough that the model reliably picks the right one?

The answers to those questions will fix more production failures than a month spent tweaking wording.

The prompt engineering era was about making better requests. The context engineering era is about building better pipelines. The first made demos impressive. The second makes agents survive production.

If you are building agents that have to work reliably, spend your time on the second.