Context Engineering for Frontend Developers

Introduction#

There is a phrase that has quietly become the defining challenge of building AI products: “the model is fine, the context is broken.” Teams spend weeks refining prompts, upgrading to more capable models, and tuning temperature settings, only to find the real problem was never the model itself. It was what they were feeding into it.

Context engineering is the discipline of designing what goes into an LLM’s context window, when it goes in, and how it is structured. It has emerged as the successor to prompt engineering as the primary craft of AI product development. Where prompt engineering focused on how to write instructions, context engineering focuses on the full information architecture that surrounds every model call, including the system prompt, conversation history, retrieved documents, tool call results, user state, and everything else that competes for space in a finite context window.

For frontend developers building AI features, this matters more than it might initially seem. Every chat interface, every AI assisted workflow, and every agent driven UI is ultimately a system for populating and managing context. Understanding how that context works at a technical level is what separates AI features that degrade unpredictably from ones that remain accurate and useful at scale.

Understanding the Context Window#

The context window is the total number of tokens a model can see during a single inference call. Everything goes in here: the system prompt that defines the model’s behavior, the conversation history, any documents or data retrieved from external sources, tool definitions, tool call results, and the user’s current message. The model’s response is generated from all of this together.

Modern flagship models have context windows measured in hundreds of thousands of tokens. This might suggest that context management is a solved problem. It is not, for two reasons.

First, filling a large context window is expensive. Every token processed increases latency and API cost. A context window stuffed with irrelevant information is slower and more costly than one containing only what the model actually needs for the current step.

Second, and more significantly, larger context does not mean better performance. A 2025 research study tested 18 leading models across multiple providers and found that every single one performed worse as the amount of input grew. Some models dropped from 95% accuracy to 60% once the input crossed a certain length. More context is not always better context.

The Lost in the Middle Problem#

The attention mechanism in transformer models is not evenly distributed across the context window. Research consistently shows that LLMs pay the most attention to tokens at the beginning and end of the input, with a significant drop in attention for content placed in the middle. Accuracy can drop by over 30% when relevant information is placed in the middle of the input compared to the beginning or end.

This has a direct implication for how you structure context. High signal information belongs at the top or bottom of the context window. Critical instructions, the most relevant retrieved documents, and key user data should never be buried in the middle of a long context block.

Context Rot and Why It Degrades Agent Performance#

In a simple chatbot with a short conversation, context management is trivial. Problems appear when conversations get long, when tool calls accumulate, or when an agent runs for many turns. This is the phenomenon Anthropic’s engineering team has called context rot: as the number of tokens in the context window increases, the model’s ability to accurately recall information from that context decreases. This characteristic emerges across all models, regardless of provider or architecture.

Context rot is the primary reason why agents that work flawlessly in a demo with five turns start producing inconsistent results after twenty turns in production. The context has grown too noisy, too long, or too repetitive for the model to maintain accurate attention across all of it.

Frontend developers building multi turn AI features need to treat context as a finite resource with diminishing marginal returns, not as a buffer to fill as freely as possible.

The Five Components of a Context Window#

To manage context deliberately, it helps to think of it as five distinct layers, each with its own management strategy.

1. Instructions#

The system prompt and any injected instructions. This is where you define the model’s persona, behavior constraints, output format, and task framing. Instructions should be stable across turns and should not grow dynamically unless there is a specific reason.

2. External Knowledge#

Retrieved documents, data fetched from APIs, or content pulled from a vector database. This is where RAG retrieval results and tool call responses live. External knowledge is the most common source of context bloat, because teams tend to retrieve too broadly rather than precisely.

3. Working Memory#

Intermediate results and state that the agent accumulates during a task. In an agentic workflow, this might be partial results from earlier steps, a list of facts the agent has established, or a structured data object being built up incrementally.

4. Episodic Memory#

Relevant history from previous sessions or earlier parts of a long conversation. This is often represented as a compressed summary rather than raw transcript, since including full conversation history from earlier sessions quickly overwhelms the window.

5. The Current Turn#

The user’s current message, the most recent tool results, and the immediate context of what the model is being asked to do right now. This should always be at the top or bottom of the window to benefit from the attention concentration at those positions.

Practical Context Management Techniques#

Selective Retrieval Over Broad Retrieval#

The most common context management mistake is retrieving too much. A large customer support application might have access to the full user manual, every previous ticket, and the entire knowledge base. Injecting all of this into every request produces a bloated, expensive context that the model struggles to navigate.

The correct approach is targeted retrieval: extract key terms from the current query, search the knowledge base semantically, retrieve a small number of high relevance chunks, and rerank them before injecting only the top results. A context containing 6,000 tokens of highly relevant information consistently outperforms one containing 140,000 tokens of loosely related material.

async function buildRetrievedContext(query: string, knowledgeBase: VectorDB) {
  const embedding = await embedQuery(query);

  // Retrieve more candidates than you need
  const candidates = await knowledgeBase.search(embedding, { limit: 10 });

  // Rerank to keep only the highest-relevance chunks
  const reranked = await reranker.score(query, candidates);
  const topChunks = reranked.filter((c) => c.score > 0.7).slice(0, 3);

  return topChunks.map((c) => c.content).join("\n\n");
}

Conversation Summarization#

For multi turn interfaces, the raw conversation history grows with every exchange. A chat that has been going for thirty messages contains a lot of redundant content, repeated acknowledgments, and context that is no longer relevant to the current question.

A practical approach is progressive summarization: when the conversation exceeds a token threshold, use the model to summarize the oldest portion of the history into a compact summary, then replace those raw messages with the summary in future context builds.

async function buildConversationContext(
  messages: Message[],
  tokenLimit = 4000
): Promise<string> {
  const fullHistory = formatMessages(messages);

  if (countTokens(fullHistory) <= tokenLimit) {
    return fullHistory;
  }

  // Summarize the older portion of the conversation
  const oldMessages = messages.slice(0, -10);
  const recentMessages = messages.slice(-10);

  const summary = await model.complete({
    system: "Summarize the key points and decisions from this conversation segment concisely.",
    messages: formatMessages(oldMessages),
  });

  return `[Conversation summary: ${summary}]\n\n${formatMessages(recentMessages)}`;
}

Working Memory as Structured State#

Instead of accumulating raw tool call results in the message history, extract the important outputs into a structured state object and include that compact representation in the context rather than the full verbose output.

interface AgentWorkingMemory {
  taskGoal: string;
  completedSteps: string[];
  currentFindings: Record<string, unknown>;
  pendingActions: string[];
}

function buildWorkingMemoryContext(memory: AgentWorkingMemory): string {
  return `
Current task: ${memory.taskGoal}
Completed steps: ${memory.completedSteps.join(", ")}
Key findings: ${JSON.stringify(memory.currentFindings, null, 2)}
Next actions: ${memory.pendingActions.join(", ")}
  `.trim();
}

Representing working memory as structured state rather than raw message history keeps the token count predictable and ensures the model always sees a clean, organized view of what has been accomplished.

Context Isolation with Sub agents#

For complex multi step tasks, splitting the work across multiple agents with isolated context windows is often more effective than running a single agent with an ever growing context. Each sub agent receives only the context relevant to its specific sub task, works within a focused window, and returns its result to an orchestrator.

Anthropic’s own multi agent research found that architectures with multiple agents in isolated context windows outperformed single agent approaches with large shared contexts, precisely because each sub agent’s attention budget was focused on a narrower problem.

Context Poisoning and How to Prevent It#

Context poisoning occurs when incorrect or outdated information enters the context and influences subsequent model outputs. In a long running agent, this can compound: the model hallucinates a fact, that fact gets included in the accumulated working memory, and the model treats it as established truth in subsequent reasoning steps.

Prevention strategies include validating tool call results against expected schemas before adding them to context, using structured working memory objects rather than free form text accumulation, and periodically resetting the context for long running tasks by synthesizing a clean summary rather than carrying forward every previous output.

Structuring System Prompts for Maximum Signal#

The system prompt is the most stable and controlled piece of context you have. A well structured system prompt significantly reduces the amount of in conversation context the model needs to perform correctly.

Effective system prompt structure for production AI features:

Place the core task definition and most critical constraints at the very top, where attention is highest

Separate behavioral rules, output format instructions, and domain knowledge into clearly labeled sections

Keep the system prompt focused; avoid injecting large amounts of domain knowledge into the system prompt when that knowledge should come from retrieval

Use explicit formatting instructions rather than relying on the model to infer the desired output structure

Place any critical few shot examples near the end of the system prompt or at the beginning of the human turn, not in the middle

From Prompt Engineering to Context Engineering#

The distinction Andrej Karpathy drew, comparing the LLM to a CPU and the context window to RAM, is useful here. Just as an operating system’s job is to curate what fits into RAM at any given moment, a production AI feature’s job is to curate what fits into the context window for each model call.

Prompt engineering asked: how should I phrase this instruction? Context engineering asks: what is the minimal, maximally relevant set of information this model needs to perform the next step accurately, and how should it be structured and positioned within the window?

Teams that make this shift stop debugging prompts and start designing information pipelines. The results are AI features that perform consistently at turn fifty the same way they performed at turn five.

References#

Anthropic Engineering: Effective Context Engineering for AI Agents

LangChain: Context Engineering for Agents

Weaviate: Context Engineering, LLM Memory and Retrieval

LogRocket: The LLM Context Problem, Strategies for Memory, Relevance, and Scale

ByteByteGo: A Guide to Context Engineering for LLMs

Prompt Engineering Guide: Context Engineering Guide

Mem0: Context Engineering AI, How to Build Smarter LLM Agents