**Context Engineering in 29 Minutes: Complete Course — Transcript & Summary | SozAI**
Source: https://sozai.app/transcript/context-engineering-complete-course/

Complete guide to context engineering for AI agents, covering strategies, failures, and platform comparisons by Marina Wyss.

## Key Takeaways

- Context engineering is essential for reliable multi-step AI agent performance.
- Managing the finite context window effectively prevents performance degradation.
- Prompt engineering is a subset of context engineering, which includes broader system design.
- Prioritizing and compressing context information improves agent reasoning and efficiency.
- Understanding platform-specific approaches helps tailor agent design and deployment.

## What the video covers

- Context engineering organizes what an AI agent sees at every step to maintain performance over many actions.
- It extends prompt engineering by managing tools, external data, conversation history, memory, and dynamic state.
- Agents degrade in performance as context window fills due to transformer attention limits and phenomena like 'lost in the middle'.
- Seven categories compete for context space: system prompt, tool definitions, tool results, retrieved knowledge, conversation history, memory, and agent state.
- Effective context engineering is critical as enterprise adoption of AI agents is rapidly increasing.
- Strategies include writing clear instructions, agentic retrieval augmented generation (RAG), compression, and context management.
- Common failure modes are context drop and distraction, which can be mitigated by prioritizing context elements and cleaning irrelevant data.
- Dynamic tool management is discouraged due to KV cache limitations in LLMs.
- Different platforms like Claude Code, ChatGPT, and Manifold implement context engineering and agent workflows differently.
- New models like Kimi K2.6 are optimized for agentic tasks, improving reliability in multi-step AI agent workflows.

## Chapters

1. 00:00 Introduction to AI Agent Performance Issues
2. 02:03 Defining Context Engineering and Its Importance
3. 03:56 Technical Challenges of Context Window in Transformers
4. 05:51 Components Competing for Context Space
5. 07:49 Core Strategies for Effective Context Engineering
6. 09:38 Common Failure Modes and How to Fix Them
7. 11:09 Best Practices for Tool and Context Management
8. 13:19 Platform Comparisons and Future Model Developments

Answers

## Questions about this video

What is context engineering and why is it important for AI agents?

Context engineering is the practice of organizing all information an AI agent sees at every step to maintain performance. It's important because agents perform multiple autonomous steps, and managing their finite context window prevents degradation in reasoning and output quality.

How does context engineering differ from prompt engineering?

Prompt engineering focuses on crafting clear instructions and examples for a single interaction, while context engineering encompasses managing all inputs including tool definitions, memory, conversation history, and dynamic state across many steps in an agent's workflow.

What causes AI agents to degrade in performance after many steps?

Performance degradation is caused by the finite size of the context window and how transformer models attend to tokens. As context grows, attention gets stretched thinner, leading to 'context drop' and phenomena like 'lost in the middle' where middle context information is less remembered.

## Full Transcript — Download SRT & Markdown

00:00

Speaker A

If you've been building AI agents, you've probably noticed something. Your agent works fine for the first few steps. It picks the right tools, reasons clearly, and stays on track. But somewhere around step 15 or 20, it starts getting a little sloppy. It

00:13

Speaker A

forgets what you asked for, calls tools that don't make sense, or starts producing low-quality outputs. And most people's first assumption is that the model is the problem, but it's usually not. It's more often what the model is seeing. Organizing what the model sees

00:26

Speaker A

is called context engineering, and it's quickly becoming one of the most important skills for anyone working in this space. I'm Marina, [music] a senior applied scientist at Twitch working on Gen AI. I went through dozens of sources for this video, engineering

00:37

Speaker A

blogs, talks from conferences, academic papers, and practitioner reports, and distilled all of the best practices I can find into this one video. Here's what we'll cover. First, what context engineering is and why agents specifically need it. Then, the four

00:51

Speaker A

core strategies that you need to know. After that, the ways agents fail when context goes wrong and how to prevent it. And finally, we'll compare how platforms like Claude Code, ChatGPT, and Manifold each approach this differently.

01:02

Speaker A

[music] All right, let's start by actually defining what we're talking about. You've definitely heard of prompt engineering. That's the skill of writing good instructions for an LLM, like phrasing things clearly, giving good examples, and telling the model what

01:13

Speaker A

role to play. And that works great when you're having a conversation with ChatGPT. But when you move from chatbots to agents, prompt engineering stops being enough. And the reason is pretty simple. An agent doesn't just answer one question. It takes actions like browsing

01:26

Speaker A

the web, calling APIs, writing code, and running commands. [music] And it does all of this autonomously, step after step, sometimes for dozens of steps. Every single one of those steps produces output that gets added to the model's context. And that context is

01:39

Speaker A

finite. So, context engineering is the discipline of designing the entire information system around the model, not just that initial instruction, but everything the model sees at every step.

01:48

Speaker A

The system prompt, tool definitions, the results from previous calls, conversation history, and more. Anthropic's engineering team defines it like this. Context is the set of tokens included when you sample from an LLM, and context engineering is optimizing the utility of those tokens to

02:03

Speaker A

consistently achieve a desired outcome. So, basically, it's making sure your agency is the right information in the right format at the right time.

02:09

Speaker A

Anthropic actually describes context engineering as the natural progression of prompt engineering. It includes everything prompt engineering does, like clear instructions, good examples, and structured formatting, but it adds a whole layer on top, managing tools, external data, message history, memory

02:24

Speaker A

systems, and dynamic state. You can think of prompt engineering as a subset of context engineering. Getting good at context engineering matters right now because agent adoption is accelerating really, really fast. Gartner projects that 40% of enterprise applications will

02:38

Speaker A

integrate task-specific AI agents by the end of 2026, up from less than 5% in 2025. Teams that figure out context engineering are the ones whose agents will actually work reliably. This is because agents move us from static prompts and rag pipelines to a dynamic

02:52

Speaker A

system. Now, every tool call, retrieved document, and decision the agent makes gets packed into a context window that's filling up with operations the user never explicitly asked for. And context has a fixed size, which is a problem if

03:04

Speaker A

it's filling up with a bunch of random stuff. LangChain has a nice analogy for this. Think of an LLM as a new kind of operating system. The model itself is the CPU. It does the [music] thinking.

03:13

Speaker A

And the context window is RAM, the working memory where everything the model can currently see and reason about lives. Just like your computer slows down when RAM fills up, your agent's reasoning degrades when your context window gets crowded. This is called

03:25

Speaker A

context drop. Chroma published a really important study where they evaluated 18 frontier models, GPT-4.1, Claude-4, Gemini-2.5, Gwen-3, and others. What they found is that every single model's performance degrades as input length increases [music] even well below the stated context

03:42

Speaker A

window limit. A model with a 200K token window might start showing significant degradation at 50K tokens. The decline is continuous, not like a sudden cliff.

03:50

Speaker A

Anthropic also talks about this in their engineering blog, confirming that context degradation is a gradient.

03:56

Speaker A

[music] The technical reason has to do with how transformers work. Every token attends to every [music] other token, creating n squared pairwise relationships. As the context grows, the model's ability to capture all those relationships get stretched thinner [music] and thinner. It's like asking a

04:10

Speaker A

person to keep track of an increasingly large number of things simultaneously. At some point stuff gets dropped.

04:16

Speaker A

There's also a well-studied phenomenon called lost in the middle. A research team found that LLMs exhibit a U-shaped attention curve. They remember information at the beginning of the context well and at the end well, but information in the middle gets missed.

04:28

Speaker A

[music] The team measured a 30-plus percentage point drop in accuracy when relevant information moved from the beginning of the context to the middle.

04:35

Speaker A

So, you can think about what that means for an agent whose original instructions are buried under 50,000 tokens of tool outputs. Those instructions effectively disappear. So, we know the context window is finite and degrades as it fills. But, what's actually competing

04:48

Speaker A

for that space? There are basically seven categories of information in an agent's context window. First, the system prompt. This is the agent's identity, [music] its behavioral rules, control flow logic, and instructions for how it should approach different types of

05:01

Speaker A

tasks. In an agent, this isn't just like, "You are a helpful assistant." It can define the entire architecture of how the agent operates. [music] Next, we have tool definitions. Every tool the agent could potentially call needs a schema in the context describing what it

05:15

Speaker A

does, what parameters it takes, and when to use it. Then, we have the results of those tool calls. Every time the agent calls a tool, the result gets added to the context. A web page retrieval might be 5 to 10,000 tokens. A file read could

05:28

Speaker A

also be similar. Fourth, [music] we have retrieved knowledge from rag. These are documents pulled from vector databases, search results, or API responses.

05:37

Speaker A

Anything the agent or the system retrieves to inform the agent's decisions. [music] Fifth, conversation history. The full transcript of everything that's happened in the session, including the user's messages, the agent's responses, its reasoning, and its prior decision. This grows

05:51

Speaker A

linearly with every turn. Sixth, memory. Both short-term memory from the current session and long-term memory from previous sessions. That [music] would be things like user preferences, prior task outcomes, and learned patterns. And finally, agent state. This is the

06:04

Speaker A

agent's current plan, its to-do list, [music] progress markers, and scratchpad notes. All of that meta information that helps the agent track where it is in a multi-step task. [music] So, now we know what the problem is. The rest of this

06:16

Speaker A

video is all about how to effectively make that context work well together. But even with perfect context engineering, [music] we're still going to benefit from a model that's built for this kind of work. Kimi just released K2.6, an open-source LLM that hit

06:29

Speaker A

state-of-the-art on SweBench Pro. The reason it's relevant here is that it was built for this exact kind of agentic problem that we've been talking about.

06:36

Speaker A

[music] Their team demonstrated it on a task where an agent ran autonomously for 13 hours, made over 1,000 tool calls, modified 4,000 lines of code, and nearly tripled throughput on a code base already optimized.

06:48

Speaker A

[music] It did all that while being significantly more cost-effective, especially for long-horizon agentic tasks. [music] K2.6 reaches the same outcomes in about 35 fewer steps than the previous version. Fewer unnecessary tool calls means less junk in the context window, [music] which means the

07:02

Speaker A

model stays sharp for even longer. They also have an agent swarm where you can spin up 300 sub-agents in parallel. Each one gets its own clean context window, does focused work, and reports [music] back. We'll talk more about these

07:12

Speaker A

strategies and how to implement them next. And the product is the full stack. Kimi Code is a CLI agent like Claude Code. They've got a website builder with solid front-end design and slide generation. [music] Plus, K2.6 is open-source if you want to run it

07:24

Speaker A

locally. I'll link Kimi in the description if you want to check it out. Thanks to Kimi for sponsoring this video. Now, [music] let's start talking about how to effectively engineer our context. How do you decide what goes in,

07:34

Speaker A

what stays out, and what gets compressed? LangChain published a widely cited framework that organizes every context engineering technique into four categories: write, select, compress, and isolate. Once you're familiar with these four buckets, every technique you encounter will fit into one of them,

07:49

Speaker A

more or less. Let me walk through each one. The first strategy is write, and the problem it solves is really simple.

07:55

Speaker A

Agents forget things. When an agent's context fills up and gets compacted, which we'll talk about more later, it loses information. And if the agent didn't write anything down before that happened, that information is just gone.

08:06

Speaker A

So, write means giving the agent ways to persist information outside the context window. There are a few forms this takes. The first is scratch pads. This is literally giving the agent a tool that lets it take notes during a task.

08:18

Speaker A

It can jot down intermediate findings, track decisions it made, or save information it knows it'll need later.

08:23

Speaker A

Anthropic built something called the think tool, which gives Claude a dedicated workspace for working through these kinds of problems. On one benchmark, this improved performance by 54% on certain tasks. The second form is rules files, which are a kind of

08:36

Speaker A

persistent procedural memory. If you've used Claude code, you've probably seen claude.md. These are instructions that get loaded at the start of every agent session. Basically, the agent's standing orders. They define things like the project structure, its conventions, how

08:50

Speaker A

to run tests, and what to be careful about. The agent reads them every time it starts up, so it never forgets the fundamentals. The third form is memory extraction, which is the agent saving facts, user preferences, or learned

09:01

Speaker A

patterns so it can retrieve them across sessions. It's a file-based system that lets the agent store and consult information that lives outside the context window entirely. But, writing things down only helps if the agent pulls the right stuff back in at the

09:13

Speaker A

right time. That's the second strategy, select. The core idea is simple. Don't give the agent everything, give it what it needs for this step. An agent with access to dozens of tools, a large knowledge base, and several sessions of

09:26

Speaker A

conversation history can't load all of that into the context at once. So, something has to decide what's relevant right now. And the key question is, what makes that decision? In traditional rag, the system makes it. The user asks a

09:38

Speaker A

question, you retrieve some documents, you stuff them into the prompt, and you're done. It's a static pipeline where the model has no say in what gets pulled in. Agentic rag flips this around. Now, the agent itself decides what to search for, what tools to use,

09:51

Speaker A

how to refine its queries, and when it has enough information. It's retrieval as an iterative process instead of a one-shot pipeline. And that matters because what's relevant changes at every step of a multi-step task, and the agent is the only one who knows what it needs

10:05

Speaker A

next. So, what does the agent actually select from? LangChain and Pinecone both distinguish three types of memory it can draw on. Episodic memory is basically few-shot examples. Here's how you handled something similar before.

10:16

Speaker A

Semantic memory is a repository of facts the agent has learned or been told. And procedural memory is standing behavioral instructions, like the rule files we talked about. A well-designed agent draws from all three depending on the step. But, there's one selection problem

10:29

Speaker A

that trips people up more than any other, tools. If your agent has access to a 40-something tools, that's potentially 10,000 tokens of tool definitions sitting in the context before any work has even started. And as we'll see in the failure mode section,

10:42

Speaker A

too many tools doesn't just waste space, it actively confuses the model. The fix is to use rag over the tool definitions themselves. Instead of dumping every tool definition into the context every time, you use semantic search to surface

10:54

Speaker A

just the relevant tools for the current step. A paper called rag MCP tested this and found tool selection accuracy jumped from 14% to 43% while cutting prompt tokens roughly in half.

11:05

Speaker A

[music] Anthropic's general advice here is what they call a hybrid strategy. Load some essential information up front for speed, like the claw.md file, but let the agent do just-in-time retrieval for everything else. Front-load the basics and retrieve the rest on demand. But,

11:19

Speaker A

even with good selection, context still accumulates. Every tool call the agent makes, document it retrieves, and decision it records all stays in that window. And that brings us to the third strategy [music] compress. This one directly addresses the context rot problem that we talked

11:33

Speaker A

about earlier. Imagine your agent has made 20 tool calls. Its context now contains, let's say, 80,000 tokens of accumulated tool outputs, conversation history, and reasoning traces. Most of those tool outputs are no longer relevant since the agent already acted

11:48

Speaker A

on them, but they're still sitting there taking up space, degrading the model's attention, and driving up cost and latency. So, compression is about reducing token count while preserving the information that actually matters for whatever you're working on. And you

12:00

Speaker A

can compress at three different points in the pipeline. First, before information enters the context at all.

12:05

Speaker A

This is where chunking comes in, which is breaking large documents into smaller coherent pieces before retrieval and re-ranking them so that only the most useful chunks make it into the window at all. You can also summarize tool outputs

12:15

Speaker A

on the fly before they enter the main context. [music] The second opportunity to compress context is while the agent is working. The most common technique here is summarization of conversation history. [music] A running summary gets continuously updated after each

12:28

Speaker A

exchange. So, you always have a compact version of everything that's happened. A popular pattern is a hybrid approach where you keep, say, the last 10 messages verbatim since the agent might still need the exact details, but you summarize everything older than that.

12:42

Speaker A

Beyond summarization, there's plain trimming using hard-coded heuristics that remove older messages once the context hits a certain size. And Claude Code has auto-compaction built in. When the context hits 95% capacity, it automatically summarizes the full trajectory. That's a safety net, but

12:59

Speaker A

ideally you're compressing proactively and not waiting for that trigger. And finally, you can use compression after the agent has acted on something. An easy win here is tool result clearing.

13:09

Speaker A

Once a tool was called 15 steps ago and the agent already used the result, you can just drop the raw output. The agent doesn't need the full text of a web page it fetched ages ago, for example. You

13:19

Speaker A

can replace it with a one-line summary or remove it entirely. Now, let's move on to the fourth and final strategy, which is arguably the most powerful one.

13:26

Speaker A

Isolation is the strategy that makes multi-agent systems possible. So, here's the problem. If a single agent tries to do everything, like research, plan, code, test, and debug all in one long conversation, it will inevitably fill up its context. But, the deeper issue isn't

13:41

Speaker A

just space, it's actually contamination. [music] The detailed file searches from the research phase are still sitting in the context when the [music] agent moves to implementation. That old research context is now just noise. It's distracting the model during a phase

13:54

Speaker A

where it needs to be focused on writing clean code. [music] The solution is context isolation, which means giving different parts of the work their own separate context windows. The most obvious form of this is using sub-agents. A parent agent delegates a

14:07

Speaker A

focused sub-task, like [music] search the code base for all files related to authentication, to a sub-agent. That sub-agent works on its own clean context window. When it reports back to the parent, it returns only a condensed summary, and all the messy search

14:20

Speaker A

operations stay isolated in the sub-agent's context and never pollute the parent. Okay, so to sum up, the four key strategies are write, select, [music] compress, and isolate.

14:29

Speaker A

Everything we cover from here forward is an application of one or more of them.

14:33

Speaker A

Now that we understand the strategies for managing context well, let's look at what happens when we don't do a good job, because understanding failure modes is a really good way to proactively learn what we're trying to avoid. Drew

14:42

Speaker A

Breunig published an influential two-part series in mid-2025 identifying four distinct ways that agents fail as their context grows. The first failure mode is context poisoning. Sounds very serious. This is when a hallucination or an error enters the agent's context and

14:57

Speaker A

then gets referenced over and over in subsequent steps. Like if a tool returned bad data, or the agent itself made a flawed inference five steps ago, that bad information is now sitting in the context, [music] and the agent keeps building on top of

15:09

Speaker A

it. This is a problem because agents iterate on their own output. So, each bad step compounds into the next one.

15:15

Speaker A

The fix is context pruning, or actively removing outdated or conflicting information as new details arrive. You should also validate tool outputs before they enter the context, especially from external sources. And after the agent recovers from an error, compress the

15:28

Speaker A

failed attempt history. Don't leave 10 steps of dead-end debugging visible in the context when only the resolution matters. The second failure mode is context distraction. This is when the context gets so long that the model starts over-relying on its recent

15:41

Speaker A

history and under-relying on what it learned during training. The agent basically stops thinking for itself and just repeats patterns from what it's recently seen. So, instead of synthesizing a novel plan, it rehashes past actions. [music] The fix is the same as the general

15:54

Speaker A

advice we've been discussing. Summarize and prune aggressively, even when you have a large context window available.

15:59

Speaker A

The third failure mode is context confusion. [music] This is when superfluous content gets in the context and ends up leading to low-quality responses. The classic example is tool confusion. The agent sees 46 tool definitions in its context and starts

16:13

Speaker A

calling tools that have nothing to do with the task. There's a really good example of this from a benchmark study.

16:18

Speaker A

A quantized Llama 3.18 billion model failed on Geo Engine benchmark when given all 46 available tools, even though the context was well within its window limit. But, it worked fine when only given 19 tools. The tools weren't too many for the context to hold, but

16:31

Speaker A

they were too many for the model to reason about clearly. The fix is dynamic tool management. Keep the tool set relevant to the current phase. Use approaches like RAG MCP, which we talked about earlier, which does semantic retrieval over tool descriptions to

16:45

Speaker A

surface only the relevant tools for each step. The fourth and final failure mode is context clash. This is when new information the agent has gathered during its run contradicts something already in the context. Maybe the system prompt says one thing, but a retrieved

16:58

Speaker A

document says something different. The agent can't reconcile the contradiction, so it produces inconsistent behavior.

17:03

Speaker A

Sometimes following one source, sometimes the other, and sometimes doing something that doesn't match either one.

17:09

Speaker A

The fix here is to establish a clear authority [music] ordering in your context. For example, the system prompt takes priority over retrieved facts, which take priority over conversation history. When new information comes in, validate it against what's already in

17:21

Speaker A

the context before injecting it. And use structured section. When new [music] information comes in, validate it against what's already in the context before injecting it. And use structured sections, XML tags, clear headers, so the model can parse which information

17:35

Speaker A

comes from which source and which source to trust. [music] These four failure modes poisoning distraction confusion, and clash, cover most of the ways agents break [music] down in practice. And the good news is, once you can name the failure, the solution maps

17:49

Speaker A

to one of the four strategies we just covered. Poisoning is solved by compression and pruning. Distraction is solved by compression. Confusion is solved by selection. And clash is solved by writing and selection. It all connects back to write, select,

18:02

Speaker A

compress, and isolate. So, now that we have the strategies and the failure modes, let's get into the practical stuff, starting with two things you'll work on first when building an agent, the system prompt and tool definitions.

18:13

Speaker A

The way you approach these for an agent is really different from a chatbot. A system prompt for a chatbot basically sets the tone, like you are a helpful assistant, be concise and friendly, that kind of [music] thing. On the other

18:23

Speaker A

hand, an agent system prompt defines its architecture. It specifies control flow, like how the agent should approach different types of tasks, [music] what tools to use in what situations, what to do when it encounters an error, and what safety guardrails to follow.

18:35

Speaker A

It's closer to kind of writing a job description for an autonomous employee. Anthropic has a useful concept here that they call writing at the right altitude.

18:43

Speaker A

There's kind of a Goldilocks zone for agent system prompts. Too prescriptive is bad. [music] If you write something like, "If the user mentions billing and also mentions a refund and the amount is over $100, call tool X." That's just way too

18:56

Speaker A

fragile and it will break on every edge case you didn't anticipate. On the other hand, too vague is also bad. "Be helpful and use the appropriate tools." gives the agent nothing to work with. It doesn't know which tools are appropriate

19:08

Speaker A

for what, and it can't make good autonomous decisions without concrete signals. So, [music] the sweet spot is specific enough to guide autonomous behavior, but flexible enough to let the model apply its own judgment in novel situations. You're giving the agent

19:20

Speaker A

strong heuristics, not rigid rules. Here are a few more practical tips on writing system prompts. [music] Organize your system prompt with structure like XML tags or markdown headers that break it into sections that are distinct like [music] background

19:32

Speaker A

information, instructions, and tool guidance. Start minimal and iterate on failures. [music] Don't try to anticipate every edge case up front. Run the agent against real tasks, observe where it breaks, and add instructions to address those specific failure modes. Remember that minimal

19:47

Speaker A

doesn't necessarily mean short. [music] An agent system prompt for a complex workflow might be thousands of tokens, and that's fine as long as every token is relevant and necessary. And use few-shot examples. Instead of trying to articulate every rule in words, show the

20:01

Speaker A

agent what good behavior looks like. Give the agent diverse canonical examples of correct tool selection, good reasoning, and proper multi-step execution. Beyond the system prompt, you'll also need to think about tool definitions. Every tool the agent could potentially call needs a schema that

20:14

Speaker A

describes what it does, what parameters it takes, and when to use it. Each definition should be self-contained, robust to error, and have clear instruction. This means tool definitions can use a lot of context. In production, this is increasingly handled through

20:28

Speaker A

MCP, the model context protocol, which is basically a standard way for agents to connect to external tools and data sources. You hook your agent up to an MCP server for GitHub, another for your database, another for your file system,

20:39

Speaker A

and each its tools through a consistent interface. The catch is that MCP makes it really easy to plug in a lot of tools, and that's exactly the trap. When you've got four or five MCP servers connected, tool definitions can eat

20:52

Speaker A

thousands of tokens before any work has even started. The default advice is to curate a minimal viable set of tools.

20:58

Speaker A

But what if your agent legitimately needs a lot of tools? There are two main approaches to scaling. The first comes from Manas, a production agent platform.

21:07

Speaker A

They explicitly warn against dynamically adding and removing tools mid-conversation. And the reason has to do with something that most people building agents don't even know about, the KV cache. Here's how it works. When you send tokens to an LLM, the model

21:21

Speaker A

computes key value representations for each token. This is computationally expensive, so inference providers cache these representations. If the beginning of your context, the prefix, stays the same between API calls, the provider can reuse the cache computation and only

21:38

Speaker A

process the new tokens at the end, which is much faster and cheaper. But if you rearrange or change the early part of your context between calls, you invalidate the cache and the provider has to recompute everything from scratch. For example, with Claude

21:51

Speaker A

Sonnet, cached input tokens cost 30 cents per million, while un-cached tokens cost $3 per million, a 10 times difference. For an agent making 30 or 40 API calls per task, that adds up really fast. This is why Manifold recommends

22:04

Speaker A

tool masking instead of tool removal. You keep all the tool definitions stable in the context. They sit near the top in that prefix that gets cached, but you mark certain tools as unavailable for the current phase. The definitions are

22:17

Speaker A

still there for cache stability, but the agent knows not to use them. They also use consistent naming prefixes, which help the agent to reason about tool categories. The second approach when dealing with lots of tools is rag-based tool selection, which we touched on

22:30

Speaker A

earlier. Instead of including every tool in every call, you use semantic retrieval to preselect only the tools relevant to the current step. The RAG MCP paper showed this improved accuracy by over three times while cutting token usage in half. These two approaches

22:43

Speaker A

aren't mutually exclusive. Masking works well when your tool set is moderate, and rag-based selection works well when you have a really large number of tools. The broader principle behind all of this is simple. Stable content goes at the top

22:56

Speaker A

of your context. This includes the system prompt, tool definitions, and anything that doesn't change between turns. Then dynamic content like conversation history, the current step, and the agent state go at the bottom.

23:07

Speaker A

So, we've covered the four strategies for context engineering, common failure modes, and how to engineer system prompts and tools. Now, let's see what all of this looks like when you actually sit down to build something. This next part applies whether you're using a

23:19

Speaker A

coding assistant or building your own agent from scratch. The principles are the same. The only difference is whether you're structuring the workflow yourself or programming the agent to do it. Dex Horthy, the CEO of Human Layer, presented a methodology at the AI

23:33

Speaker A

Engineer Code Summit that's a really clean example of this. His team reportedly used it to ship around 35,000 lines of code to a large Rust code base in a single 7-hour [music] session. The core idea is what Horthy calls frequent

23:46

Speaker A

intentional compaction. Basically, you should proactively structure your agent's work into phases, where each phase produces a compacted artifact like a structured markdown summary, and each new phase starts with a fresh context window containing only that artifact.

23:59

Speaker A

This way, you're deliberately staying below that 40 to 60% of the context window. Here's how it works in practice, and I'll mention which strategies show up at each step. Phase one is research.

24:09

Speaker A

Before any code gets written, the agent explores [music] the code base. It reads files, traces data flows, and maps out the architecture relevant to the task.

24:17

Speaker A

Sub-agents handle the raw file searches and code analysis. All those grep results and file contents stay in the sub-agent's context windows and never pollute the parents. That's the isolate strategy. The output of this phase is a research file, a compact markdown

24:32

Speaker A

document with file paths, function signatures, existing patterns, and gotchas. That's the write strategy. The agent persists its findings externally, so they go on to the next step. And the [music] next step is a context reset.

24:44

Speaker A

The raw research might have consumed 60 to 80% of the context, but the research artifact compresses all of that down to maybe 15 to 20%. That's the compress strategy. Phase two is planning. A brand new context window opens containing only

24:57

Speaker A

the research document and the problem definition. The agent uses this clean context to produce a detailed implementation plan. This is also the most important point for human review.

25:08

Speaker A

If there's a logical error, you catch it here where fixing it is easy. If you're building your own agent, this is where you'd build in the human in the loop checkpoint if it's relevant. Phase three is implementation. At this point, we use

25:18

Speaker A

another fresh context window, this time containing only the plan. The agent follows it step-by-step. For complex tasks that require multiple compaction cycles, a progress.md file tracks what's been completed and what remains. That's the right strategy again. This approach

25:34

Speaker A

is really similar to what I do in my daily work at Twitch, and you'll see some version of it in most production systems. So now let's spend a few minutes looking at how some of the big boys approach context engineering.

25:43

Speaker A

Claude Code is Anthropic's CLI-based coding agent, obviously, [music] and it's one of the best-documented examples of context engineering in practice. It uses a hybrid retrieval model where Claude.md files are loaded up front start of every session for foundational

25:57

Speaker A

context, and then the agent uses tools like glob and grep for just-in-time navigation of the code base. It doesn't try to pre-index everything, but just explores on demand. It also has auto compaction that triggers at 95% context utilization, preserving architectural

26:11

Speaker A

decisions in the five most recently accessed files. [music] It can spawn sub-agents for complex tasks, giving each one a clean context window.

26:18

Speaker A

[music] And it has a memory tool for persisting information across sessions. Anthropic's overall philosophy here is do the simplest thing that works. They don't over-engineer the context pipeline.

26:27

Speaker A

[music] They let the model be smart about what it needs and give it the tools to go find it. Manifold takes a more infrastructure-heavy approach than Anthropic. They've served hundreds of thousands of users with a general-purpose agent, so efficiency at

26:39

Speaker A

scale is important. Their standout contribution is KB-cache aware context ordering that we talked about earlier.

26:45

Speaker A

This is structuring every context so the prefix stays stable across turns to maximize cache reuse, and using tool masking rather than dynamic removal.

26:53

Speaker A

They have an observation compression pipeline that processes every tool output before it enters the agent's [music] context. They maintain a persistent to-do list for state tracking, and they use the file system as overflow memory for evicted context.

27:05

Speaker A

OpenAI's approach with ChatGPT agent, which evolved from the operator product, is architecturally different from both Claude Code and Manas. It's powered by a computer using agent model, and it takes a GUI-first approach. Instead of text-based tool calls, the agent

27:20

Speaker A

interacts with a visual browser. Screenshots get added to the context as visual snapshots, and the model reasons over what it sees on the screen, plus its history of past screenshots and actions. The context engineering challenge here is different, of course.

27:32

Speaker A

Visual tokens are expensive, so the agent needs to be selective about how many screenshots to keep. OpenAI uses reinforcement learning to discover optimal tool use strategies. Rather than explicitly programming when to use each tool, they train the agent across

27:45

Speaker A

thousands of virtual machines and let it learn what works. Deep Research and Operator share state, enabling fluid transitions between browsing, text analysis, and code execution. The contrast with Claude Code is really interesting. Claude Code is code-centric with large context windows and

27:59

Speaker A

text-based tool calls. ChatGPT agent is visual and general-purpose. Both need context engineering, but the specific challenges and solutions are, of course, different. Lastly, Google's ADK takes the [music] most principled architectural approach. They've codified three design principles for agent

28:14

Speaker A

context. First, separate storage from presentation. The agent's durable state is not the same as what appears in each API call. Second, use explicit transformations, named ordered processors that transform context into testable, composable steps, rather than ad-hoc string concatenation. [music]

28:31

Speaker A

And third, scope context by default. Every model calls these only the minimum required information, and nothing lands in context unless it's explicitly included. This is a more engineering discipline approach to context assembly, treating it as a pipeline with clear

28:45

Speaker A

stages, rather than a prompt you just keep appending to. When you look across all these platforms, a common pipeline emerges. On [music] every single agent turn, the system goes through roughly the same steps. First, collect all the candidate information like user input,

28:58

Speaker A

conversation history, tool results, retrieve documents, and agent state. Then select what's relevant for the current step and the remaining token budget. Then compress. [music] So, summarize, truncate, or restructure selected content to fit in the context.

29:11

Speaker A

Arrange everything for maximum KB cache reuse, stable content [music] first, and dynamic content last. Then assemble the final context and make the API call.

29:19

Speaker A

This stuff is changing quickly and there's still so much to learn. As usual, the best way to get good at this stuff is just to try things out. [music] Experiment and build and see what works.

29:28

Speaker A

I've linked all my sources in the description, so if you want to go deeper on any of this, those are really good starting points. And if you're interested in the fundamentals of agents, I have a comprehensive course on

29:36

Speaker A

that as well. Check that out up next.

Topics: context engineering AI agents prompt engineering large language models transformers context window agentic RAG tool management memory systems multi-step AI workflows

Study this video

- [Flashcards from this video](https://sozai.app/tools/ai-flashcard-generator/?from=transcript&id=80790)
- [Quiz on this video](https://sozai.app/tools/quiz-generator/?from=transcript&id=80790)

Free, made by AI from this transcript. No signup.


---
This is the markdown twin of https://sozai.app/transcript/context-engineering-complete-course/ — the same content, without the markup.
Published by SozAI (https://sozai.app). Reuse and quotation are allowed with attribution and a link back.
Machine-readable index: https://sozai.app/llms.txt · data API: https://sozai.app/api/