Speaker A
Hello, community. Yes, today we talk about some really adaptive harness structure. And yeah, you see here in the middle we have our fixed LLM, the core of our agent, and this beautiful harness structure. And today we have a new one,
Explore OpenGo, an adaptive harness for LLMs enabling runtime adaptivity and structural composability in complex coding agents.
Key Takeaways
- Adaptive harnesses enhance LLM agents by integrating runtime feedback to improve decision-making during long tasks.
- Structural composability enables scalable and modular agent systems without redesigning execution logic.
- OpenGo introduces a novel harness design combining composability with runtime adaptivity for coding agents.
- Simplifying complex technical terminology helps better understand the core concepts of adaptive harnesses.
- The harness acts as a control layer that verifies and influences LLM actions, improving reliability in complex coding scenarios.
What the video covers
- Introduction to a new adaptive harness structure for LLM-based agents developed by UFI Technologies, focusing on coding agents.
- Explanation of runtime adaptivity, where complex coding tasks produce new evidence that dynamically influences subsequent LLM decisions.
- Discussion of structural composability, allowing scalable, modular agent system topologies from single agents to swarms without rebuilding orchestration logic.
- Presentation of OpenGo, an open-source harness designed for developer composability and adaptive task integration during inference or training.
- Comparison with existing harnesses like Pi and DeepSea, highlighting OpenGo’s focus on adaptability beyond plug-in composability.
- Critique of the paper’s use of new technical terms and a simplified interpretation emphasizing familiar concepts like context construction and semantic feedback.
- Core idea: the harness should control the LLM’s decision conditions and verify task completion with a deterministic control layer.
- Example scenario illustrating how an adaptive harness can diagnose and handle complex code changes beyond simple edits and tests.
- Discussion on managing large contexts, compressing dialogue history, and maintaining relevant context handles for long-running tasks.
- Final reflections on the paper’s contributions, limitations, and comparison with recent models and progress in AI.
Chapters
- 00:00Introduction to Adaptive Harness Structure
- 01:32Runtime Adaptivity in Complex Coding Tasks
- 03:15OpenGo: An Open-Source Adaptive Harness
- 04:58Understanding Structural Composability
- 06:51Simplifying Technical Terminology and Core Concepts
- 08:35Example Use Case: Adaptive Harness in Code Modification
- 10:24Context Management and Long-Running Task Handling
- 12:46Paper Evaluation and Comparison with Current Models
Full Transcript — Download SRT & Markdown
Speaker A
an open-source one. And you might say, which company did it? Well, it is UFI Technologies.
Speaker A
Now keep in mind this is not a research university. This is not here a team of university researchers or academia or whatever. This is a pure corporate, if you want, technology publication here by, so you will understand immediately why I
Speaker A
give you this introduction published August 28, 2026, beyond the static harness configuration here for our agentic systems, for particular the long horizon coding agents, and you understand why they go with coding. Coding is the easiest to verify. You can execute the
Speaker A
code and you see immediately if it's correct or not the result. So long horizon coding agents are the simplest agents. The worst agents would be here open-ended agents. But here coding agent, this should be here a walk in the
Speaker A
park. Now they give us here, if you want, here GitHub repo here for this warm intelligence here. Beautiful. Great. And they have now two new terms. So let's look at this. When you [snorts] start to read the paper, you discover here this
Speaker A
term runtime adaptivity. What does it mean? Complex coding task continuously produce here new evidence.
Speaker A
Especially if you have real complex long running task somewhere, task subtask 15, you get new results that you did not have when you started to plan here the execution trace. So this sub results that come back have maybe or they
Speaker A
indicate new directions, new possibilities, new methodologies to you, and now you have to be as adaptive as possible running now here the complex coding task that this sub results, the sub elements are now integrated here in with the reasoning trace for the next
Speaker A
step, for the next decision by the LLM, for the next action. So therefore this should now dynamically during the run influence subsequent runtime decisions.
Speaker A
And the second term that we encounter here that is really interesting here, structural composability.
Speaker A
And the idea is as developers you need to compose capabilities. Now or you have to reconfigure execution logic here or you have to scale increasingly complex agent system. Topology goes from a single agent to a multiple agent system
Speaker A
to a swarm intelligence and you got the idea and you do not want to continuously rebuild here the orchestration logic, the execution logic, the planning logic. You want to have kind of a coherent system from everything from a single
Speaker A
agent to a swarm logic. So I understand here for reducing here the cost and the complexity in the runtime, UI is going here for structural composability. What is this new system that they call OpenGo?
Speaker A
OpenGo. Sorry if I pronounce it in any way. An open-source harness. This is beautiful. Designed for both the developer composability and some adaptive task and subtask integration and execution where the new result during the inference run or during the
Speaker A
training is able to alter here the trace of the procedural logic. You can also say OpenToYourVan is a composable and adaptive new harness design.
Speaker A
Now you might say, but wait a minute, we have Pi no agent harnesses here, beautiful, but you remember Pi emphasizes here rather minimal provide a flexible coding runtime whose advanced behaviors are introduced via the extensions, and you say, but yeah, but what about the new
Speaker A
DeepSea corners there? Well, look at it, yeah, beautiful, and it has 200,000 stars already on GitHub, but it pushes now further toward here this plug-in level composability making here the core components fully replaceable. But what if we go another step if you will make it
Speaker A
really adaptable here, and this is here this new paper, real nice. Now if you look at it, they say, okay, this was your Pi, then we have DeepSea corners, then yeah, of course we have CodeX here, OpenAI Code, an entropic, you got it
Speaker A
here, and then they ask what is the next step after CodeX, what is the next step after OpenAI Code, and they say now, okay, maybe we have here this open with the structural composability and with the runtime adaptivity.
Speaker A
And I looked at this and if you start to read the paper, I was kind of overwhelmed by the new scientific terms, you know, I mean, it's great if you have new ideas, if you have new constructs that
Speaker A
you build, but just to invent new terms, new wordings for things that already exist can be a little bit confusing. So for the structural composability and this is here on your right-hand side here, the top corner, this open provides
Speaker A
you a shared inner loop and an outer loop execution substrate. And you are familiar with this. Now we know here multiloop or the interlooping structure together with the rail mechanism for attaching here capabilities through life hooks or the composition and visibility
Speaker A
gating. And if you start to read, I was not familiar. I know this rail here from I think a year ago and entropic had something for the safety and security, you know, this rail and was not
Speaker A
really sure what is this rail mechanism, but okay, but in general in my understanding to help you here start up with this paper, this design reduces here simply the need for topology specific execution engines. Right now the second part is here the
Speaker A
runtime adaptivity and here it changes here the framework controlled runtime state rather than the model parameters, and you see here in the four corners here context management, LSP, passive feedback, server protocol, self-reflection, goal mode, and it sounds
Speaker A
really technical and first time reading this I said I do not understand that word, what is this? So I was a little bit unhappy with this formulation. I think here they tried to invent some beautiful technical terms,
Speaker A
but there is another way to explain this, to explain it in a much simpler way, and I have to tell you I feel that your autos failed to provide here some simpler insight because what we're interested in is the main idea, not the
Speaker A
highly technical wording or that you come up with some new technical terms. So let me reformulate this in my words. Those four mechanisms you see here with the runtime adaptive, for example, is simple. They allow evidence emerging during the complex coding trajectories
Speaker A
to influence now the subsequent execution. True. And now hold on to your socks because you have never heard this before. The context construction, the semantic feedback, continuation or stopping criteria, and cross-task reuse.
Speaker A
And you might say, "Hey, we know this. This is just what we are doing right now."
Speaker A
You see this what I mean? This is just some familiar terms. You don't have to invent technical terms with stuff that we already know. So therefore, allow me to present you this paper not in its original form but in my
Speaker A
transcribed version where I say keep it simple because I want to understand the core and not learn new terminology that is redundant. So the core idea of this paper now in my interpretation is yeah that the coding agent harness should not
Speaker A
merely execute here only the LLM decision and the action that the LLM decided on, but we want to have here this deterministic control layer. This harness structure should really continuously control the condition under which here the LLM decides and decide
Speaker A
then whether the resulting work here is actually finished. So we have now in the control cycle of the harness itself we give it more functionality, more power.
Speaker A
So and I come up now with a simple example because I'm not that intelligent and I need simple examples. So imagine we have the task where we ask an agent here, "Hey, change the repo payment API so that the refund returns now a structured
Speaker A
result instead of a boolean without breaking here the existing callers." Now normally the mall finds here a refund function here in the code, edits its return value, and runs a local test, and the test passes and everything is great,
Speaker A
and you think this is done now, but in this repo somewh-
Speaker A
invalidated here an assumption somewhere else somewhere deeper in the codebase. So the edit succeeded as a file operation but the task is not correctly completed because yeah you see this case here was not corrected by the file edit.
Speaker A
And now if you want in a very simple idea those are if you want three separate question that are now answered that first is the action execution did the file editing tool work and this is what we know this is here simply the
Speaker A
wrapper that we have and the answer might be yes but we miss out on the next two and then is the code consistency did this particular file change this edit introduce errors somewhere else as I just showed you in the example and the
Speaker A
third is a task completion. Does the entire repo now satisfy the request and now simple this open go tries to make all three parts all three question that you see here part of its runtime control part of the harness. Therefore we do
Speaker A
have an adaptive harness structure. Now in this example after the edit no the idea is now that the language survey analysis produces now a diagnostic on its own about some other maybe theoretical incompatible caller somewhere down the code. So the agent
Speaker A
does not have now explicitly to remember to ask hey did my change here in line 37 introduce a type or a symbol problem somewhere else? No, because now this particular harness configuration, this adaptive harness automatically guard is now relevant diagnostics filters
Speaker A
duplicates and injects here the important ones into the subsequent execution. So we have a little bit more of an intelligent harness structure if you want. Now this means that a successful file edit now produces two feedback signals. The first one is the
Speaker A
classical one. Yeah, the requested uh code modification was done was written. Beautiful. And now there's a second one.
Speaker A
This modification to this particular code uh in line 37 has exposed a new code inconsistency further down the road.
Speaker A
So this means the harness changes now what the model sees. But you say hey yeah we know this. I know this is exactly the point I want to make in this paper. This is not some new brand new
Speaker A
innovative structure. They refer to things that we already know. So the agent has already generated many search result logs and abandoned hypothesis instead of continue to accumulate them now unchanged. Now the context manager can do now the following. Either it
Speaker A
compresses here the older dialogue parts. It reduces the repetitive logs. It can collapse repeated unsuccessful attempts. move large artifacts out of the active prompt out of the context window or retain a handle so which original context can be recovered and
Speaker A
this will prove to be really helpful later on but you know this now because in my last two videos with Ricky skill here and with skill state I showed you this is exactly what is happening in those two videos so the next model call
Speaker A
can therefore emphasize here the new diagnostic and relevant caller while reducing here all the obsolete data stream here in the context window that is fed by the harness into the llm context window. So this is not something new but now let's be honest thinking
Speaker A
about this when uh H or Y started this study maybe half a year ago they did not know about here this new development by Google like wiki skills or skill state so for them it was new when they started
Speaker A
this 6 months ago okay another problem is that the model answer does not automatically end the task and you might say yes we know this I know but for theers of this Okay, but this was something that they wanted to make
Speaker A
sure. So they say the LM may finish one sequence of reasoning and finish one sequence of tool calls. But we have now to establish in the if you want adaptive harness structure an outer control loop.
Speaker A
And the job of this outer control loop is to evaluate whether the task should be continued, completed or simply stopped because it is blocked and waiting for something.
Speaker A
And you might say, hey, we are familiar with this. Yeah, I know because you a subscriber of my beautiful channel and I just showed you this in the video here in my last two videos. But yeah, it is
Speaker A
part of this new publication. And then there's this new feature quotation mark here by URI that useful experience can survive the particular task and you might say yes of course and I know this too but okay and they say
Speaker A
now the authors in industry and you see industry is a bit a little bit lagging behind no academia research but okay they say after the execution finishes here of a particular job a reflection mechanism can now take place to extract
Speaker A
here a reusable lesson learned for example in our case it would be when changing a public return type inspect downstream callers and update assumption about the old interface and this experience can be retrieved during a later task and you might say yeah of
Speaker A
course we put this in a memory and yes of course wiki skills of course but half a year ago when UI started this project this was unknown so they have now you buy an open harness that operates on two different time
Speaker A
scales no within a particular job within a particular task. Adapt execution to the newly available evidence. If you have long running sequences here, maybe at step 17, you have a complete new result and you have a loop across the
Speaker A
task retain selected lessons from completed trajectories. So this is what UI in their terminology call now beyond static harness.
Speaker A
And now let's come to the unique part of the preprint. And having read the technical abstract here on the first page, I was really excited by this paper because I said, "Hey, this looks interesting." But it turns out that sometimes the
Speaker A
highly technical wording that you have in in the introduction is not really mapped to the mathematical and technical context. in the next specific chapters.
Speaker A
Now I would say this preprint is primarily a system architecture contribution but not at all a new neural architecture at all or a learning algorithm in any case. Now its novelty claim lies here in connecting more as two properties that we have often
Speaker A
implemented already maybe separately maybe already a little bit connected by okay UI has done this publication it shows us yes we connected now the first is a we use the same execution machinery and they say okay suppose a developer starts with one
Speaker A
coding agent and then we add here planning component a semantic tracker memory component debugging agent reviewer structure and you know this. So then they say open proposes now one shared execution core that can be reused for multiple topologies a single agent
Speaker A
sub agent delegated sub agent or agent inside a multi- aent flow structure. Yes we know this but okay this is yet a version by UI. So the topology might change but the underlying agent execution semantics remain the same remain identical
Speaker A
and B the second great integration is this system might vary here the behavior of the AI system without learning any new weights this mean the LLM at the core of the agent is frozen we do not have supervised finetuning or any
Speaker A
reinforcement learning nothing we are really talking about a pure harness structure and evolution of a harness that is now with an adaptive component that is I would argue now having the current literature that is state-of-the-art but okay at the
Speaker A
beginning 6 months ago when you started is yeah you know it so during the execution the harness now changes but what exactly it simply changes what information can reach now the LLM which code diagnostic are now automated here
Speaker A
as a default option and become visible all the time and that the execution should continue should stop and these are topics that we had in EI academia 6 8 months ago but now we have it in industry in URI now implemented into
Speaker A
this prototype. Okay. So again the LM model remains fixed but the effective agent behavior changes because of the surrounding control process equal harness changes again to the loop structure. The inner loop is simple. The inner loops just ask
Speaker A
hey what should I do next? This is a bounded react style from the stone age of EI. It is purely harness based construct here the context for the LLM.
Speaker A
invokes the LLM, executes then the requested tools by the LLM, records the observation, has a file system, has a memory structure and continue until an answer or interruption or step limit says stop it. And all the time while we
Speaker A
have this beautiful harness-based operation, remember the LLM itself performs the local reasoning and the active tool selection. So the primary intelligence is with the LLM. The harness is just here if you want a deterministic security level, the control level and an outloop. And the
Speaker A
auto loop is simple. Should this task continue? So you see it's it's simple. Don't let you fool here by some highly specific technical terms because the core of this paper is trivial. So one complete inner loop invocation constitutes here one outloop round and
Speaker A
after that round the harness evaluates either continuation or completion or blockage and you got the idea yeah rail. Yeah, I mean just as an example, this is here a screenshot here chapter 312 rail ordered capability composition and they define a rail abstract as a
Speaker A
tuple from three element here and what they do and what space rail lives and attached to the hooks and everything.
Speaker A
But you know it in my interpretation it can be so much simpler. A rail is simply how capabilities attach to execution. A rail is a capability attached to a specific life cycle event. Think of it as a module that now runs at defined
Speaker A
boundary conditions such before a model call or after a tool call. My goodness, we don't have to invent technical terms for this. Huh? Or a tech diagnostic capability can react after a coding editing operation or a context capability as they call it can intervene
Speaker A
before the model invocation. So let's come now to the question. What actually adapts here in this adaptive harness? Wow, what a surprise. Now what is here? Everything available to the framework. You have the interaction history. You have the repo diagnosis,
Speaker A
the test outcomes, the goal state, the context pressure, the remaining resources and everything. But you know from my last two videos, the LLM does not necessarily see all of it in its limited context window. Yes, I know you
Speaker A
have maybe 2 million token, but this is not enough for a long complex task. So they have now a context constructor and this produces now a reduced or a compactifified context giving here that the harness influences here the action
Speaker A
selection by the LLM. How does it do with this now? By controlling which evidence will condition now the LLM. which evidence is now made available in the context window of the LLM by the harness itself. So context constructor you know this no
Speaker A
this is a familiar term and then yeah and then I think I have to show you I have to show you equation seven the conceptual value of an execution trajectory here in regard to a conceptual task utility and the runtime
Speaker A
cost contributions at a particular epoch t now please have a look at this and tell me do you immediately understand what we are talking about Yeah, great.
Speaker A
Because this is not it. Because this is just a conceptual lens. And listen, in theoretical physics, we have a lot of conceptual lenses and ideas and whatsoever. When we do some brainstorming, great. I love conceptual ideas, but don't give it to me here as kind of
Speaker A
a validation structure. And if you talk about runtime cost distribution that are not proven as a numerical model.
Speaker A
Okay. Yeah. Now what I really liked here at scanning here this particular paper and deciding that I want to show you this paper at the very beginning in my decision process. This was here one of the thing where I said hey this looks
Speaker A
cool. This is here the constraint single task runtime adaptation. And here again you have here on top you see equation seven. And I thought yeah this this might be something where I can explain here the complexity but it turns out the
Speaker A
complexity is known this is nothing new and I mean just look here at the axis look at the dimension of this particular space.
Speaker A
You have here let's say this is the x-axis here the context construct and on the yaxis you have here two subaxis in your coordinate system from a diagnostic feedback injection and some acceptance or stopping conditions here and they
Speaker A
built here a one-dimensional extended axis in a coordinate system and the closer you look and you understand this is just a conceptual lens and this This is not a numerical calculation. This is just Yeah. Okay.
Speaker A
So, just to make it clear, equation seven expresses here quite interesting design objective. Now, achieve useful task progress here in long context writing terms while controlling the execution cost. I think the idea is beautiful and it's great that they have
Speaker A
some concepts about it. But the authors explicitly also state that this is a conceptual lens, not a numerical optimization algorithm. And it is not demonstrated that there's any gradient update of it or not shown that there's an reinforcement learning training
Speaker A
procedure. It is not that there's any generic search involving or solving equation 7. And it is not an automatic redesign of the hornest shown. So to have some concepts is great. But if you have a scientific paper that you present
Speaker A
here to the world. Okay, let's have a look at the results. And maybe you understand why I'm a little bit disappointed by this paper.
Speaker A
We have two main benchmark classes. And the first one is benchmark terminal benchmark 2.1. And I love that they do not go with 2.0 by 2.1. Great check mark go. But look at the accuracy. So we have here cloud code, you know, entropic with
Speaker A
fable 5 on x high and beautiful. And for this particular terminal bench 2.1, we have here an accuracy of 83.8 plus - 1.2. And for the new one here, we have 84.04 plus - 1.12.
Speaker A
Now I don't have to tell you that. Let's look at fable 5. Let's really go with here the latest one here. This new adaptive harness structure open has here an accuracy of 84.04, but claude code has 83.8. eight.
Speaker A
So let's look at this and say okay so the margin over claw code is only 0.24 percentage point and this is even smaller than the reported approximately one point uncertainty value. So to claim now that this outperforms everything
Speaker A
else is interesting. And the second benchmark that they the authors refer to is SWUB bench verified.
Speaker A
Beautiful. So there you see here and we have here a lot of agent harness combination. But just look at the date we have here. The comparison is only to malls that were created or conceptualized in 2025. Some of them in July 2025.
Speaker A
And now we compare this to a model here from end of August 2026. And honestly if you are familiar with the progress in the n in the last 9 month in EI if you see that a 2025 combination still managed 79.2%
Speaker A
accuracy or resolveness and this new methodology only has from 79 to 82. I'm not really sure how massive the jump is because yeah, just look at the data and you will understand what I'm talking about.
Speaker A
In addition, I have to continue my critique on the paper. There are no detailed component ablation that are absolutely standard in a scientific publication. The authors do not do this.
Speaker A
The artists do not provide the data establishing how much improvement comes from what component because we do have the goal mode. We have the context management. We have the passive diagnostic modules. We have the reflection modules. We have the tool
Speaker A
availability and all the complexity that goes with a multi- aent coordination. We have no idea about the contribution ablation effect of each of those elements and I have to say the orers are absolutely transparent professional and say we acknowledge these limitations.
Speaker A
So what are the insights now I take from this paper? And I have to tell you at the beginning at reading here the technical abstract in the first page I was really excited and really thought we have a new adaptive harness
Speaker A
configuration here by a technology company UI and I was I was excited but absolutely there is now for whatever reason and maybe this is just a strategic publication by a Chinese company for strategic reasons but from a scientific point of view I'm
Speaker A
disappointed. Now what is positive about it that yeah again they show us that long horizon coding agent here they do not fail only because for the wrong action the decision by the LLM is incorrect but they can also fail because the decisive
Speaker A
evidence was never shown here in the context window to the LLM. So the control group by the harness failed.
Speaker A
Some obsolete information dominate here the context window of the LLM. The diagnostics of the harness arrive too late. A delegated agent receives inappropriate capabilities here. Let's think about skills, think about tools.
Speaker A
And a system mistakes an answer for completion command. Now I do understand that OpenG tries to move these issues into an explicit control layer and I think this is the way to go. No question about it. But this particular paper and open
Speaker A
demonstrate here that it does not established that its proposed runtime adaptivity causes here the performance improvement. nor does it establish that its composibility architecture reduces the engineering complexity.
Speaker A
So therefore we do not know what element contributed to the quotation mark success because the evolation does not directly isolate either of these two factors. No, but don't get me wrong this is does not mean that the architecture
Speaker A
in general could be ineffective. No, I'm just referencing here what the authors present to us. this particular preprint and it just means that the explanation for the effectiveness for the superiority of this new adaptive horn structure by UI and remember we are
Speaker A
talking about 0.24 two four percentage points compared to cloth code. This remains let's call it undetermined because we are a polite audience.
Speaker A
I hope you enjoyed it. I hope I provided you some data. Sometimes I also want to show you that it is not the technical terms that is not the shown complexity at the first pages of a paper that you
Speaker A
say wow there's something amazing new. You have to read the paper. You have to go through the terminology. You have to go to the core the core understanding of this new method. And then you should judge the particular advances or
Speaker A
innovation or new capabilities of an adaptive horn structure. I hope you enjoyed it. Some new information would be great to see you in my next video.
Topics:adaptive harnessLLM agentsruntime adaptivitystructural composabilityOpenGocoding agentsagent architectureAI system designlong horizon tasksopen source AI
