Speaker A
Hello community. So great that you are back. We have a brand new scientific paper about a new technology to finally implement self-evolving agents. So let's have a look now just as a grounding experiment. Imagine you have an
Explore next-gen self-evolving AI agents with a new agent trajectory data protocol enabling framework-agnostic, self-learning enterprise AI systems.
Key Takeaways
- Self-evolving agents leverage failures and successes as valuable training data for continuous improvement.
- The Agent Trajectory Data Protocol enables framework-agnostic, scalable self-learning AI systems.
- Mathematical optimization and reinforcement learning underpin the evolution of agent policies and memory.
- Collaboration between industry and academia is driving innovation in next-gen AI agent technologies.
- The approach supports enterprise deployment with flexibility across multiple AI frameworks and models.
Summary
- Introduction of a new scientific paper presenting technology for self-evolving AI agents.
- Focus on enterprise applications such as customer support, marketing, finance, HR, and production teams using diverse AI frameworks.
- Concept of recording all agent interactions, successes, and failures as training data for continuous self-improvement.
- Presentation of the Agent Trajectory Data Protocol (ATDP) as a framework-agnostic method to enable self-learning across different AI systems.
- Mathematical formulation of self-evolving agents involving policy updates, reinforcement learning, and prompt engineering.
- Discussion of system components including planning, tool execution, memory, sandbox environments, and control planes.
- Collaboration between Ant Group, Hong Kong University of Science and Technology, and Jingua University on this research.
- Use of reinforcement learning, distillation, and optimization techniques to evolve agent capabilities without manual intervention.
- Implementation details including use of Nvidia Nemo, Megatron FSDP, and various LLMs or vision-language models.
- Potential for transferring learned knowledge to local models behind firewalls for enterprise security.
Chapters
- 00:00Introduction to Self-Evolving Agents and Enterprise Use Cases
- 02:02Overview of Ant Group and AI Applications in Fintech
- 03:59Agent Trajectory Data Protocol and Control Plane Introduction
- 08:04Mathematical Formulation of Self-Evolving Agents
- 10:15Data Proxy and System Architecture Details
- 12:44Reward, Credit Assignment, and Causal Diagnosis in Agents
- 17:07Reinforcement Learning and Distillation Techniques
- 19:10Training Frameworks and Model Deployment Strategies
- 22:10System Evolution and Knowledge Transfer to Local Models
Full Transcript — Download SRT & Markdown
Speaker A
enterprise and you're really now here for production ready self-arning agents. Let's say you have a customer support team or a marketing team, a finance team, HR, production, whatever you have.
Speaker A
enterprise and you're really now here for production-ready self-learning agents. Let's say you have a customer support team or a marketing team, a finance team, HR, production, whatever you have.
Speaker A
signal what is working what is not working and why we just throw it away.
Speaker A
All of these different teams use different harnesses, use different frameworks, use different agent configurations, and the idea is now, hey, whatever they use, even if they fail, those are training data. So the idea was, hmm, we have tons and tons of learning
Speaker A
Tuesday, Wednesday, whatever you code, whatever you fail, record it. Let the system learn from maybe your mistake or the mistake of your agent or the mistake of your harness or the mistake of your LLM, whatever. Use it to understand how
Speaker A
signal—what is working, what is not working, and why—we just throw it away.
Speaker A
self-reinforcement learning about the agentic system now really being able to self evolve. And you might said okay July 2nd 2026 and now let's have just a moment here on the participants and all those beautiful orchers here who thought
Speaker A
If we have a clever new idea how to build those learning signals into training data, we can train our model on all these learning signals. But it's not just for enterprises. Also, if you are a single individual and you have Monday,
Speaker A
what is it and here this is an AI generated text you see AI overview it's a leading Chinese fintech and technology conglomerate here from Alipe digital payments, global solution, all the digital technology from software as a service to blockchain to whatever all
Speaker A
Tuesday, Wednesday, whatever you code, whatever you fail, record it. Let the system learn from maybe your mistake or the mistake of your agent or the mistake of your harness or the mistake of your LLM, whatever. Use it to understand how
Speaker A
Chinese main player and of course Jingua University. This is an image of the recent commencement ceremony for 2026.
Speaker A
to improve your machine. This is what we're going to talk about. New paper. This is here the next generation of agentic reinforcement learning systems enabling now self-evolving agents. So we are talking here about a complete AI system with a
Speaker A
available for you whatever you want beautifully. So what is the main idea? The main idea is listen all your departments or maybe on Monday you work with an Hermes agent or Tuesday you go to lang and maybe you have an old
Speaker A
self-reinforcement learning about the agentic system now really being able to self-evolve. And you might say, okay, July 2nd, 2026, and now let's have just a moment here on the participants and all those beautiful authors here who thought
Speaker A
your framework or your workflow works Because normally a genetic workflow we have a planning we have a tool execution maybe we have a memory for the execution we have a sandbox environment this is all standard things notice is more or
Speaker A
about this. Now in the middle, if you do not know, this stands for Hong Kong University of Science and Technology, famous. But the first one, Ant Group. I received a question last time when I had here the Ant Group here, and they said,
Speaker A
since this is from a university we have to define What is a self evolving agent?
Speaker A
what is it? And here, this is an AI-generated text. You see AI overview. It's a leading Chinese fintech and technology conglomerate here from Alipay digital payments, global solution, all the digital technology from software as a service to blockchain to whatever, all
Speaker A
say this is interesting. So we have here all the system parameters that we now interested in and that we have to collect and understand how they work together. But let's start at the beginning. We have our pi theta t. This
Speaker A
the financial solutions, AI and open source. This is just amazing. I just love here, um, these augmented reality glasses here. They have here a payment solution for smart glasses. This is so nice. So they are one year of the
Speaker A
But this is here the in context harness. M then stands here of course for memory.
Speaker A
Chinese main player and, of course, Jingua University. This is an image of the recent commencement ceremony for 2026.
Speaker A
instruction or some developer instruction or some prompt template. Maybe we have some new tool description or we have here different routers. Maybe we have an updated memory retrieval policy or we have why not add some memory policies brand new to this. Maybe
Speaker A
This is here the MIT of the East if you want. So absolutely interesting that those found together here to write this paper. So let's have a look. Yeah, of course, you have a GitHub repo. You have all your code, you have everything
Speaker A
Maybe we have new definition of sub agents. Maybe we have skill libraries we want to add or neglect or implement or augment whatever.
Speaker A
available for you, whatever you want, beautifully. So what is the main idea? The main idea is, listen, all your departments or maybe on Monday you work with a Hermes agent or Tuesday you go to Lang and maybe you have an old
Speaker A
this mathematics afterwards now there's now a control plane I will explain this in the next 10 minutes in detail what this is but just think about it we have now an agent trajectory data protocol so they implement a new protocol because
Speaker A
application in Crew AI, whatever, it doesn't matter. No, because the agent service, okay, this is a basic structure, but what we want, we want to add some framework-agnostic interception that makes it here a self-learning system. So this works for every idea how you think
Speaker A
So a new agent trajectory data protocol and we have here an evolution action. Now what is this particular action that now our mathematical optimization theorem should optimize. Now if we say hm the evolving action set could you could include now the update of the
Speaker A
your framework or your workflow works. Because normally, a generic workflow, we have a planning, we have a tool execution, maybe we have a memory for the execution, we have a sandbox environment. This is all standard things. Notice it is more or
Speaker A
files our skill patches here or maybe we just go with prompt edits. No third option is that we update here the memory itself. No, we remember our capital M the retrieval policy update here for the memory or another option would be to update the
Speaker A
less common here for all the different frameworks that exist. But what is new here on the right-hand side? What was the idea here? To have a really framework-agnostic new add-on. Now, of course, we can start here and
Speaker A
really a mathematical optimization theorem. Now we include a lot of new data. We don't say we just train the LLM. We don't say oh we just update here one harness element here. We don't say oh we just take care about the memory.
Speaker A
since this is from a university, we have to define what is a self-evolving agent?
Speaker A
Beautiful. So for the policy model weight update no the back end may involve on policy or neon policy reinforcement learning process reward learning this is important or you have seen this in my last video on polic distillation from
Speaker A
And please be precise. Now they have a prototype formulation here from Chinua and Ant, and they say we introduce a prototype formulation here for a self-evolving agent a at a time t_s, and now we have a mathematical tuple, and you
Speaker A
this may involve the simplest is a prompt optimization. You notice prompt engineering or context engineering here for a better context evolution itself or for code search or trajectory aware verbal editing everything that it goes with a skill MD
Speaker A
say this is interesting. So we have here all the system parameters that we are now interested in and that we have to collect and understand how they work together. But let's start at the beginning. We have our pi theta t. This
Speaker A
reasoning objects put it in your memory say hey this is something that worked and the control plane should organize all these different paradigms so you see this is not a simple optimization theory therefore I would like to start a simple
Speaker A
is here the policy LLM parameterized. Then we have h. H does not stand for the Hamiltonian like in theoretical physics.
Speaker A
chain and a car this is called crewi and a call this is called Hermes and a car this is called whatever you have here as a framework now when a standard taxi crashes now the mechanic checks here the log simply you
Speaker A
But this is here the in-context harness. M then stands here, of course, for memory.
Speaker A
should it op or she optimize Now the AI itself or was it not the LLM? Maybe it was a harness. Maybe it was a memory problem. Was the map wrong? Was the visual data wrong? Did the LLM hallucinate a green light or did just
Speaker A
T is here the complete tool repertoire and the tool schemas that you have. And guess what? Guardrails here. The guardrail and secure and safety governance configuration, whatever. Now the target objective of self-evolution could be we include now a system prompt
Speaker A
lot of open question and this is here where Aunt Cropia and Jinua University in Hong Kong University say let's solve this now at first there's this data proxy the data proxy intercepts now here the driving interface if you want and
Speaker A
instruction or some developer instruction or some prompt template. Maybe we have some new tool description or we have here different routers. Maybe we have an updated memory retrieval policy or we have, why not, add some memory policies brand new to this. Maybe
Speaker A
we have now dependencies. So what is it now? O you know O stands for the observation. So this is the tool output, the retrieval snippets, the user messages and the environmental state. H stands for the hidden internal status.
Speaker A
we have a new template here for the planning process here of single-agent and multi-agent configuration itself.
Speaker A
here for a particular action at a particular point t so this is the tools return user accept or user delete or some exit code r of course is the reward the reward signal the reward function either in the simplest case it's a
Speaker A
Maybe we have a new definition of sub-agents. Maybe we have skill libraries we want to add or neglect or implement or augment, whatever.
Speaker A
parameters why I'm going to tell you now this is the secret is this is about reproducibility so you as a coder you become now that I would with this system exactly see where you made a mistake where you here in
Speaker A
All of this here in total is yet the mathematical representation of a tuple. And this tuple, what they call a self-evolving agent a, and during this self-evolving, and of course everything is in mathematics because, hey, what else do we have to code
Speaker A
everything becomes standardized and it is like like a flight recorder that you have on an aeroplane and it is recording everything and I can absolutely replay every single system in the complete company. So M is for metadata. This is
Speaker A
this mathematics afterwards? Now there's a control plane. I will explain this in the next 10 minutes in detail what this is, but just think about it. We have now an agent trajectory data protocol, so they implement a new protocol because
Speaker A
want to be able to replicate this and learn from it. So the role if you're one of the data procs is not simply to export here the reasoning traces or the execution traces but to convert here some production work
Speaker A
they say, hey, listen, if you work with her or you work with OpenExplore or whatever framework you use, we need to standardize this.
Speaker A
And some persisting trajectories in a form suitable for replay and training because we will only put in our training data set those things that we know that we can replay it. We can modify a little bit. We can increase the temperature. We
Speaker A
So a new agent trajectory data protocol, and we have here an evolution action. Now, what is this particular action that now our mathematical optimization theorem should optimize? Now if we say, hmm, the evolving action set could include now the update of the
Speaker A
Let's go here. Any other skill set? Let's choose another template. We want to have a complete replay and if we have verified that the replay is working then we feed it forward into our training signal into our training data and then
Speaker A
policy of the LLM. So this is here really supervised fine-tuning or distillation or on-policy reinforcement learning, but also here under two you have the update of an in-context harness structure. Remember this is everything where we have our skills, our memory
Speaker A
They say listen we have to understand it is a system and all the components of this system are interwoven.
Speaker A
files, our skill patches here, or maybe we just go with prompt edits. No third option is that we update here the memory itself. No, we remember our capital M, the retrieval policy update here for the memory, or another option would be to update the
Speaker A
revelation. So you want to have a self- evvolving agent that needs access to enough structure to support now for your reward the credit assignment the causal diagnosis and the complete replay of your complete system.
Speaker A
repertoire in the tool schemas. No, the tool description added to tool schema change itself. Of course, in general, we can also have a rollback or no option under the safe government's control of our guardrails. So you see this is here
Speaker A
can debug in all the different places. Plus credit assignability. This new protocol must make the trajectory possible to answer the question of hey exactly which observation or which prompt fragment or which retrieval result from rag or which tool call
Speaker A
really a mathematical optimization theorem. Now we include a lot of new data. We don't say we just train the LLM. We don't say, oh, we just update here one harness element here. We don't say, oh, we just take care about the memory.
Speaker A
approach from our colleagues in China where aren't here and they do find they are the financial of the fintex system more more or less here one of the player of China with more than 1 billion customers so they say we have to make
Speaker A
We don't say, hey, let's just optimize here the tool schema. Everything is connected with everything else. And this is a mathematical optimization theorem that we have to solve in total.
Speaker A
returned, how was the interaction with the memory item and you see the complexity, the complexity manifold here in the numerical space is extreme dense with data. But we need all those data to really have a complete replay. And this
Speaker A
Beautiful. So for the policy model weight update, now the back end may involve on-policy or off-policy reinforcement learning process, reward learning. This is important, or you have seen this in my last video on policy distillation from
Speaker A
past with this AI system. Leadbound learning signal. Let's talk about this now. Many useful rewarding critiques arrive after the acting step.
Speaker A
different directive feedback. Distillation is really an important topic here, not just reinforcement learning, and I showed you in my last video we can use almost the same mathematical tricks for an RL optimization and for a distillation optimization. Then second, the harness space adaptation,
Speaker A
later. Anyway, this protocol should therefore allow an events reward field to be updated or augmented even after an initial logging while preserving immutability of the original causal record. So depending on your specific task, you will get here a reward signal
Speaker A
this may involve the simplest is a prompt optimization. You notice prompt engineering or context engineering here for a better context evolution itself or for code search or trajectory-aware verbal editing, everything that it goes with a skill MD
Speaker A
data set for the learning here, the reinforcement learning and the training here of our AI system.
Speaker A
file or a JSON.
Speaker A
participated exactly what to this decision policy LLM versions or the different checkpoint everything is there because if you would not have it, the agent experience becomes statistically useful but nonreproducible and therefore not suited for any learning data. It is
Speaker A
not enough to know anymore. Yes, it failed. Now we can have a deep dive and a complete system debugging and exactly know where it failed, what it failed.
Speaker A
Yeah. Government observability. Yeah. Including here the privacy security right from the beginning they take care about this. Great.
Speaker A
So let's go back to this example. Yeah. So when the car crashes now this let's call it here this is the flight recorded to make it really simple and visual for you. So what we have what we capture now
Speaker A
as from the completely system the complete observable state of the system. So this is the exact frame of the dashboard camera approaching here intersection with a faded stop sign for example. Yeah or a hidden internal state. What is it? That's the core
Speaker A
internal CPU reasoning scratch pad. And the AI is thinking here. Hm. The sign is red but an octagon shape and it is occluded. So I assume that it is not a traffic sign but maybe just an advertisement and you understand later. Uhhuh. This is
Speaker A
exactly where it where it happened because the next action by the eye system is here the physical actuation.
Speaker A
So the steering angle is now minus 50° and accelerator 20%. And the outcome of this particular action is now a collision. The collision was detected air deployed. And now we understand maybe 90 seconds later the reward. Now a
Speaker A
delayed signal comes in or 3 days later when the insurance companies officially declares that the tax is at fault. No.
Speaker A
So you have a late bound reward of minus 100 points for this particular action.
Speaker A
Everything that happens in this system is be has not a potential to become a learning training data set to improve the performance of the eye system. We don't throw away anything. No mistake, no error, no success, nothing.
Speaker A
Everything is analyzed till the very last layer and everything gets a reward and we learn from everything. And therefore we need all the metadata. No, the tire pressure, the time, the outside temperature, inside temperature, GPS, map version, software version,
Speaker A
everything that we have. And because this we have everything perfectly structured mathematically unified data protocols then in this new system here by our friends in China pipes it now into the reinforcement learning algorithm and let's go with our
Speaker A
good old friend PO from I don't know almost a century back now so the algorithm penalizes now the AI score core the LLM the mal weights here the tensor structure for misinterpreting here the occluded stop line.
Speaker A
So you see we have now if we have this data and we have the reward and we have here from the insurance then everything coming back. Okay, this is the mistake.
Speaker A
And if we do down here the debugging we see, okay, here the CPU the reasoning scratch. Maybe whatever it is says, okay, it was a misinterpretation of a sign that was partly occluded that it is just a an ads and not a traffic sign.
Speaker A
This is where it happened in this simple case. It's just for you that you understand what we are talking about and sometimes example help.
Speaker A
And if we then train here the reinfor the the LLM itself with this new reinforcement learning training data and we have to synthesize a little bit more for this. Then we have an evolved agent where a real world environment interface
Speaker A
not some synthetic training data but really we have been there we have crashed in this or whatever it was a simulation. We understand something went wrong and we could correct it. But we cannot just let this newly evolved agent
Speaker A
drive the next real taxi yet because aunt tells us yeah it's it's it's too expensive. No, it's too dangerous. The risks are too high. So what we do? Well, we sandbox it. And since we have recorded everything of this complete
Speaker A
system, guess what? We let it replay in a simulation. This is here where the control plane here. So the the main control element here of this new idea initiates a counterfactual replay. So we're not just tweaking here the car temperature dial
Speaker A
or the brake sensitivity in autonomous car driving or any hyperparameter here of the parameter. Oh, we asking, hey, if this new LLM, this new AI brain were in that exact car yesterday and now that we have corrected it a
Speaker A
little bit with a reinforcement learning algorithm, would it still crash or would it now understand that this is a stop sign that it has to stop and not enter here the traffic?
Speaker A
And only when we clear this particular replay problem and we are successful then this data are forwarded to the training data set. So you see this is an idea absolutely beautiful. Everything has to be registered, understood, recorded, software versions and we can
Speaker A
replay everything or let's think about this data proxy know it's a holo simulation because what we have we have this the saved empty this is the exact map this is exact tire pressure we have everything and our observation is the exact camera frame
Speaker A
from all the eight or 12 cameras that we have in the car so it creates a perfect simulated markup for this past intersection here on a holo deck if you have ever seen the enterprise. If not, forget it. So, [laughter] our updated
Speaker A
vision language model is now placed here as the central command element here in this holo deck. And the hologram is shown now this faded stop sign. Yeah.
Speaker A
And now with this new training, it says, oh, thinking look a red blob. It could be an ad, but given intersection context that I'm coming closer to this intersection, I must also assume that this is a stop sign that is a little bit
Speaker A
occluded. So therefore, if it's not really identifiable at this moment because it is blocked here, I have to take care that it could be a stop sign.
Speaker A
So we have now that the LLM decides here a new action has to take place here. And this action is activate the break with 100% not allowed here to enter here this traffic.
Speaker A
So you see the proxy verified that the new action prevents here the crash in this simulation in this holodex simulation whatever you like as an example.
Speaker A
So you see the agent has safely self evolved now without any manual human engineering and only because let's call it the enterprise proxy forced here a heterogeneous cartic locks into some strict replayable mathematical simulation.
Speaker A
I think this is now really a meter agent because the agent is not hardcoded.
Speaker A
Yeah, maybe at the very beginning. No, but then this agent is really learning from its environment and its interaction with the environment. It has to fail. It has to fail often before we really can allow DCI to be placed into an
Speaker A
autonomous driving taxi cab and that yeah it can enter here the real environment. So now that we understand exactly what is going on, let's come back here to the schema. So as I told you this here on the left side is here
Speaker A
for any custom code query lench chain heras agent whatever you have but here this interface it becomes now interesting for this framework agnostic interception what is happening we have here an llm API call and we come now to
Speaker A
the inference service we have three elements inference service the trajectory storage here and the training server so let's have a look so we enter here and we have here a router beautiful and then we have our data proxy. This
Speaker A
block of data proxy, this is where those cic string logs of the past and all of the failures are mapped now into our strict mathematical protocol, our ATDP tupil representation. So here everything is broken down into a standard protocol
Speaker A
and then the trajectory storage guess what is exactly where this clean replayable histories that are now standardized here with the mathematical tool that I just showed you before and 2 minutes ago are now saved and here we have hundreds and thousands and tens of
Speaker A
thousands of those tools for all the different situation. It is really learning by doing learning by interacting with the environment.
Speaker A
And then we have here the training. Now they went here with yeah you can go with whatever reinforcement learning you like here they go with the Nvidia Nemo framework container here they go here with the Megatron FSDP simple if you
Speaker A
want to code this here. This is the Megatron in Python it's just here you see one line and it can go but what is happening? What is the theory that you understand what's happening? So this training service this block here the
Speaker A
third element here pulls here the data from the trajectory storage processes it in its train workers this is your Nvidia structure one to end how many workers you can afford to pay Nvidia and then there's a dotted error labeled the
Speaker A
weight sync service back here to the inference worker saying okay now what is happening these training workers here 1 to n are are acting here as the reinforcement learning algorithms update and you go with an algorithm from Nvidia or from
Speaker A
hugging phase whatever you like. So we have now here in this training workers they're updating here the weights the tensor weights of the different layer of the transformer architecture of our LLM based here on this ATDP trajectory structure that is here in the trajectory
Speaker A
storage in a standardized protocol and once here this counterfactual replay validates that this new weights here from the train workers here won't crash in our next computer simulation Then we have here the weight sync service. Then this act this link become
Speaker A
active here and the weight synchronization represent here the deployment of the evolved agent back into the live environment.
Speaker A
So a very simple idea might say yeah but a very beautiful idea because they really understand we have to take care about the system complexity and about the system dynamics and it is not enough just to optimize the tool execution and
Speaker A
memory the planning or whatever wait so if we have all the departments and all the different departments have a different harness and a different agent and a different financial agent and a HR agent and they have but maybe and
Speaker A
hopefully a central LLM or a vision language mall. Go with whatever you like. Fable 5 to I don't know some open- source mall whatever you have as an LLM or vision language mall. It would be great if you
Speaker A
have one central intelligence layer that is now learning here for all the different department here exactly what each department needs. But if you can combine here everything in that's going on in your enterprise in your conglomerate of whatever company
Speaker A
structure you have from customer support to marketing to finance to HR to product teams this would be really something positive because then you have here really a corporate brain a corporate central intelligence you have everything from pattern recognition root cause
Speaker A
analysis insight generation you have all your databases whatever you need here and you can say Yeah. Okay. The marketing team has a particular harness or the finance team has a particular framework that they like to work with.
Speaker A
No problem. So let's bring this together. And I think this is the power here by Chinua University that they approached it here from a real industrial from a commercial side. No, because guess what? Maybe they want to use it here for also their application
Speaker A
in their companies in their enterprises. No. Yeah. and then you can optimize the hell out of it. This is just a mathematical optimization theorem.
Speaker A
And they prove in this paper that a continuous online reinforcement learning can really operate on a live enterprise agent traffic. So whatever is happening in all of those departments and let's say 90% maybe is not immediately successful. This is great because we can
Speaker A
use this data to understand what went wrong and then make it a success continuous online RL. So in this corporation this if you want vision language model never sleeps is always improving whatever is coming in a complete new way of a dynamic learning.
Speaker A
Of course this is not limited to an enterprise. If you are a single person, a single developer, a single coder, whatever you do on Monday, Tuesday, Wednesday, you got the idea. You can accumulate here your professional experience, how you evolve, how your AI
Speaker A
system evolves, how your harness complexity evolves, how you go to a high and higher LLM and maybe then you have learned so much that you can transfer or distill it down here to a local LLM that sits behind your firewall.
Speaker A
What are the insights of this paper? Highly recommend you have a look at this paper. The authors tell us so in principle we just have a three design pillar. No, we have a standardized agent trajectory data protocol because it
Speaker A
should work with all departments with all frameworks with all agents. Great. Then we have an enterprisegraded agentic data proxy where I showed you here multiple example what the proxy is exactly doing. And then we have here if you want a unified agent evolution the
Speaker A
control plane that is now understanding hey is this a job from this complexity of the training of of what happened here of the recording of the observation and the action that have been taken and the reward that is coming back time delayed.
Speaker A
Is this now that I just can write a new skill memory file markdown file or should I go into a memory markdown file and add some memory or should I write here a complete new workflow definition for this particular case in finance or
Speaker A
should I go and say this is important because this is interlin to multiple of my departments. Therefore, I have to go to the LLM itself to the core of the agent and train the LLM and modify the tensor weight. So
Speaker A
really this becomes a inherent knowledge of the LLM and it is not a deterministic element in the whess of this agent.
Speaker A
These decisions are not simple. I understand that aren't here as a financial uh service provider here for global services and whatever that they are interested that this system the complex system the complete corporate structure should become self evolving
Speaker A
which is absolutely fascinating if you think about it. Okay, first pillar just if if if [laughter] it was a little bit too much, we have here the data protocol. No, ATTP the agent the agent trajectory data protocol. What does it
Speaker A
do? It reframes the agent experience as type step level reinforcement learning rate event data rather than some blah blah blah logs or some response pairs.
Speaker A
So such a protocol should preserve the decision context, all the actions, all the outcome, all the reward signal that come immediately or a little bit delayed, all the particular metadata of each single element of the system components for provenence, governance
Speaker A
status needed for the credit assignments and for the replay. Second pillar, the data proxy specifies how the heterogeneous production interactions across the models work. How the tools can be intercepted, redacted, how the retrieval system can be annotated, how the memory stores can be
Speaker A
manipulated, I mean uh augmented and how the human feedback channels can be interpreted redacted persisted annotated and replayed as a real learning ready trajectory.
Speaker A
And the fourth pillar is simply then the complete control plane. This determines when and how the agent should do what.
Speaker A
Let's say inserting here new memory elements or patching here the skillmd file or editing here the complexity of the harness itself or maybe add new harness elements or changing here the tool schema. We're updating here the policy LLM weights during the
Speaker A
reinforcement learning that we have here and you can go with the RPO. No, never mind. Or have a roll back or simply do nothing.
Speaker A
So this is a mathematical optimization process that is now currently implemented. I've given you the GitHub repo if you want to play this out yourself. But I think what an intelligent systemwide approach.
Speaker A
Everything comes back together. Everything that we know about LLMs, everything that we know about harnesses, everything that we know about mathematical optimization, now it's back into one system and they try here to solve. And hopefully it's not the box of the Pandora. I hope you
Speaker A
had a little bit fun. Would be great to see you in my next
Topics:self-evolving AI agentsagent trajectory data protocolreinforcement learningenterprise AIframework-agnostic AIAI optimizationAnt GroupHong Kong University of Science and TechnologyNvidia Nemolarge language models
![[CC]FULL《炽热的夏天》EP01：寻找最合拍搭子！包上恩周柯宇礼物同频默契拉满｜#包上恩 #周柯宇 #炽… — Transcript](https://i.ytimg.com/vi/oHa4dHs7-2A/maxresdefault.jpg)
