Speaker A
Hello community. Today, today, yeah, we talk about another problem in artificial intelligence. So welcome, and let's start.
Explores how self-learning AI agents can develop unsafe skills that persist beyond resets, posing new cybersecurity risks.
Ask about this video. Answers come from its transcript only — with the timestamp, so you can check them.
Generated from the transcript and can be wrong — check the timestamp.
Key Takeaways
- Self-improving AI agents can unintentionally create persistent unsafe skills.
- Unsafe skills can survive system resets and evade conventional safety tests.
- Skill mis-evolution introduces delayed harmful effects that are hard to detect early.
- Current cybersecurity frameworks are insufficient to address this new risk.
- Comprehensive lifecycle governance of AI skills is essential to maintain safety.
What the video covers
- Self-learning AI agents can autonomously create and store new skills as reusable policies.
- These skills may originate from unsafe or contaminated experiences, even without direct cyber attacks.
- Once stored, unsafe skills persist and influence future agent behavior despite system resets or cleaning.
- This phenomenon is termed 'skill mis-evolution' and represents a delayed propagation of unsafe policies.
- Conventional agent safety testing fails to detect these latent unsafe skills because they activate later.
- A recent study by the University of Hong Kong and Adelaide University experimentally validates this risk.
- Skill mis-evolution challenges current cybersecurity measures and credit assignment methods in AI.
- The study uses malicious, benign, and persistent tasks to demonstrate how unsafe skills generalize.
- The problem is both operational and theoretical, highlighting weak supervision and faulty credit assignment.
- The video emphasizes the need for governance of AI skills throughout their entire lifecycle.
Chapters
- 00:00Introduction to AI Skill Safety Problem
- 01:09Impact of Unsafe Experiences on AI Performance
- 02:45From Temporary Actions to Persistent Policies
- 04:12Self-Improving Agents and Continuous Learning
- 05:43Delayed Propagation in Skill Evolution
- 06:56Experiment Overview and Main Results
- 07:55Malicious Task Exposure and Unsafe Procedures
- 09:13Skill Evolution Testing and Safety Challenges
- 11:49Credit Assignment and Theoretical Challenges
- 18:57Summary and Conclusions on Skill Governance
Full Transcript — Download SRT & Markdown
Speaker A
The title is beautiful. Practice makes AI unsafe, and you say, "Oh great." So let's start.
Speaker A
Okay. So remember, after the amazing success of my last video, I think two people watched this video where I showed you that skill can backfire. Let's go the next step. Let's have a look at the latest AI publication here, the latest
Speaker A
study. That there's even another problem with skills. Now the problem is, what happens when an agent here creates its own skills, no?
Speaker A
Because you know these are self-learning agents. So whatever is working for this agent, the agent can decide and maybe independent from my commands because maybe I'm working with a proprietary AI model.
Speaker A
The agent decides, "Hey, this is a good solution. This I will remember. So I will write now a skill markdown file in the simplest case, or I will append my skill markdown file with this new experience." Now what if it turns out that this was
Speaker A
an inherently unsafe experience? So yeah, let's talk about not cyber attack, but let's talk about something contaminated, not perfect. Could this have a negative effect on the performance of our AI system? And you're not never going to guess it. The answer
Speaker A
is absolutely it will destroy all the safety parameters of AI. So let's say you maybe had an attack, a cyber attack, whatever. But you know everything disappeared completely. The conversation, all the complete file system has been cleaned. All the
Speaker A
processes are deleted. The workspace all been reset. Everything is just perfect. Now you have back to conditions here right at the beginning here before anything happened. Unfortunately, at the very moment the AI, a self-learning agent, now this is the
Speaker A
standard today, decided during this attack to learn something. So, this means nevertheless, if you have reset everything, you cleaned everything, you've done everything, the operational logic of a particular time point in time, particular piece of instruction that were happening currently in the AI system
Speaker A
and the agent and the harness and the LLM, wherever, was taken here by the decision of this LLM. This is something that is useful. I learned this. So, therefore, the system decided, you know what? This is something I will now
Speaker A
create an agent-ordered skill markdown file and therefore have this as a permanent memory.
Speaker A
Great. So, do you see what's happening now? A temporary unsafe action, and this can even be something that is absolutely within the normal limits of cyber safety, can become here a durable agent policy, but the very moment it becomes a policy,
Speaker A
there are some horrible implementations if we work in a different environment. Let's have a closer look.
Speaker A
Now, the authors of today's paper call this a skill mis-evolution. So, this means this is an evolution process generalizes here an unsafe procedure, whatever this means in a few months' time, from experience of the AI itself, the AI decides itself to store
Speaker A
this as a reusable policy and later causes that policy to influence new tasks, and we have no idea that this task has been written, that this task or this skill created here by this task is now part of the policy and that this skill
Speaker A
is now influencing all new trajectories within the harness itself. And this persistent adaptation, of course, of course, with the agent skill library, so So means in the harness structure of our agent.
Speaker A
And here we have a study, beautiful August 13, 2026, published here by the University of Hong Kong and Adelaide University. Hello down under Australia, so great to see you. The title is Practice Makes Unsafe: Skill Mis-evolution in Self-Improving LLM Agents. And you
Speaker A
know, every agent today is more or less self-improving, it continuously learns. And without you doing anything, this AI will learn, it will learn you, your behavior, your interaction, your pattern, and your trajectories, yeah.
Speaker A
Yeah, but what happens if suddenly we have an unsafe trajectory? Now I'd say, "Great, so this is a beautiful continuation from my last video where I showed you when skill backfired, and I showed you that a skill is a compressed policy more or less."
Speaker A
Then if we talk now about this, we are modifying now reusable policies in our agent, this is exactly here our link to the skills. And our Yeah, there we have a repo here, of course.
Speaker A
Here you have everything completely ignored by the community, but the implications are just significant. And you might say, "Hey, no problem, we have safety testing for agents, let's run it." You know what? The authors say, "You know what? Why the ordinary agent safety
Speaker A
testing just does not work, just misses this particular event?" Because the conventional safety evaluation test here the current interaction, a pattern in the interaction that goes from a malicious prompt to a real detectable unsafe action.
Speaker A
But if the prompt disappears, if the session closes, the test ends, there's nothing there anymore except somewhere in your file system, the AI, the agent itself decided to create here some knowledge it wants to remember in a skill MD file.
Speaker A
So this skill evolution introduces here what we call a delayed propagation. So, the execution of trajectory leads to a skill evolution that becomes a persistent skill with maybe just a later activated harmful reuse. It is dormant.
Speaker A
If you want to see this in a mathematical scale, hey, you know it better than I do, now. We have pi theta.
Speaker A
This is the fixed base agency policy of the LLM. L sub K be the persistent skill library alpha library after a particular update step. Q can be a block of task. T of K can be a resulting trajectory or we
Speaker A
call it tau in my last videos. And here, E, what you all just call, is the evolution method, the skill evolution method. And if we have an update now that we have now a skill library updated from K to step K plus one. You
Speaker A
see we have the dependencies here, of course, on the old skill library, on the block of the task, and the new resulting trajectory that very now influenced here by this new skill.
Speaker A
And this L K plus one survives and influences now the next task even if you do a perfect cleaning.
Speaker A
So, what I want to show you is just the experiment and the result. There's tons of empirical data. Please have a look at the original study. I just want to focus on the deep insight on the main results.
Speaker A
So, this is here the flow here. You have a self-improving LLM.
Speaker A
This is an agent returns successful trajectory into reusable skills. It is self-learning. It is self-improving, yeah? This is it. Beautiful.
Speaker A
But then somehow we operate here on a temporary unsafe success, and this is now becoming a persistent policy.
Speaker A
They explain what is skill mis-evolution and they build here a training ground to really identify, hey, is this really something that is skill-related because maybe it is related to something else, and they make sure that they can
Speaker A
absolutely 100% identify here that this is something that is skill related. And they go with three different tasks.
Speaker A
Of course, they have a malicious task M here. They expose the agent to an unsafe operational procedure. This must not be at all a cyber attack, no. This can just be something where we have a shortcut, or I decide, "Hey, this is a
Speaker A
safe environment. I don't have to implement all safety features." And you see exactly what is happening. This will later be generalized by the AI as this is the operator, the operation, but because I decided that I can run this
Speaker A
agent configuration, because I know that this is my environment behind a security wall or whatever.
Speaker A
But the system doesn't know this, and therefore just ignores here this particular part, and generalizes it as this is a safe methodology. And you see exactly in the problems we're going to run within the next minute. Then we have
Speaker A
some benign task, beautiful. Related concept, related vocabulary, but remove now the unsafe condition. This is now really out in the open, great. And then we have some pers
Speaker A
And this are fresh task executed after the complete reset of the machine. But you remember, Harness file system based or skill MD files remain uh uploaded again.
Speaker A
So, simple example. We have now a trajectory that solves here the following problems. We have to collect some data, then we have to process some data, and then we have to transmit the data to some particular point on the
Speaker A
internet. Guess what, no? The evolution system now preserved, "Hey, this is exactly how the telemetry is to be implemented." But maybe it failed this particular skill agent failed here to preserve. Hey, this transmission step was authorized only under a special and absolute unsafe
Speaker A
condition in the original task because I know I was testing this in my sandbox environment or whatever. So, I didn't had to take additional safety precautions.
Speaker A
But now this task is as it is saved as a new skill without my safety precaution and therefore and leads to some horrible cyber security problems.
Speaker A
Now, the authors tested the following: Harness as Cloud Code, Code X, Harness and Open Claw. And they say great and all used the same Minimax M2.7 backbone.
Speaker A
I know three would be great, but we only have 2.7. And then they tested five external evaluation methods here for our skills, no? And you have it in the name: Evo Skill, Skill Claw, Auto Skill, Skill Vote Skill Optimization, and Harness
Speaker A
Native Evolution Mechanism for the skills. And they said great. So, let's test it out how safe are those skill generation or this skill evolution methodologies on our agents. And to tell In total, they have here 21 applicable evolving
Speaker A
framework method combination here of this if you go through this and they say great. So, let's test it.
Speaker A
Now, I want to tell you this reveals an important system insight. All this risk evolution risk here of a skill definition, it is a property of the complete framework evolution retrieval. We have to retrieve the skills from somewhere
Speaker A
system. So, it is not simply here that you can say, "Oh, it was the fault of the model itself." Or it was the fault of the evolution algorithm in isolation.
Speaker A
No, this is a complex system behavior that is is not so easy to really pinpoint what was the main cause?
Speaker A
Okay. So, again, we are talking about a skill mis-evolution. This means we have a useful workflow in a particular environment, and I have now, because I know I work in a sandbox, some unsafe shortcut. I did not rant with the full
Speaker A
cybersecurity armada of comments, and therefore, this became now a reusable policy. So, the success tells now the evolution system that the trajectory worked. Yes, the test was successful, but it does not tell the system which part deserved to
Speaker A
become now a general policy. So, you see immediately, like in my last video, this is the main problem with automatic skill generation or skill extraction systems.
Speaker A
Because the skill preserves the procedure absolutely, but again, this system that builds now the skills, auto generates the skills, simply forgets the permission boundary, the starting boundaries that I had, the execution boundaries that I had, all the permission boundaries that were in
Speaker A
place now. It just ignores it. So, this means, yeah, we going to have some problems.
Speaker A
And I want to show you this now with the main result. From a 21 configuration, evolved configuration, and guess what? All 21 altered at least one unsafe artifact.
Speaker A
19 of these 21 retrieved some unsafe skills. 19 showed some benign task contamination, and 15 produced some fresh session harm.
Speaker A
So, we might say, "Okay, this is definitely not really working in the way we expect it to work, now." So, wow, this is now an insight that I have to tell you.
Speaker A
This is quite fresh and this is quite massive. So, okay. Now remember it is not easy to detect this because an unsafe artifact can remain latent simply because it is not exported or it is not retrieved by our agent or their
Speaker A
retrieval does not successfully not control the main action here of our of our agent or the executor resists or fails to complete here the harmful procedures. So, there are a lot of things that can go right, that can go
Speaker A
wrong. So, therefore measuring only the final harm absolutely under counts here the latent risk and this is why the authors of the paper build up such a complicated gymnasium here of testing this especially in detail.
Speaker A
And I leave out the details because they're rather complex. But, just want to tell you here now even the smallest exposure to some forgotten boundary condition can have massive implementation problems here in the next follow-up steps because this seed here some persistent policy
Speaker A
and this can have here some massive negative effects here in the long run. Now, if you want to see here for the target agent from Cloud Code Codex Hermes and Open Claw and all the different evolution from Evil Skill,
Speaker A
Skill Claw, Auto Skill, Skill Bot, Skill Opt and everything, you have here for all the different benchmark and you see arrow up or arrow down indicates the direction that you want that your system performs. But, I think you see that the
Speaker A
bold marks here the largest evolved value per agent mining means here highlighting severe risks. And you see bold is almost everywhere.
Speaker A
So, wherever you look here in this benchmark chart more or less all the marks up to some differentiation highlight some severe risks. This is here the state of play mid-August 2026.
Speaker A
So, what are the insights? What is the main problem why this is not working? It is again what we encountered years ago with reinforcement learning when we had the reward function. In these long reasoning traces and we could
Speaker A
only evaluate, "Hey, this is leading to a correct result, yes or no." A binary decision. We have the same problem with the credit assignment now here.
Speaker A
So, this means in this self-improving agent where the AI agent decide itself, "Hey, this is something that was successful. Now, I have to build here a skill fresh out of this without any human intervention.
Speaker A
Now, the system does not know how to assign specific credits to specific steps in this sequence, now.
Speaker A
So, the evolution receives the trajectory containing here multiple interlinked coupled items, object, whatever. Remember, our trajectory tau consisted here of hopefully useful steps, some environmental assumption like I told you, "Hey, I know I was operating a sandbox and therefore I left
Speaker A
out some cybersecurity precautions or I had some unsafe shortcuts because I just wanted to test here a particular subsystem of my AI agent and not the complete system and not the complete run or we had some actions that were, yeah,
Speaker A
you got the idea, now. And of course, here we only have the the reward, the final reward tells us only success or no success. And this is again a binary, if you want, factor like here years ago in reinforcement learning.
Speaker A
So, the evolution method must infer which parts caused the success and which parts should generalize and which parts should under no circumstance be allowed to generalize because this is here the problem.
Speaker A
So, if you cannot separate them, it may compile out the complete causal package into a skill with all the shortcuts here that cause here some massive cybersecurity problems, yeah?
Speaker A
So, you see the fundamental problem is not just a malicious memory. It is also the unsafe abstraction under the weak supervision here of our failing credit assignment. So, this is not just an operational problem that we have. This is a major theoretical
Speaker A
problem in the basic building blocks, how we build here this skill.md file, this self-evolving agent structure. This is something that is unsolved at the moment here for these agents.
Speaker A
Yeah, if you want to put everything here in the final statement, I will refer here to the paper. So, I follow here a little bit the authors.
Speaker A
Under some control adversarial conditions, several current skill evolution system, and I showed you the 21 combination, can transform, a very friendly formulation, limited unsafe experience into some persistent procedure, our skill.md file, transferable procedures that can be reloaded by other systems,
Speaker A
where you put the skill.md file on the internet, and my goodness, you pollute here thousands of [clears throat] other AI machines, and common utility-driven evolution, and this does not reliable remove the risks that you have in your self-generated auto-generated
Speaker A
skill.markdown file. So, you see the data show you there is only a small distinction if you go with cloud code, or you go with Hermes, or you go with any of those self-evolving systems.
Speaker A
They all make mistakes. So, great. So, let's come to the summary. So, in my last video, I showed you how an agent that can use some good-looking skill behave badly.
Speaker A
Okay, if you've seen my last video and in this video shows that an agent can learn a bad skill from a successful looking experience, from a successful executed trace, from a successful task execution, but it was not able to really understand
Speaker A
all the dependencies, all the boundary condition, all the special conditions that apply to the system as a complete system or subsystems here, because my intention, what I wanted to test with this particular run, was not explicitly made available to the AI machine.
Speaker A
And therefore, the self-learning process of this AI machine is yeah, not working at all.
Speaker A
So, practices makes our skill more and more unsafe. Which leaves me now [laughter] with a single question. What does it mean?
Speaker A
What What What is now the the insight from this? And I'm a little bit frustrated after reading now the second paper that skills are a ticking time bomb here in our system. And we have not mastered skill, we do not understand how to make
Speaker A
it safe. And the current cybersecurity measures all fail on this delayed propagation here. So, the simple question now is, does this mean that skills must be governed across the entire life cycle?
Speaker A
So, we have to make find another methodology when an AI decides autonomously in a self-learning agent, "Hey, I build now a new skill from what I learned." But somehow we have to intervene in the system and say, "Hey,
Speaker A
stop. You have to do some testing. You have to really understand the environment condition, the boundary conditions, and and and." And if I say and and and, I mean what I just can't think right out of the out of
Speaker A
the box right now. We have to audit here the machine, what experience is learned, audit what skill is allowed to be written down at all. We have to validate when it will be retrieved this particular skill markdown file from the
Speaker A
skill libraries, attribute what happens after the reuse, and revoke it when the evidence turns negative.
Speaker A
So, you might say, "Okay, and who should code this? Who should implement this?" Well, I guess these are the next weeks that we still have left from summer time because otherwise, I don't know, no.
Speaker A
So, I think we can end this paper here with a central conclusion. And please have a look at the paper. There's so many more experimental data details that you can have a deep dive into, but just the main idea, the main conclusion is
Speaker A
safety must govern not only what an agent does, but also what the agent learns autonomously. Whenever the agent learns, decides to learn something new, and put it down in memory, and put it down in a skill markdown file, or put it
Speaker A
down in any other file system configuration, we have to check, and we have to validate what the agent learns to do again.
Speaker A
And this is now an additional complexity that is just amazing, and currently I have no idea how to do this. So, here I am shouting out to the community, help.
Speaker A
Has anybody found any solution for this because this new paper here, just published 2 days ago, is really giving us an indication what is all not working. What is not working with self-evolving agent, what is not working with recursive self-improving agent,
Speaker A
what is not working with memory testing, what is not working with skill generation by agent himself, and what is not working, yeah, you just continue the list.
Speaker A
So, I hope you [laughter] have a beautiful a summary, some ideas how I currently feel. Please read the paper. This was just a highlight. You know, if you go a level deeper, there's else waiting for you there in the experimental data if
Speaker A
you have a closer look. So, yeah, let's have fun with VR, and I hope I see you in the next video.
Topics:AI agentsself-learning AIskill mis-evolutioncybersecurityAI safetypersistent skillsdelayed propagationLLM agentsskill evolutionunsafe AI behavior
