Speaker A
Hello community. So great that you are back. It is August 15, 2026, and we do have a problem with skills. Let me explain. Now, we always had this idea that skills are kind of a critical time bomb. But now today, we really have a
Exploring skill-induced failures in LLM agents, this video analyzes a 2026 study revealing how relevant skills can cause incorrect AI behavior and efficiency regressions.
Ask about this video. Answers come from its transcript only — with the timestamp, so you can check them.
Generated from the transcript and can be wrong — check the timestamp.
Key Takeaways
- Relevant skills can induce failures by causing incorrect or incomplete task execution in LLM agents.
- Compactification of complex workflows into skills can omit critical details, leading to errors.
- Agents may over-rely on skill instructions over their own knowledge, causing contradictions.
- Every skill should be checked for compatibility, necessity, and cost before use.
- Skill-induced failures include both functional errors and efficiency regressions impacting system performance.
What the video covers
- The video discusses a 2026 study by HU Song University of Science and Technology and Microsoft on skill-induced failures in large language model (LLM) agents.
- Skills, though relevant and well-matched by cosine similarity, can cause agents to omit or incorrectly implement task elements, leading to wrong answers.
- The study identifies 307 failure cases, including 125 functional failures and 182 high-confidence efficiency regressions related to skill compactification.
- Skill compactification refers to compressing complex workflows into skill markdown files, which can lose critical boundary conditions and cause errors.
- Agents may trust skill instructions encoded in markdown files more than their own parametric knowledge, leading to contradictions and failures.
- The paper highlights the importance of checking skill instructions for compatibility, necessity, and cost before execution, with cost being a surprising factor.
- Skills are reusable knowledge bodies optimized for LLM context windows but require thorough validation to avoid incomplete or incorrect workflows.
- The video emphasizes that skill failures are not due to irrelevant or contaminated skills but due to seemingly relevant skills that are incomplete or improperly compacted.
- The author experiments with new designs to understand and mitigate skill-induced failures and stresses the need for checks to warn about conflicting defaults.
- Overall, the video provides insights into the complexity of skill management in AI agents and the risks of blindly trusting skill implementations.
Chapters
- 00:00Introduction to Skill-Induced Failures in LLM Agents
- 02:18Experimenting with New Designs to Understand Agent Failures
- 04:28Importance of Checking Skill Instructions and Costs
- 08:40Understanding Skill Compactification and Its Risks
- 15:23Insights on Skill Relevance and Failure Causes
- 21:04Impact of Skills on Task Cost and Efficiency
- 27:43Towards Better Skill Management and Validation
Full Transcript — Download SRT & Markdown
Speaker A
look at the failure class that are induced by skills in our LLM agents. So here we have it. This is the study I'm going to talk about August 12th, 2026 published here by HU Song University of Science and Technology, Microsoft and
Speaker A
look at the failure class that are induced by skills in our LLM agents. So here we have it. This is the study I'm going to talk about, August 12th, 2026, published here by HU Song University of Science and Technology, Microsoft, and
Speaker A
"Hey, we just load here relevant skills." No, but they often make the agent incorrectly implement or even omit some task required implementation elements and then our system will provide incorrect answers. So let's have a look. Yeah, the author is here from
Speaker A
University of Illinois Champagne. Agent skills can be harmful. A study about the skill-induced failures in LLM agents, and the idea is really absolutely fascinating because it is not that we load some irrelevant skills or some contaminated skills. No, we go and say,
Speaker A
The preprint studies here the skill induced agent failure. the complete agent fails a failure class here induced by our skills and they put build a particular data set and what they really found is hey wow you know we have 307
Speaker A
"Hey, we just load here relevant skills." No, but they often make the agent incorrectly implement or even omit some task-required implementation elements, and then our system will provide incorrect answers. So let's have a look. Yeah, the author is here from
Speaker A
have 182 high confidence efficiency regression here with our agentic failures. They are also absolutely fascinating if you're interested in the cost in the token of your system. So let's have a look. Let's start. Now you know I always try to learn a little bit
Speaker A
Song University of Science and Technology, real close to Wuhan. So here we have it. China, Hong Kong, Shanghai, Beijing. And here this is our university here. Beautiful. So let's have a closer look.
Speaker A
experiment here in this video. I say okay I want to bring in maybe a new design for understanding a complex topic like here agent induced or skill induced agent failures. So let's see if it works.
Speaker A
The preprint studies here the skill-induced agent failure, the complete agent fails, a failure class here induced by our skills, and they put build a particular data set, and what they really found is, hey wow, you know, we have 307
Speaker A
previously in another example in another system for another topic in another task at another complexity and now we say okay so we use this skill we reuse this skill we upload the inform skill bank from some I don't know a repo from the
Speaker A
confirmed cases of skill-induced agent failures, and I said, oh, that's great, and they have here even distinction because they have 125 functional failures that are really interesting to have a look why here the agent fails, and then they
Speaker A
effect of a compactification that is now causing massive errors in our skill implementation. H so this means the compactification of a skill introduced by a complex workflow. So something that really worked previously beautifully but this compactification now in a skill markdown
Speaker A
have 182 high confidence efficiency regression here with our agentic failures. They are also absolutely fascinating if you're interested in the cost in the token of your system. So let's have a look. Let's start. Now you know I always try to learn a little bit
Speaker A
So an agent may trust here that a generic instruction encoded in a skill markdown file more that its own inherent knowledge more than its own parametric knowledge of the LLM for a specific task. You know this is because this is
Speaker A
also from my own videos. So remember last time we talked about our design harness and how to make it interesting, how to optimize a design in a communication, and I will try to implement this. This is now my
Speaker A
our requirement with the skills default but you know what sometimes it omits here the detail that were absent from the skill template itself or the skill map itself. Maybe it chooses here a wrong implementation path or turn some
Speaker A
experiment here in this video. I say, okay, I want to bring in maybe a new design for understanding a complex topic like here agent-induced or skill-induced agent failures. So let's see if it works.
Speaker A
out in this study they say the harm of a skill comes from absolutely relevant skills. skills that have been given to us by a cosine similarity that is really spot-on top-notch but being obeyed here by our agentic system to literally and
Speaker A
Let's start here with understanding what is a skill so that we're all on the same level. Now, a skill is something that is compressed somehow in some code for an AI because this sync whatever it worked, let's assume it's a workflow, this worked
Speaker A
access to this. So therefore we use now a skill template or a skill markdown file as the complete workflow. But of course you have to define everything about this workflow. The temperature, the condition, I don't know any other
Speaker A
previously in another example, in another system, for another topic, in another task, at another complexity, and now we say, okay, so we use this skill, we reuse this skill, we upload the inform skill bank from some, I don't know, a repo from the
Speaker A
This representation might not be complete and this will cause massive problems in the next minutes. So design lesson here from the preprint is simple.
Speaker A
internet, wherever you find it, but you know what, it is interesting because the authors found that often this skill is not complete. Sometimes it loses you the boundary condition that made it correct in the first place. So we do have the
Speaker A
as a potential hypothesis. No, something that might fail but definitely not as a validated workflow. And this is now an absolute disagreement with with I how I used your skill. I saw the skill is here if you want an encoded solution. So we
Speaker A
effect of a compactification that is now causing massive errors in our skill implementation. So this means the compactification of a skill introduced by a complex workflow. So something that really worked previously beautifully, but this compactification now in a skill markdown
Speaker A
The orers tell us every skill instruction should be checked for compatibility, necessity, and costs before execution. And cost was really the surprise element in this preprint.
Speaker A
file introduces now unknown degrees of unknown. And if you think, what is he talking about? Well, let's have a look.
Speaker A
You have some metadata with the name rack back end helper and the description. You need this description that this skill is found here because co and similarity here. This is used for backend rack application. Then you have the official skill name. Hopefully this
Speaker A
So an agent may trust here that a generic instruction encoded in a skill markdown file more than its own inherent knowledge, more than its own parametric knowledge of the LLM for a specific task. You know, this is because this is
Speaker A
But now allow me to reframe this. We have to change here our understanding a little bit our perspective to have a little bit more specific understanding here. Now a skill is usually introduced to an AI as a reusable knowledge a body
Speaker A
why we have skills. Skills live in the harness of our agent and therefore have an optimized data preparation for the context window of our LLM at the core of the agent. So therefore say, okay, yeah, let's go with this. So the agent fills
Speaker A
performs the internal validation. It says this is correct and it turns out if you have an external evaluation it is incorrect. This is a contradiction in itself and it is beautiful and explains some of the things that I encountered
Speaker A
our requirement with the skills default, but you know what, sometimes it omits here the detail that were absent from the skill template itself or the skill map itself. Maybe it chooses here a wrong implementation path or turns some
Speaker A
what the agent notices, how it interprets the task itself, which file it has to edit, which tools it will invoke, what it considers here a correct solution or not, what are the parameters for the evaluation, how much verification it will perform. And we
Speaker A
optional checks into mandatory work. So something is happening that is not good. So a skill, let's say we are now looking for an explanation why the harm of a skill rarely comes from obvious irrelevant skills because as it turns
Speaker A
module tells you hey we reached here our threshold this is a valid result. So therefore I would say or argue that the correct conceptual model that we should view as skill is and now hold on to your socks. A skill is an inference
Speaker A
out in this study, they say the harm of a skill comes from absolutely relevant skills, skills that have been given to us by a cosine similarity that is really spot-on top-notch but being obeyed here by our agentic system too literally and
Speaker A
loading here the skill let's say the skill s we do have a probability distribution that depends of course on the model itself because the model is doing the reasoning the harness itself the x this is the the input query and
Speaker A
not task specific. So this is an interesting case. Think about it. We do have a parametric knowledge of the core of the agent, our LLM. But since we are working here in another structure in the harness of our agent, we don't have
Speaker A
So this means if we load a scale the model weights of the LLM and the core of course remain unchanged because we operating on the harness structure but the probability distribution over the action they will definitely change.
Speaker A
access to this. So therefore we use now a skill template or a skill markdown file as the complete workflow. But of course, you have to define everything about this workflow. The temperature, the condition, I don't know any other
Speaker A
structure here on the query on the particular skill skill densities given here all of this now produce here a particular trajectory and in this trajectory we also have here the action the decision that are taken by the LLM
Speaker A
parameter that is relevant. And if you just forget one single parameter from an experiment that worked yesterday for you, you abstracted this successful run into a skill representation.
Speaker A
now produces here a specific trajectory to S drawn here from a distribution of possible trajectories and in this trajectories of course you have here the decisions that are taken by the LLM which action is now the next action that
Speaker A
This representation might not be complete, and this will cause massive problems in the next minutes. So design lesson here from the preprint is simple.
Speaker A
not in a rack system. It really changes here the path through which the agent reaches for the answer or it induces here a wrong idea that will make here the agent fail here to reach the solution.
Speaker A
No, the skill retrieval from somewhere, database, internet, some repos. A skill retrieval is simply not enough that you really can trust your AI system that now your job will succeed. A skill should be treated here as the orders node,
Speaker A
will produce a changed trajectory. Yes, of course, this is what we want. We want that our um LLM that is frozen. It does not know the right mathematical procedure learns this mathematical procedure, this mathematical workflow even described in a skill markdown file.
Speaker A
as a potential hypothesis. No, something that might fail but definitely not as a validated workflow. And this is now an absolute disagreement with how I used your skill. I saw the skill is here if you want an encoded solution. So we
Speaker A
However, the orers and here your screenshot from the paper have now different evaluation ideas here. The skill setup the evalation of this and the result with no skill and high cost and multiple skill activation and everything. I don't want to go into
Speaker A
have here workflow. I know exactly what mathematical methodology to use. What is the workflow? What are the parameters that I have to fill in? And then I get a correct result. Turns out I was wrong.
Speaker A
Then they had augmented benchmark with skills without skill, agent execution, verification, internal external verification, data lab, functional failures, efficiency regression. Yeah.
Speaker A
The orders tell us every skill instruction should be checked for compatibility, necessity, and costs before execution. And cost was really the surprise element in this preprint.
Speaker A
No, we filtered out all the bad skills, all the contaminated skills, all the skills you can download from the internet where the cyber security system tells you red alert. Yeah, those are good selected pre-selected confirmed valuable skills.
Speaker A
So you know a skill. Now if you have never seen a skill before, this is it.
Speaker A
We have an environment mismatch, broken dependencies, a broken runtime, environment state mismatch. We have obstructive workflow guidance. We have incorrect requirement element fill requirement element omission and some artifact misplacement. And here you have all the definition and all the accounts
Speaker A
You have some metadata with the name rack backend helper and the description. You need this description that this skill is found here because cosine similarity here. This is used for backend rack application. Then you have the official skill name. Hopefully this
Speaker A
at a global corporation but I have some experience this and whenever there is something you don't trust your internal expert now you hire here an experienced consultant and you know the four top consultants in the world so and this
Speaker A
is unique. Then you have the procedure body. Then you have the validation guidelines, supplementary materials, examples and templates and Python files and whatever you have. Those are all your skill components that you know.
Speaker A
your problem is we identified it and we have a solution for for you. I mean, of course, you pay them six figures. So, yeah, you hope that they have a solution. But you know what's happening?
Speaker A
But now allow me to reframe this. We have to change here our understanding a little bit, our perspective to have a little bit more specific understanding here. Now a skill is usually introduced to an AI as a reusable knowledge, a body
Speaker A
And this is their standard workflow. This is their standard solution. And guess what? It will not work in your particular company because your company is not absolutely 100% identical to the other companies they have been to.
Speaker A
of knowledge that consists now of particular instructions, of particular examples, of particular default values, procedures, given work structures, workflows, and of course validation rules. We always need the validation rules. Now it turns out that the validation rules are there. The skill
Speaker A
solve. So they say, "Hey, we don't have to investigate it anymore. We know the answer. The answer is ABC. And therefore we tell you you have to do ABC and this is it." Turns out this is absolutely incorrect. also for the real world case
Speaker A
performs the internal validation. It says this is correct, and it turns out if you have an external evaluation, it is incorrect. This is a contradiction in itself, and it is beautiful and explains some of the things that I encountered
Speaker A
I understand what is the solution and we have the relevant understanding and the solution for this it will not mean that it is also compatible with your specific configuration of your company or of your AI problem.
Speaker A
with my skills. But just think about it now from a pro-al point of view. A skill is no longer a passive documentation. A skill is something that is actively integrated into the workflow of the complete agentic flow. No, it will influence here
Speaker A
successful run of another AI system, there were certain parameters. Let's say hundreds different parameters were on default or had specific values and then it was abstracted the skills encoded compactified compressed further every compression you lose you lose data you
Speaker A
what the agent notices, how it interprets the task itself, which file it has to edit, which tools it will invoke, what it considers here a correct solution or not, what are the parameters for the evaluation, how m
Speaker A
with the runtime environment or with the available budget. You say hey I just have 35 minutes per run and not 3 days.
Speaker A
So the insight first number insight is the principal problem was not that the agents loaded obviously irrelevant skills or contaminated skills or cyber security critical skills. No, it was they looked relevant. The cosign similarity from their description was
Speaker A
absolutely spoton. But it turned out that the relevant skills uncontaminated beautifully written with the best intention of a human or AI order were dangerous. Why?
Speaker A
The orders show us that the dominant failure here was the task implementation itself. And this was in 86 of 125 functional failures. It was the task implementation. So this means the relevant skills cause the agent to implement the required elements in an
Speaker A
incorrect way or to completely omit them partially or completely or become trapped in an obstructive workflow that does not fit the actual task. So what is happening?
Speaker A
We have now that the harness defines here let's call it simplest way a a workflow and this is it. You have to follow this. forget your parametric knowledge of the LLM because the LLM knows a lot of but we have now this is
Speaker A
the override to your AI knowledge you have to follow the skill instruction word by word or token by token and guess what if you lose intelligence and you come back here to let's quote here a template structure guess what you will
Speaker A
encounter problems now I know that we have in the past Wherever you go here in the literature, you know, you select the skills from a skill database or a skill library, skill bank or whatever. We have a semantic
Speaker A
similarity. No, because you have description here what the skill is doing, what it needs, on what domain it operates on, everything. But think about the semantic similarity answers what exactly in what mathematical space. This is a vector space. And this is just more
Speaker A
or less the question. Hey, is this skill about the same topic as my task as my query?
Speaker A
And you know what? In a lot of cases, the answer might be yes. And then and then we all look away and say cross our fingers and say, you know what? Now we leave the rest to the intelligence of
Speaker A
the agent. Yeah. Because the agent will now figure out if I load 25 skills, the agent will now select the best of the 25 and find all the correct parameters of the 25, implement the correct parameters, implement the correct
Speaker A
workflow, makes sure the temporal dependencies correct, implements this here in the general workflow of my project. And of course this is why I have NEI. And then 3 days later after two days of debugging you find out nothing of this was true. Everything
Speaker A
failed. There is no intelligence of the agent. If it was not trained for it was not in the pre-training data or in the post-training data. The agent failed completely to implement it.
Speaker A
But safe skill use requires here something else because we now have to answer a different kind of question and the question is are the assumption inside skill compatible with every binding requirement of our particular query of our particular task in our
Speaker A
complexity of our domain knowledge. If not the skill looks good has a similarity cozen similarity of absolute topnotch and is not able operationally to fulfill the task.
Speaker A
So this is the part of the paper I do not like this brutal insight. I mean we know it all along but they really show us that it is collapsing.
Speaker A
Take a step back. Think about what is what is the skill mechanisms you and I and maybe the rest of the world implemented up until now. No, you had a skill and a skill supplies here a reusable representation of a particular
Speaker A
task of a solution of a task that ran yesterday the day before yesterday successfully. So we extracted the skill.
Speaker A
No, we said okay this is here the workflow. Let's go with a workflow example. And this is now the compactification into a skill. And the skill is now a representation of the correct pause from the given complexity at the beginning of the problem to the
Speaker A
final solution and the complete pathway. Now this representation becomes now if you want cognitively outstanding. Yeah. And the agent now maps the task onto this very specific representation. But remember this is just a limited representation. You cannot map here the complete complexity
Speaker A
of the successful runs into a compressed representation. So this means details will not be represented by the skill and they will become easier to omit and even if they're represented some EI system will just ignore them or overwrite them or
Speaker A
misinterpret them or misunderstand this is not a percentage but this is something else. And even if the agent then verifies its own internal coherent solution with its own internal parameter with its own intel verifier the yeah will the agent will come back and says
Speaker A
yes it is verified and then you have an external verification by another agent and it tells you this is absolutely incorrect.
Speaker A
So this is the beautiful loop we have currently with skill implementation. Yeah. So remember the main cause is simply we took here something that worked in a particular environment in a particular computer environment in a particular complexity and we compressed it. We
Speaker A
compressed it into a particular representation and you have either a code representation or you have a verbal representation or you have mixed representation you have visual representation it doesn't matter but whatever compression you have whatever compactification you have you lose
Speaker A
information you lose data. So we do have past solution we have a compression and we have now a SQL representation in a markdown file or HTML or whatever you like but the compression removes some of the boundary condition and if you sync a
Speaker A
physical system and suddenly you're not operating within the parameters of your experiment but suddenly you think that this is a general example of all parameter that are allowed then you immediately understand what is wrong here.
Speaker A
So beautiful. So skills from the internet from a database. Well, yeah, there is no major problem.
Speaker A
So the artist set up a particular experiment, a particular testing. So they said, you know what we keep fixed?
Speaker A
We keep fix the LLM, the model, the agent framework, the tools, the repo, the container, the input data, the verifier. The only thing that we modify and that we really evaluate here in this configuration is the skills. the skill
Speaker A
configuration singular and plural. So as I told you we have two forms of problems, two forms of harm that are caused now by skills. The first cause of harm [laughter] is if you want the skill that you implemented is a correct skill
Speaker A
that worked yesterday for you but now you have a little bit of a modification in your system and you hope that the eye is able to compensate for this and it turns out no way that this is happening.
Speaker A
The skill will change the solution. It will change the result. It will change the trajectory. It will change the outcome. And the study found 125 functional failures. So if you want to have a deep dive into this 125, have a
Speaker A
look at the study. I just give you the dominant problem that we identified here, the corrupted implementation of a skill. Because a task implementation incorrect fault here accounted for 86 of 125 cases. So roughly 70%. And from the
Speaker A
70% you have 46 cases incorrectly filed a required element or filled a required element. 36 cases omitted a required element. Just the I decided no it's not so important and four became obstructed here by the prescribed workflow.
Speaker A
The second form of harm is interesting. This was a surprise because they say you know now we look at time we look at budget we look at costs.
Speaker A
that the skill changes the cost structure of your task and the paper identified 100 additional in 182 high confidence efficiency regressions. So this means if you have a run with different skills the runs pass but the skill run may be increased both the time
Speaker A
and the token use significantly with at least one more than doubling. So if you have either the double the time or the double of the token, so instead of 10 millions tokens, you have now to pay for 20 million token usage, you are not
Speaker A
going to be happy. As I told you, they identified here 182 high confidence efficiency regression.
Speaker A
The main category here was excessive procedure of the agent in 62% of the cases. This means 114 out of 182 and one of these out of this 114 17 cases of excessive exploration. Yeah, I started to explore here the mathematical space
Speaker A
of solutions here and just went on exploring exploring exploring pseudo cases with a heavy implementation pipeline. You just go there and I will show you something. And one is the absolute dominant effect with 67 cases of excessive verification. And because
Speaker A
if we have 67 cases of excessive verification, this is an indication our agent has been over trained, over safeguarded, over guardrail or whatever you want to call it. Yeah. Here you have this 182. We have category context bloat an excessive
Speaker A
procedure and excessive verification here 67% 67 cases here the skill leads the agent to run excessive or repeated testing repeated debugging repeated rebuilding or checklist verification of the implementation. If you have seen my testing videos of the different LLMs,
Speaker A
you see this is exactly when you see in the reasoning trace. Hey, wait, let me recheck. Wait, let me recheck this again. Let me recheck the recheck of the again of the recheck that we have rechecked already five times. This is
Speaker A
here our guardrails that have been reinforcement learned by human feedback or by GRPO where the creators of our AI agents wants to make sure that the agent is behaving correctly and this excessive verification is now also integrated if
Speaker A
we have here a skill execution and it will bite us into the you know where so let's look at skill scale become now real different we have a different view on skills. Now a skill is if you think
Speaker A
about it a little bit reframed again a compressed policy that we have no a skill is a compressed policy derived from previous successful behavior of a particular agent of an LLM. It improves the performance when this new task that
Speaker A
we want to implement it shares both the topic and all the hidden conditions of those previous situations. So if you have a numerical simulation of a star or for the expansion of a star of a supernova and you have absolutely one to
Speaker A
one the same identical program you can say okay I use here the same skills but the skill will become harmful immediately when the topic matches yes we're in the same domain we're in the same complexity but the conditions do
Speaker A
not match anymore and even one parameter that is off is enough to make the skill not helpful at all.
Speaker A
Now the authors here they come up with a new workflow of a skill triage and this is a beautiful idea of the differential evidence and the attribution which ski what happens here what reasons uh go wrong what are the failure modes what
Speaker A
are failure classes here of skill inefficiencies if you want to have a look I just tell you this is a postfailure diagnostic system so it helps us answer why a skill caused harm it does not explain hey what I have to
Speaker A
do in my next task because I don't want that my skills that I sort are helpful are destroying my task.
Speaker A
So if I want to have all the information skill triage have a look at the paper itself. Beautiful. Now for me now I have read the paper and I said oh great.
Speaker A
So I think I have to implement something there. I have somehow code now a system where I say that the task authority so the importance of my particular task let's say science something is now more important this must precede now or
Speaker A
exceed now the skill that I upload here in authority it must have alpha priority to the skill authority itself because if the skill authority can override here the intelligence of my particular trained LLM you know something is terribly probably. Yeah, it's going
Speaker A
wrong. Also, if the skill authorator that lives in the harness is overriding some other files that also live in the harness and is overriding this harness constitute, this harness component, this is also not helpful. So somehow this will be this will [snorts] be a
Speaker A
challenge. Second, somehow I have to test for compatibility of this new skill that I download from the internet, from other universities, from friends, whatever. I have to check for absolute compability given all the parameters of my task before I even
Speaker A
should think about executing this particular skill. And third, yeah, this opened really my So the skill need costs in the background. It is great to have a skill, but if the skill burns five times or 10 times as much token as another
Speaker A
skill and provides the same correct result, why should you pay 10 times more? And I do not want to mention that some of the skills by our propriatory global AI corporations have maybe theoretically a tendency to burn more tokens than absolutely
Speaker A
necessary. And of course the agumentation is only to keep you safe. So future research to action. uh what is there coming up with skills if you want to continue use skills but now we learn from all the failure and failed
Speaker A
implementation of skills I think three skills and I go also with the orders of the paper so we have to have skill task compatability checks no because many functional failures arise simply from topically relevant skills that distorted the task required
Speaker A
implementation elements no so we have to check for the skill content and the compatibility before we can think about executing this file within our programs.
Speaker A
Now, and somehow we must find here checks or security or whatever that warn us when a skill will introduce conflicting defaults or that anything is going wrong with the implementation of the skill but with a sensitivity level that is unseen up until now regarding
Speaker A
example templates path and you got it. What is really interesting is cost aware skill packaging and selection. No, I never thought about this. But yeah, skill can cause extreme cost explosion here because it is token maximizing here or you go into a verification 15 times
Speaker A
into the same loop. So yeah, always loaded context that some of the harnesses continuously have to provide to the LLM because the LLM is a stateless function. This is really something I have to think about when I have to bring here this new intelligence
Speaker A
or this new methodology this new pattern into the parametric knowledge here. So really train here the tensor weight of the LLM structure train here really the transform architecture and not leave the intelligence in the harness region. I have to bring it down into the LLM
Speaker A
otherwise yeah I just make here the tokens explode because some of the physical task if they go for a week or so this is here yeah it's a cost factor no again budget aware execution policy against excessive verification
Speaker A
agents need explicit policies somehow for adapting here yes you need a verification scope but somehow it must be limited and I think if you have verified it three times I I mean why 15 times come on sometimes your guardrails
Speaker A
are just becoming nonsense implementation pipeline depth is another topic here from science but yeah more about this later so this is it I think this is here the the main idea of the study here hope I've given you some
Speaker A
first feeling what is happening currently here with skill skill sensitivity skill compatibility skill yeah yeah cost awareness I think there's something really changing here quite massively currently and maybe a skill definition a new skill definition will come up in some other network I don't
Speaker A
know yet but it seems what we how we currently use and implement and utilize skills is not the way forward I hope to see you in my next
Topics:LLM agentsskill-induced failuresAI skillsskill compactificationagent workflowsefficiency regressionskill validationAI task executionskill costlarge language models
