Outsmart Cloud AI Models w/ a Local Scaffold

Learn how to optimize local LLMs with harness structures and mathematical scaffolding to rival cloud AI models.

Ask about this video. Answers come from its transcript only — with the timestamp, so you can check them.

Generated from the transcript and can be wrong — check the timestamp.

Key Takeaways

Local LLMs combined with optimized harness structures can rival cloud AI performance.
Mathematical scaffolding is crucial for efficient data representation and computation in AI harnesses.
The Adam framework provides a model for integrating retrieval and analytical memory in AI systems.
Harnesses handle data pre-selection and prompt formatting, enabling task-specific AI optimization.
This method reduces dependency on costly cloud models by leveraging local computational resources.

What the video covers

The video explores running large language models (LLMs) locally using harness structures instead of relying on expensive cloud AI models.
It introduces the concept of mathematical scaffolding to optimize data preparation and computation within local AI systems.
Discusses the Adam framework, which integrates retrieval and analytical memory for temporal data dynamics.
Explains how harnesses manage data input, context window selection, and task-specific information routing for LLMs.
Highlights the importance of geometric scaffolding to represent task-relevant data complexity in a navigable computational form.
Emphasizes that local harness optimization can achieve results comparable to cloud-based AI without incurring high costs.
Describes the role of structured coordinates in high-dimensional mathematical spaces to efficiently route AI computations.
Clarifies that this approach is distinct from mixture of expert models and focuses on internal agent-harness optimization.
Mentions retrieval augmented generation (RAG) as part of the harness data preparation strategy.
Encourages community feedback and further exploration of these concepts.

Chapters

What is the main advantage of using a local harness with an LLM?

Using a local harness allows you to optimize data preparation and computation on your own machine, reducing reliance on expensive cloud AI models while maintaining competitive performance.

How does mathematical scaffolding improve AI harness performance?

Mathematical scaffolding provides a structured, high-dimensional representation of data complexity, enabling efficient routing and processing of information within the harness for better AI task execution.

What role does the Adam framework play in this system?

The Adam framework integrates retrieval and analytical memory to model temporal dynamics in data streams, enhancing the harness's ability to provide accurate and context-aware AI responses.

Full Transcript — Download SRT & Markdown

Speaker A

Hello community. So great that you are back. You may say, my goodness, what are we talking about today? Well, imagine you can pay for the most expensive LLM and you go here with Claw Code, one of the most expensive harness structures, and you

Speaker A

said, great, I just pay for everything. But what if you say, hey, I want to have this on my local machine. I don't want to go in the cloud. I want to have a local LLM and I want to have a harness

Speaker A

that is running here locally on my network. And then we need to be a little bit smarter than the rest of the others because then we have to care about the mathematical scaffolding optimization. So the data preparation,

Speaker A

how we prepare the data for our harness and how the harness can operate on a small local LLM on a small local machine to provide solutions that are as good as the best cloud-based models. So today we try to outsmart the cloud-based AI

Speaker A

system with a local optimization theorem. We have to understand this. And of course, in my last video, where we talked about the topological condensation and the next phase of the agents, you noticed paper number five. This was not really presented. So I got

Speaker A

some replies here from you, my beloved you, and said, hey, what is it about this Adam framework that jointly represents retrieval analytical memory structure?

Speaker A

Can you just explain it? What a beautiful idea. So answering the user queries may require either retrieving past specific events or performing some analytical operation such as temporal selection and temporal aggregation. So what we do now, we want to have a look

Speaker A

here at the temporal dynamics that is also now to be modeled within this AI system so that we get here the perfect result. And this is here more or less exactly what Adam is trying to provide to us. But the way it is done fits

Speaker A

perfectly in our discussion from my last video and in today's video because it kind of provides further insight. So we go with an analytical memory and a retrieval memory like we discussed here in my last video. What is really

Speaker A

interesting is that Adam here, one of the main jobs is to discover recurring field structure in the temporal dynamics here of the data stream. And yes, exactly. We are back here to the perfect mathematical topological representation here for some analytical access. So

Speaker A

let's have a closer look. By the way, yeah, it has here in the definition some provenance linked. And you know, English is my third language. So I said I have to look up provenance linked meaning here. Well, easy connects data, digital

Speaker A

assets, or physical objects directly to their documented origins, their creation history, and chain of custody. So it ensures full traceability, verifies the authenticity, and tracks how information or items transform across network owners over time. Got it? Beautiful. Let's go

Speaker A

with the structure. Now the difference is that at inference time a memory-aware planner decomposes n queries into some retrieval operation and some analytical operation given its specific harness definition and routes each operation to the appropriate tools in your harness structure. Let's talk

Speaker A

about this at first. You notice this is not done by the core LLM but the core of your agent. It is done by the harness.

Speaker A

So therefore remember this picture from my last video. We had the LLM. Yeah, the pure tensor structure, learned, pre-trained, beautiful. But then we have to have the harness construct like a Claw Code or whatever you go with that

Speaker A

kind of prepares the data for the input to the restricted context window here in the prompt of your LLM. And now, as presented in my last video, we have to build here a mathematical scaffolding for the numerical computation within our

Speaker A

ER algebra. Now, Adam solved this here with a schema-induced analytical memory. But we want to go a step further and remember in my last video I tried to explain to you the difference what it is and I used here

Speaker A

this image here from my background, physics, CERN particle accelerator, and I told you the harness dictates how the active beam DLM itself interacts with the detection medium, the scaffold. And I got some questions from my audience there. I said, hmm, what if you

Speaker A

are not a theoretical physicist working at CERN? How can you explain this? So let's do this. So this is now a sentence that I came up with last time, so let's have a look and let's optimize

Speaker A

this. Can I say was the question here that the LLM needs the right data pre-selection and format? This is now handled by the harness. The harness inputs now this specific data configuration to the prompt of the LLM and then that the

Speaker A

harness itself needs the geometric scaffolding to capture the data complexity itself. And I saw that this is a valid sentence here in my last video. But you know, some time passed. I went around a little bit, had a night walk, I thought about this

Speaker A

and I want to optimize this sentence here. I want to provide a better understanding for you and also for me because I'm fighting also to understand exactly on a mathematical level what the scaffolding exactly is doing for the

Speaker A

harness configuration when I optimize this and then how the harness configuration plays into the LM input selector. How is this connected back here to the perfect mathematical scaffolding? How can I find a mathematical optimization theorem to have here all three in one run? So

Speaker A

therefore I'll try to formulate it today in a more precise way. So the LLM needs the harness or the harness was built, the agent was built. Agent, remember, LLM plus an external harness to select, organize, and present the right task-relevant information,

Speaker A

let's say from RAG, retrieval augmented generation, to our LLM context window, to our prompt structure.

Speaker A

Now this nice idea is you don't need to pay more for a more powerful new AI model in the cloud for GPT-6 or whatever comes up, but I think even more important is whether you can do also locally on your

Speaker A

PC, on your notebook, that you build a specific harness structure for your specific complex task that you give to your AI system because your AI machine needs some data preparation in a very specific way that can be done by the

Speaker A

harness locally and you do not have to pay up to a cloud model. This is my idea. This is my motivation. Now, as it turns out that the harness in turn benefits here from a task-specific computational scaffolding as I've shown

Speaker A

you in my last video when we put here this computational lattice structure here from quantum field here over our unknown complexity and this transforms the raw problem now into, hey, just find the structured coordinates here of a

Speaker A

particular flow function here in a high-dimensional mathematical space. And this structured coordinate I showed you in my last videos is not an X, Y, Z axis like some of my viewers questioned here in my last video. No, this structured

Speaker A

coordinate is a complex high-dimensional mathematical sync. It can be objects, can be relations between objects. It can be epsilon neighborhoods defined in a medical way that we understand what does it mean to be close by or nearby and

Speaker A

some other operations I'm going to talk about in about 2 minutes' time. So structured coordinates over which the AI computation can be routed efficiently and no, we're not talking about mixture of expert models or anything like this.

Speaker A

We are just talking here the complexity of inside an agent, the LLM to the harness and the optimization of the harness we compute giving a geometric scaffolding optimization.

Speaker A

So this is the way I see the world and maybe I'm wrong, so please leave a comment. So I say, okay, we have a raw problem. This is here my text query. No, and maybe you need to search 1,000 books

Speaker A

on the internet for this. But if I now optimize here a scaffolding. So what do I create? I create from a pure text blah blah blah space with millions of English words. I transfer, I map them with a mathematical operation into a structured

Speaker A

search space. And let's take an example. This is here exactly what we do with RAG vector representation. We convert a raw text, a word or token or embeddings here into

Speaker A

this is a representation if you want that you know this is what I mean with scaffolding here and we build a structured search space but this is here a simple example from years ago and then if we have this structured search space

Speaker A

you remember what we did we wanted to have here now with a hornous element to have computation on a selected computational region. What does it mean?

Speaker A

Remember in rag the vector space what we were looking for was we have a query and we convert the query via a mathematical mapping into a query vector in this structured search space and then we have a cosine similarity and this cosine

Speaker A

similarity is nothing else than a tool in our harness now so you immediately see here how the complexity is rather simple I just use here some more general words because we will go a little bit higher the complexity in the next

Speaker A

minutes. And if we have now this selected computational region, if we have the top 20 neighboring uh vectors now to our query vector that we identified in this embedding, now we take this simply with a rack system.

Speaker A

Yeah. And with the LLM now that is now provided this top 20 vector representation in its context window in its prompt and now the LLM can find from this reduced set maybe a concrete local solution.

Speaker A

So you see this is all that we do here in the complexity of an LLM with a harness and with a scaffold. It is rather simple. If you abstract it here on a view that I as a radical physicist

Speaker A

might have, maybe you have an even better view. Maybe you have an even clever idea. So, please comment.

Speaker A

Let's focus here on scaffolding. The scaffolding converts what? An unstructured problem in my English blah blah blah 5,000 sentences into something that the hornness can navigate where the harness can play out its powerful construction. Now to find a solution. So

Speaker A

we go from an army girl with based on my query here. This is an enormous raw solution space. This is every possible solution here but not predetected not selected in any way. This is just more or less everything. And we want to map

Speaker A

this into a restricted search space where we have less complexity or a limited volume of the search space. We want to make it easier. We want to make it computationable. So C is now a smaller set of meaningful regions or

Speaker A

equivalence classes if you see this from a mathematical point of view. So this is a simple mapping a normal solution space and we map it here into this much smaller region and this smaller region is here the meaningful region where we

Speaker A

will find a solution. So how we build this? Now this C what is it? C can be in our example here of our five papers here from our five preprints of the last video can be either a mathematical

Speaker A

representation itself like it was in rack can be an objective that we want to optimize remember here the back propagation for example it can be a mathematical optimization in itself it can be about the capacity of our system

Speaker A

or it can be about a new architecture so you see don't limit your mind in the understanding because this is just whatever it can be whatever element you can think of from the complete architecture from a sub architecture to

Speaker A

a mathematical representation of a particular problem in a highdimensional vector space. So rather than asking now the LLM to search everything our omega the harness is now or the main idea of the harness is to select now a

Speaker A

particular element in this set of possibilities and let's go for example with the one the first one this is the representation itself so and then ask the L&M to generate a concrete proposal within that region of a representation where we have chosen a

Speaker A

particular metam mathematical representation in a particular mathematical space with a particular let's say x y z coordinate system I don't know if you know a lot about um general relativity or special relativity from Einstein but also there we had to

Speaker A

choose a particular coordinate system that Einstein enabled to find his new ideas because if we stayed in the classical coordinate system we have a problem but if we modify our view of of seeing this. If we modify the framing

Speaker A

how we address this problem and we find a different mathematical representation that is more on point for our specific problem like say special relativity then the task will become so much simpler and we're doing now the same thing here to computer science AI this

Speaker A

is it now in my sentence of yesterday we also had oh sorry I this this we had here the geometric scaffolding to capture the data complexity I think we I have to correct this. We have to correct it. I have to correct

Speaker A

it. I think a successful scaffold does not preserve every detail of the original complexity because our task in the eye is to make it computationable so that we can compute new probability distribution. But we do not need all

Speaker A

data because we will focus here on the right coordinate system. Let's say here with Einstein to find the right representation for our specific task. So this means a successful scaffolding performs a task dependent course graining. So we go from a raw complexity

Speaker A

that might may include everything of our 10,00 dimension to an operational useful structure. Maybe we just need a 12thdimensional space where we have the most important 12th dimension of our complexity space that represent here a first approximation to the solution.

Speaker A

So therefore I think I have to replace capture the data complexity itself with represent the task relevant complexity in a computational navigable form. So you see we don't need all the data we need just the most important relevant

Speaker A

data for the job and we have to focus on them and there we reduce the complexity reduce the compute time and we reduce that we maybe do not need a cloud-based LLM but a local LLM and maybe my

Speaker A

goodness to my American viewer maybe even a Chinese model an open source model. So again coming back scaffold the scaffold gives the task a structured a new structured representation. It may take the form of a knowledge graph like

Speaker A

we have seen in my last video yesterday or a tool ontology yesterday a skill bank yesterday a new embedding space years ago a hierarchy of memory structures today at the beginning of this video a set of experimental direction pure variance or a

Speaker A

distribution of a possible trajectory. So you see open your mind understand a scaffold is not a limited mathematical object but we are here in a wider interpretation space and a scaffold can be a knowledge graph.

Speaker A

What a scaffold now defines in itself is hey what kind what counts now as a point or a vector in a mathematical space or a complete state description if we have a vector encoding of the state of the

Speaker A

system itself. which states are related. How have we chosen the pre-training data that if a vector represents now the complexity of a complete state? What was in the pre-training data that we defined here as related? Remember in the English

Speaker A

language we went here not with the the the nearness of a word in a sentence but with some semantic complexity. Remember bird and sentence bird model. Exactly.

Speaker A

We do have to define what mathematically nearby means. What is the epsilon environment over a particular vector position in a high dimensional space?

Speaker A

How we define this with a mathematical metric? Which transition, which mappings, which crossovers are allowed at all in this particular space? Do we have here a particular differentiable structure of the space itself? And what solutions are treated now as let's go

Speaker A

equivalent or as good as. Now you know that it's not theoretical physics, it's computer science. So this is here a more simpler case. So this geometry is usually not a smooth mathematical manifold with rotation groups. So particular special symmetries

Speaker A

that we're working with in theoretical physics. Our pre in our five prerints that we had a look at yesterday and today what we are dealing with geometry is mostly a combinatorial geometry a relational one based on the semantics

Speaker A

and we have a probability distribution and a set of probability distribution and also some categorical differentiation.

Speaker A

So we are working in a simpler mathematical space. So therefore in computer science a scaffold does not always need to look explicitly mathematically complex. No think about the video yesterday. We had a textbased skill bank. My goodness. And I got a one

Speaker A

a single question not more. Okay. Uh why is a skill bank here a geometric measure? Think about it. A textbased skill bank becomes a geometric measure because we operate on the embedding similarity. We build this in a mathematical space to calculate the coan

Speaker A

similarity of our embedding function which defines which skills are close to one another. And therefore we hope that we have maybe if this skill is not available another operational skill nearby that we can choose from and that we can activate. Often this scaffold is

Speaker A

often designed over the whole task domain and it is not uh reconstructed or built from scratch for each particular query that I have in mathematics or finance or medicine or whatever you work with.

Speaker A

So what does it mean? Think about it. Let's have a little bit of fun thinking here with our own human brain. It is very simple I know but this question now mutates or morphs into this simple question. Hey, which mathematical

Speaker A

representation makes the required search in our mathematical solution space that we defined the simplest?

Speaker A

What is the shortest computational path that we have to find a solution to my human query that I want to give to my local AI machine here on my laptop? In finding this specific mathematical representations, think about Einstein.

Speaker A

maybe one of the most important sources of AI intelligence in general in the entire system configuration. No, but you know where we are? We at the very beginning of defining how to solve a problem. So this means we are not at

Speaker A

paying for the most expensive I don't know Mutos 5, Fable 5 or Fable 6 or whatever comes next loud EI model. No, we go and say let's be smarter. Let's understand what is the best mathematical representation of the problem that we

Speaker A

want to solve and then we work in this mathematical system like 100 years ago for rack it was a vector space you have to think about the complexity of your particular problem if you have your company and you are specialized in

Speaker A

a particular field in finance mathematics physics medicine whatever to find here the best solution now you can of course train UI system for this particular question and you have now options. Now I know a lot of users or

Speaker A

viewers right that they train or they post train the LLM itself and they think hey this is it this is great I just post train it or I paid OpenAI to train my model if you're a company great but you

Speaker A

know the other option is to train your LLM with a coherent harness structure already in place the sphere around your core LLM this is now your complete agent system and you can train here the LLM in cooperation with the harness itself and

Speaker A

you will have better results than training the LLM itself. Or you can have a very weak LLM, but the harness is so powerful that it analyzes all incoming queries, reduce the complexity of the incoming human query into multiple

Speaker A

lower complexity queries that even a weaker LLM can solve. If it is possible, if it is mathematically separable in your mathematical space or and this is what I want to talk about and which I have to find my way to understand why I

Speaker A

can go now with an LLM plus a particular harness configuration plus I have to think about a mathematical optimization of my scaffolding.

Speaker A

What is think about Einstein the best coordinate system to find a solution for my particular problem? I don't go just and put in a cloud llm hey this is my problem find a solution because this cloud system has been pre-trained only

Speaker A

on all available I don't know GitHub code repos and all available mathematical books on theoretical physics but maybe I need a very specific solution now you can trust like entropic tells us hey trust us we have read everything on the internet we are so

Speaker A

much more intelligent as an AI system than you as a single human being and And I say no I as a single human being I am so much more intelligent and creative to find new mathematical scaffolding since I understand your process of LLM and

Speaker A

harness and scaffolding that I can optimize this also for my local computer infrastructure for my local home PC for my laptop I don't have to pay you in the cloud so this is now my current understanding this is today really this is you

Speaker A

recording this video a scaffold regularizes the representation of the complexity. The harness regularizes all allocation of computation and the LLM generates a concrete solution within the selected region of space.

Speaker A

Now you might say I do not understand the word beautiful. I have here an example for you prepared because I want to share this maybe it helps you in your professional work. Now I could say harness just think about tool use and

Speaker A

here lime just think about rack like we already talked about it no let's make it a little bit more precise no so advanced performance increasingly in my opinion depends not only on the intelligence inside the llm that you pay for in the

Speaker A

cloud but on constructing the right external representation in which constrained relatively simple control algorithms the harness can guide the model efficiently.

Speaker A

Let's do an example. Imagine you ask an LRM to do the following job. Improve this recommended system that you have somewhere in your GitHub. And you know, you might think it's easy. No. Yeah. CL code go.

Speaker A

Think about it. The instruction defines a particular goal, but not the particular search space that this model should operate on. So the IM model has no following possibilities. It can modify either the data representation that is presented to this new

Speaker A

recommended system. It can modify the loss function for a particular specification. It can change the architecture of the harness system. It can modify the optimizer. I don't know whatever you have. Maybe you go with an AdamW or whatever. It can optimize the

Speaker A

rack system, hyperrex system, hyper graph rack system, hyper hyper systems. Whatever you retrieve from external databases or the internet, you can have here for the hyperparameters of the LLM itself. Here the training schedule here different hyperparameters or you go here

Speaker A

with a different model capacity. Maybe you have quantized LLMs. You have a 4bit or what can I go with an 8 bit? Should I go with a brain float 16? What is it? So normally the I goes here or is has been

Speaker A

pre-trained on just change here the code sequence but this is not here for the best solution. Look at what other option you have to optimize this and I think there are thousand of of plausible uh modification and alterations if you have

Speaker A

here a combinatorial multiple of this. So you know that I like here some Gdankan experiment you know. So this is your search space. This is what you see and this is here you on a foreign world.

Speaker A

Beautiful. So it's foggy. So you see here your local environment and you can put a mesh over your local environment and you say hey look there's a little bit of a maximum. No maybe this is here the perfect solution to my problem. But

Speaker A

you understand if you do this like GitHub and CL code this is just something that is real close to you know this is a code optimization problem and maybe it's gorgeous this is all you need but maybe if you just look a little bit

Speaker A

further you see oh there's a valley there there's an absolute minimum that provides you the best solution because maybe my data structure was not optimal and the code was okay all the time or maybe there are different path here to

Speaker A

different other solution spaces, different region in my solution space or in your search space, whatever your complexity you prefer. So maybe I should not stay here on my local hub but maybe go down in the valley because this is

Speaker A

really the minimum here and this is here the best solution and I have different paths that I can go down to the valley know the left one the right one. So you see this is a search problem and a bigger LLM that you pay more in

Speaker A

the cloud does not automatically solve this because it still has even the bigger mile to determine simultaneously how to present your problem in a mathematical way. Which direction to apply left or right? What concrete actions to take first in the complexity

Speaker A

to solve this? whether the action succeeded or not given that you only have limited visibility through the fog and what should be learned from your stepbystep approach. So you see if an LLM has to do this all by itself you

Speaker A

have an act architectural bottleneck and this is why an agent was invented. Huh? Now go a step further and say you know what we have at the very beginning to think about to give the problem a geometry like Einstein.

Speaker A

How do you watch the problem that you want to solve in a coordinate system that it is here everybody else using or you go like Einstein and say hey I choose a particular system.

Speaker A

So this means let's think about the scaffolding and I showed you in my last video we placed the computational latis here over the landscape to have a numerical approximation of the complexity itself.

Speaker A

Now this particular latis gives us here or divides the otherwise continuous ambitious problem into some meaningful region becomes easier to navigate easier to calculate and as I already showed you here our C is here either a new representation or you optimize for a new

Speaker A

objective or you have another data structure that you select here that you built here. So what does the scaffolding does? It redefineses here or it defines here the object in a correct way. You can have here new definition of

Speaker A

relations you have not seen before. Maybe in a hyperraph it gives you here the definition of what is nearby in a neighborhood.

Speaker A

You can have some particular mapping movements equivalent classes representation or stored outcomes. If you still say hey I don't get it.

Speaker A

Think about if you work here with wood on a wooden something. Yeah. You just have to choose the right tools for woodworking. But if you work with an iron piece, your tool set will be completely different. So this is also

Speaker A

here before we go now from here the scaffolding over the tools to the harness. You have to think about the geometry of the problem itself. The mathematical geometry that you want to solve via the harness. You want to

Speaker A

optimize the representation of the problem via the harness for the context window of your LLM. So the harness now observes here the current state. It tries to understand hey what are we in examines now a scaffolding because the

Speaker A

harness has a default scaffolding here in most of the cases. The harness selects a promising direction. Let's go here with the wave on the right hand side. then gives here that constraint sub problem the easier problem to the

Speaker A

LLM. This is why we have here this sphere of a harness around here our core LLM to optimize the data representation for the context window of the LLM. It executes a generated proposal. It is responsible for measuring the success if

Speaker A

we have feedback loops and it updates its belief. see my video two days ago about theory of mind and the possible scaffolding optimization and the op the updating of the scaffold. So remember scaffold is more or less the optimized

Speaker A

geometry that we are sinking in. This is the the geometry of our gdunking experiment and the harness is just here responsible for the dynamics over that particular geometry representation.

Speaker A

This is the way I see and I differentiate between a scaffold and a harness structure. The harness now becomes a real tangible thing for me.

Speaker A

And the scaffold is the real challenge here for an optimization of my AI system.

Speaker A

As I told you the scaffold, let's go with five papers. We have five different editor regions. No, the representation capacity objective optimization or let's say a structural jump. Why? Because the harness in itself it has to evaluate with which of those five regions of

Speaker A

search spaces to go with. No. So the harness now maintains a success estimate for each possibility for each region in the search space. So it now selects here let's say representation the very first object in our scaffolding s.

Speaker A

But however, if repeated local changes stop reducing our improvements here, this harness the scaffolding might now decide here, okay, I go to a different element in my scaffold to the structural jump. I have to jump out of this region

Speaker A

because this portion of my search space of my solution space is not providing the correct results.

Speaker A

All of this means for me either you can update the IM model and you pay for a cloud beautiful or you already in the group that update here the controller the harness structure with UI model and you find here the

Speaker A

optimum or if you really want to be on a bleeding and leading edge of EI research you think now about updating at first this scaffold to make the problem easier in a new mathematical space that you don't have to pay for the cloud AI

Speaker A

model, but you can do this locally on your laptop. What is really important is this nearby.

Speaker A

We have so much things that go here with a geometric interpretation here of nearby, close by. No. So, let's talk about this. It is that the scaffold not to harness determines what nearby means because it is the geometric definition

Speaker A

of locality. Now the five preprints construct the different forms of locality. Now we had the graph locality connected nodes are nearby quotation mark. Then we had the tool locality.

Speaker A

Compatible tools are nearby. Then we had the skill locality. Semantic similar skills are nearby. Then we had the added locality. Changes affecting the same subsystems are also nearby in our mathematical space. We had the memory locality. facts with reoccurring

Speaker A

attributes belongs somehow together into a cone into a bundle. And then we had the trajectory locality that reaches out out of the local elements here that the particles that the the trajectories here that we counted here in a mathematical

Speaker A

model as particles represent nearby possible futures when we ran the simulation of how will a particle behave the path of a particle in the future.

Speaker A

If you want to see this now allocated to exactly the preprints look at Adam we looked at the attributes that are near if repeatedly co- occur it in our first paper we looked at the facts that are near if connected within the

Speaker A

environmental knowledge graph that we built in the scientific tool agent evolution we looked at which tools are near through the ontology relations the input output comparability and the semantic similarity at Caesar we looked which skills are nearby through the co

Speaker A

and similarity in the embedded mathematical space and at hornness we looked at the modifi which modifications are nearby if they belong to the same local edit arm and jump arms are connected different architectural basins.

Speaker A

Yeah. So now you see why I selected here two days ago this five particular papers because I had a feeling there is something. So and this is now day two of reflecting on these five papers. So the scaffold therefore determines the

Speaker A

topology of the systems searchable world. If you standing there as an astronaut on top of the hill and you look into the foggy landscape.

Speaker A

If you change this scaffold you can change which solution can be reached easily nearby, which experiences appear related nearby. Which failures are treated as equivalent and which action become now really computational jeep that you don't have to pay for a cloudi

Speaker A

system. You see a mathematical optimization does not go to yeah you have to pay for claw code otherwise nothing else comes close. This is not true because if you understand it you understand the principle of of this particular harness panropic you can

Speaker A

optimize for it and maybe you don't even need it at all. So you see a difficult problem can sometimes be solved not by inventing a more complicated algorithm but hold on to your socks by finding a representation in which a simple

Speaker A

mathematical algorithm becomes absolute sufficient to solve the problem. Isn't this beautiful? Why is nobody talking about this? Okay, it needs a little bit of mathematics. But you know what? This opens up complete new ways in AI development.

Speaker A

Okay, since I wanted to do only a 10-minut video today, let's come back to the final insight and let's come to a close for this video.

Speaker A

If you think, hey, what is here a maybe a mathematical representation? Where am I today? What is my understanding here?

Speaker A

Okay, so my understanding here of the common loop that is inherent to all the five prerints we looked at yesterday and also one of this we looked today. I think this can be written in the following simple formula in the body of

Speaker A

of formulas. Yeah, you see it. I don't want to explain it. Maybe you have a feeling about it. If not, come back to me. No problem. What is now the deepest coherence of these five papers? I think all five papers show us here just

Speaker A

published three days ago that the scaffold determines the shape of the searchable world itself. This is the geometry, the mathematical geometry of the abstraction.

Speaker A

The harness determines how compute moves through that particular world. Which tools we use in that particular world, which complexities we have to com we have to combine and add on to concatenate in that particular world.

Speaker A

So the harness can be treated here as we want is in the simplest way as a collection of active operators. No, our fi maps the query into the new scaffold coordinates. Our pi data here selects where to search. This is here if you

Speaker A

want the policy of our LLM here now applied to the hornness itself. Then we have the evaluation itself. The rule that scores this outcome J and U is here our feedback our update incorporates here the feedback we receive from the

Speaker A

system. I think in the simplest mathematical abstraction you can write this as a set of operators here that define here the behavior of the harness h. There we have it. The very last sentence of today.

Speaker A

Complex AI task becomes much easier solvable when the system first constructs a searchable simple geometry, the right mathematical framework and then uses a harness to control how the computation moves through this particular geometry because things can be so much simpler. Just look at

Speaker A

Einstein and general relativity. And why not imply this here also for computer science for AI. It is not that you have to pay up to the most expensive cloud-based AI model. There are other options but therefore you have to

Speaker A

understand what is happening inside this black box and how we can optimize here those methodologies and procedures. I hope you had a little bit of fun. Maybe you'll laugh about my ideas about my insights but I wanted to share this with

Speaker A

you. If you have some comment, please of course publish it here, right down here to this video.

Topics:Local LLMHarness structureMathematical scaffoldingAdam frameworkRetrieval augmented generationAI optimizationCloud AI alternativesGeometric scaffoldingHigh-dimensional dataAgent architecture

Get More with the SozAI App

Transcribe recordings, audio files, and YouTube videos — with AI summaries, speaker detection, and unlimited transcriptions.

App Store Google Play

Or transcribe another YouTube video here →

Free tools: TXT to SRT · SRT Validator · Merge SRT · Subtitle to Text · All tools