**Stop Prompting Claude. Use Karpathy's Method Instead. — Transcript & Summary | SozAI**
Source: https://sozai.app/transcript/stop-prompting-claude-karpathys-method/

Austin Marchese breaks down Andrej Karpathy's three-layer method for prompting AI (Claude): writing specs, verifying output in agile fashion, and building a durable environment with rules and hooks.

## Key Takeaways

- Define the real goal and make AI interview you to extract context before it builds anything.
- Prefer agile, small-scope, checkpointed work over giving an agent one big waterfall task.
- Specify precise evaluation criteria up front and use a second model as a critic.
- Enforce constraints structurally (rules, hooks) rather than relying on the model to comply.

## What the video covers

- The video claims most people prompt Claude incorrectly, citing Andrej Karpathy's talk at AISN 2026.
- Karpathy's car-wash question (drive or walk 50 m away) illustrates that AI lacks context-driven signal, even when it excels at measurable tasks.
- Layer one is the spec: uncover the real goal (e.g., a decision the report drives) and have Claude interview you to extract it.
- Karpathy argues plan mode is too high-level; the real work is designing a very detailed spec collaboratively with the agent.
- Karpathy's mental model: AI is not an animal driven by intrinsic motivation but something closer to a ghost or robot librarian that doesn't know what it doesn't know.
- Layer two is verification: use agile rather than waterfall workflows, define precise evaluation criteria before work begins, and keep scopes small with clear checkpoints.
- Using a second AI model as a critic (e.g., installing the Codex plugin inside Claude Code) is suggested for verifying outputs.
- Layer three builds a durable environment: give AI working rules and context, and use pre-tool-use hooks to enforce constraints such as blocking edits to certain files.
- Precision reduces assumptions, and every assumption is a chance for the output to drift from what you want.
- The verification lever is presented as the most underused lever for improving AI results.

## Chapters

1. 00:00 Introduction: Karpathy's claim that people prompt Claude wrong
2. 00:58 Why AI lacks context: the car wash example
3. 02:47 Layer one: writing the spec and uncovering the goal
4. 04:47 Karpathy on plan mode versus detailed specs
5. 06:40 Animals versus ghosts: the mental model
6. 08:34 Layer two: agile work and verification
7. 10:47 Layer three: building a durable environment with hooks
8. 12:40 Conclusion: the verification lever

Answers

## Questions about this video

Why does the presenter say AI gets the car wash question wrong?

Because AI models optimize for measurable signals and lack the contextual information that the goal is to wash the car, which requires driving. Karpathy uses this to show that unstated context is invisible to the model.

What is the difference between waterfall and agile prompting in this video?

Waterfall gives the AI one big task and shows only the final product, while agile breaks work into small buckets with checkpoints, reviewing and adjusting output throughout the process.

What does the presenter suggest for the third layer of the method?

Build a durable environment with clear working rules and context, and use pre-tool-use hooks to structurally block actions, such as edits to protected files, rather than relying on the model to follow instructions.

## Full Transcript — Download SRT & Markdown

00:00

Speaker A

I just listened to Andrej Karpathy speak at AISN 2026, and I learned something that I wasn't expecting. Almost everyone is prompting Claude wrong.

00:12

Speaker A

Tesla, uses AI in 2026. And it turns out that Karpathy's method for building 10 times faster can be broken down into three simple layers. So, in today's video, I'll be breaking down each layer so that anybody can apply them. And then

00:24

Speaker A

So, I decided to dig deeper and see exactly how Karpathy, the former head of AI at Tesla, uses AI in 2026. And it turns out that Karpathy's method for building 10 times faster can be broken down into three simple layers.

00:36

Speaker A

simple question AI will get wrong. I want to go to a car wash to wash my car, and it's 50 m away. Should I drive or should I walk? And state-of-the-art models today will tell you to walk because it's so close.

00:48

Speaker A

So, in today's video, I'll be breaking down each layer so that anybody can apply them. And then I'll show you the one thing that Karpathy said to focus on in the age of AI.

00:58

Speaker A

brilliant at what can be measured, but for context-driven things like needing a car for a car wash, it has no signal to act on. So, how do you bridge this gap between your understanding and your contextual information and AI's

01:10

Speaker A

So, layer one is the spec. AI models are incredibly smart, but they're still missing something.

01:21

Speaker A

create a plan before building anything. But Karpathy thinks that this is too high-level. I actually don't even like the plan mode. I I would I mean, obviously it's very useful, but I think there's something more general here where you have to work with your

01:32

Speaker A

To showcase their current limitation, Karpathy explained a simple question AI will get wrong. "I want to go to a car wash to wash my car, and it's 50 m away. Should I drive or should I walk?"

01:45

Speaker A

Claude can successfully use to build what you're trying to build? The first step is you have to uncover your goal.

01:50

Speaker A

And state-of-the-art models today will tell you to walk because it's so close.

01:59

Speaker A

And what the goal actually is is something AI will literally never be able to decide. So, to help you do this, we'll tell Claude to interview me to identify the goal of this project. This is the way to get the information out of

02:11

Speaker A

At first, I actually didn't believe this, so I went to Claude, Gemini, Grok, and ChatGPT, asked them the same question, and they all gave me the same answer.

02:24

Speaker A

then you show the final product. Agile on the other hand is you break that same task into small buckets, and you show the result throughout the entire process to make sure you're going in the right direction. And people are extremely

02:35

Speaker A

And it reveals the whole foundation of this video. AI is brilliant at what can be measured, but for context-driven things like needing a car for a car wash, it has no signal to act on.

02:46

Speaker A

output, adjust it, and then repeat. To help with this, we'll tell Claude to bias towards smaller and more compartmentalized specs. Step three is you want to be precise and use your brain. The more precise you are, the less AI has to assume. And every

03:00

Speaker A

So, how do you bridge this gap between your understanding and your contextual information and AI's computational power? That's where the spec comes in.

03:12

Speaker A

help you use your brain, you can say "Make me verify key decisions explicitly to ensure nothing is missed." And when you put these three pieces together, we have a final prompt we can use in Claude to help create a tightly scoped,

03:24

Speaker A

A spec is how you deliver your understanding to Claude in a format it can use. A term you may have heard is Claude's plan mode, which essentially can be used to help you create a plan before building anything.

03:35

Speaker A

Layer two sits on top of the spec. This is the verification process. One of the most frustrating things about AI is reviewing and verifying the output. And unlike a human, it can't grasp non-measurable things. So, how can we

03:47

Speaker A

But Karpathy thinks that this is too high-level. "I actually don't even like the plan mode. I, I would, I mean, obviously it's very useful, but I think there's something more general here where you have to work with your agent to design a spec that is very detailed."

03:59

Speaker A

confusing, don't worry, I will simplify it after. And the idea is that we're not building animals, we are summoning ghosts. Why does that framing matter? And what does it actually change about how you build and deploy and evaluate or even trust

04:12

Speaker A

Now, Karpathy isn't telling you that plan mode is bad. What he's actually saying is you have to go deeper, work with these AI tools to design the actual spec.

04:20

Speaker A

going to be more competent at uh using them. I think it's just um coming to terms with the fact that these things are not, you know, animal intelligences. Like if you yell at them, they're not going to work better or

04:29

Speaker A

So, how do you create a spec that Claude can successfully use to build what you're trying to build? The first step is you have to uncover your goal.

04:40

Speaker A

Now, that's some gigabrain stuff, but let me simplify it. People, me and you, are used to interacting with people, which Karpathy is calling animals. These animals are driven by different motivators and emotions, which help produce the final product and output

04:53

Speaker A

If you just say, "Create an end-of-month report," that's a task, but the actual goal is a conclusion you're trying to draw, the decision the report drives.

05:01

Speaker A

That's because they have these intrinsic motivations. But AI is not that. Karpathy describes it as a ghost, but in my eyes that's a little too confusing, so throw it out the window. Instead, think of it like a robot librarian. If

05:13

Speaker A

And what the goal actually is, is something AI will literally never be able to decide. So, to help you do this, we'll tell Claude to interview me to identify the goal of this project. This is the way to get the information out of you and into the spec.

05:24

Speaker A

doesn't know when it's missing a specific book. So, it may just confidently make something up. And that's what's happening when AI nails math and fumbles things with context.

05:34

Speaker A

Now, step two is be agile with how you work. There are two methods of completing any task. The first is waterfall, and the other is agile.

05:48

Speaker A

better." doesn't necessarily work. Really, the only lever you have, which most people don't even think to use, is the verification lever. Because by optimizing this, it makes it so that you're playing within the actual rules that the AI follows. So, how do you help

06:01

Speaker A

Waterfall is you take a big task and you complete the entire thing, and then you show the final product. Agile, on the other hand, is you break that same task into small buckets, and you show the result throughout the entire process to make sure you're going in the right direction.

06:09

Speaker A

Before Claude touches a single thing, whether that's technical or non-technical tasks, define what good looks like with precision. For example, a vague way to evaluate an output is, "Make this report look good." Whereas, a precise way would say, "The report must

06:22

Speaker A

And people are extremely susceptible to using AI agents in a waterfall manner because they want to give them everything to do at once. The better move is agile specking. You want to have a tight scope, a clear checkpoint, you want to review the output, adjust it, and then repeat.

06:33

Speaker A

help enforce this, we'll add this to our verification Claude prompt. Outline the evaluation criteria you will use to ensure a high-quality final product. Be precise. The second step is use a second AI model as the critic. Think of this

06:46

Speaker A

To help with this, we'll tell Claude to bias towards smaller and more compartmentalized specs.

06:58

Speaker A

first librarian is right or wrong. Now, a tactical way to do this, if you use Claude code, you can install the Codex plugin, which will allow you to directly ask Codex questions within your Claude code session. So, you could say

07:09

Speaker A

Step three is you want to be precise and use your brain. The more precise you are, the less AI has to assume. And every assumption that AI makes is a chance for it to drift from the final product you actually want.

07:22

Speaker A

will help you verify an output? Here are two concrete examples. Let's say you're deploying an app and you're not sure if it's successfully deployed. What you can do instead is connect your Claude session with your system where it's

07:32

Speaker A

And when you have AI create a spec for you, you have to use your brain to think critically about what that spec actually says. So, to help you use your brain, you can say, "Make me verify key decisions explicitly to ensure nothing is missed."

07:40

Speaker A

And now, if it says that the deployment was successful, we know for certainty that it actually was. In a non-technical example, let's say you're working on a monthly report. You could bring in your historical reports to use as reference

07:51

Speaker A

And when you put these three pieces together, we have a final prompt we can use in Claude to help create a tightly scoped, well-thought-out spec that aligns with our actual goal.

07:57

Speaker A

Now, bringing in this concept with the first two points, combining this third point with the first two points, here is a prompt that you can run in Claude, which will help ensure that you are adding a proper evaluation layer where

08:06

Speaker A

This is a process that I call modern engineering, which every successful person has to become.

08:15

Speaker A

So, layer one and layer two are about creating specs and evaluating the output. The third layer, however, is where we build a foundation that can't be replicated. But, before we get to that, if this is your first video of

08:25

Speaker A

Now, layer two is the verifier. Layer two sits on top of the spec. This is the verification process.

08:37

Speaker A

that I ask is that you subscribe as part of this agreement because it helps it reach more people so that I can keep making videos like this. Also, every couple of weeks, I give away a Claude Max subscription, so comment below with

08:47

Speaker A

One of the most frustrating things about AI is reviewing and verifying the output. And unlike a human, it can't grasp non-measurable things. So, how can we help AI verify its own outputs?

08:57

Speaker A

blueprint pinned to the wall, the verifier is the quality check station by the door, and then the environment is the workshop itself. You need to create the proper tooling and the proper system so that the whole thing can function at

09:08

Speaker A

Well, first you need to understand the mental model behind this. And Karpathy explains it as animals versus ghosts. Here's him getting asked a question about this in a recent interview. And if it sounds confusing, don't worry, I will simplify it after.

09:18

Speaker A

create a proper workspace that improves over time? First is you need to set up a proper Claude MD file. Every time you prompt Claude, your Claude.md file gets injected automatically. It's essentially the first thing that Claude reads to

09:30

Speaker A

And the idea is that we're not building animals, we are summoning ghosts. Why does that framing matter? And what does it actually change about how you build, deploy, evaluate, or even trust them?

09:43

Speaker A

that you can improve this Claude MD, and here's actually mine on the screen, and I'm going to call out a couple of sections. The first is I outline how this repo works. So, think of my repo as my workspace. It gives high-level to the

09:54

Speaker A

"Yeah, I think the reason I wrote about this is because I'm trying to wrap my head around what these things are, right? Because if you have a good model of what they are or are not, then you're going to be more competent at using them. I think it's just, um, coming to terms with the fact that these things are not, you know, animal intelligences. Like, if you yell at them, they're not going to work better or worse, or doesn't have any impact. Um, and, uh, it's all just kind of like these statistical simulation circuits. It's more just being suspicious of it and, um, figuring it out over time."

10:05

Speaker A

information. And then I have key working rules that it should follow no matter what. Make this your environment. It's your world, and AI is living in it. It should not feel like the other way around. The second step is you need to

10:15

Speaker A

Now, that's some gigabrain stuff, but let me simplify it. People, me and you, are used to interacting with people, which Karpathy is calling animals. These animals are driven by different motivators and emotions, which help produce the final product and output within a team setting.

10:21

Speaker A

And this is essentially creating a folder system on your machine that you're able to ingest your own training data in a way that makes it really easy for Claude to understand where information is. This is so important because your data is your moat. And this

10:35

Speaker A

And if you say to a person, "Become an expert at SEO marketing in the next 14 days or you're fired," they're going to figure it out. That's because they have these intrinsic motivations. But AI is not that.

10:48

Speaker A

skill for that. Think of this like a handbook to complete a specific task. And the more you use these skills, the better they'll become. I have a saying that I tell my team, the best way to find a leak in a hose is to run water

10:58

Speaker A

Karpathy describes it as a ghost, but in my eyes that's a little too confusing, so throw it out the window. Instead, think of it like a robot librarian.

11:07

Speaker A

Step four is create rules for what the AI can and can't work on. Depending on the cost of getting something wrong, you need to establish different AI guardrails. So, here's how to think of this, right? So, take the Claude.md file

11:18

Speaker A

If you ask it that same SEO question, the librarian will only suggest resources and answers based on the books in its library. If it doesn't have a book, it can't help you.

11:29

Speaker A

to get wrong, then you need to introduce rule-based guardrails to ensure that the AI can't bypass them. To help you visualize this, imagine you have a folder called "Important, Don't Edit." You could have a rule in Claude MD that

11:41

Speaker A

And part of the challenge here is that the librarian doesn't know when it's missing a specific book. So, it may just confidently make something up. And that's what's happening when AI nails math and fumbles things with context.

11:51

Speaker A

Claude can still touch those files. So, instead, you add a pre-tool use hook before Claude uses the write or edit tool, and it checks to see the file that it's trying to edit. Now, Claude literally can't make the edit, and it's

12:04

Speaker A

It's brilliant because the library has the clear answers. But if it doesn't, then it's confidently wrong or uncertain.

12:15

Speaker A

The first is always do. This is things that AI should run on autopilot. The second is ask first. So, this is anything that you want to double-check.

12:23

Speaker A

Which means interacting with it like it's an animal, i.e., a human, doesn't help, right? Yelling at it, pleading, just saying, "Make this better," doesn't necessarily work.

12:29

Speaker A

Here's a prompt that brings all of these four points that I mentioned to help audit your system and create an optimized environment for Claude to interact with. That's the Karpathy method end-to-end, the spec, the verifier, and the environment. But

12:40

Speaker A

Really, the only lever you have, which most people don't even think to use, is the verification lever. Because by optimizing this, it makes it so that you're playing within the actual rules that the AI follows.

12:48

Speaker A

What still remains worth learning deeply when intelligence gets cheap as we move into the next eight era of AI?

12:56

Speaker A

So, how do you help AI verify the output so it's up to the standard you want? Well, there are three places to focus on.

13:07

Speaker A

to direct AI to start working for you. Now, if you like this video, you will love this one where I do a deep dive into four Claude projects that you need to build today using these three layers.

13:17

Speaker A

First, you want to set the evaluation criteria up front. Before Claude touches a single thing, whether that's technical or non-technical tasks, define what good looks like with precision.

Topics: Claude Andrej Karpathy prompt engineering AI spec agile AI workflow AI verification Claude Code pre-tool hooks AI agents context engineering

Study this video

- [Flashcards from this video](https://sozai.app/tools/ai-flashcard-generator/?from=transcript&id=81950)
- [Quiz on this video](https://sozai.app/tools/quiz-generator/?from=transcript&id=81950)

Free, made by AI from this transcript. No signup.


---
This is the markdown twin of https://sozai.app/transcript/stop-prompting-claude-karpathys-method/ — the same content, without the markup.
Published by SozAI (https://sozai.app). Reuse and quotation are allowed with attribution and a link back.
Machine-readable index: https://sozai.app/llms.txt · data API: https://sozai.app/api/