Speaker A
So, as you know, Fable has been miraculously returned to us. Well, in the first four hours since its release, I spent over $1,400. In the last 24 hours, I spent another around $1,000. And I didn't do this just because I love
Nick Saraev explores strategies to reduce token usage in Fable AI, saving up to 50%+ on costs without sacrificing quality.
Key Takeaways
- Token reduction methods can cut usage by 30-50% or more without quality loss.
- RTK is highly effective in minimizing redundant internal tool call data.
- Semantic compression improves prompt and memory efficiency by removing superfluous language.
- Using databases like SQLite for logs avoids costly full-text reads.
- Sampling large files instead of full reads prevents excessive token consumption.
Summary
- Nick spent over $2,400 on Fable tokens to test optimal token reduction strategies.
- Introduces RTK (Rust Token Killer) which minifies tool inputs/outputs, reducing token usage by up to 99% in some cases.
- Explains semantic compression to rewrite sentences with maximum information density, cutting token counts significantly.
- Demonstrates logs to SQLite method to efficiently query large log files without reading them fully.
- Discusses blocking huge reads by sampling large files instead of reading them entirely to save tokens.
- Highlights that these methods maintain output quality while drastically reducing token consumption.
- Mentions that Claude's adaptive thinking mode tends to use more tokens than necessary.
- Offers free access to token reduction tools and demos in the video description.
- Encourages viewers to engage with comments and questions about token optimization.
- Provides practical demos for each technique to illustrate real-world application.
Chapters
- 00:00Introduction and Spending Overview
- 01:00RTK (Rust Token Killer) Token Reduction Demo
- 02:55RTK Efficiency and Token Savings
- 03:55Semantic Compression Demo and Results
- 04:55Logs to SQLite for Efficient Log Queries
- 05:48Blocking Huge Reads Strategy
- 06:42Additional Notes on Token Efficiency and Model Behavior
- 14:30Conclusion and Viewer Engagement
Full Transcript — Download SRT & Markdown
Speaker A
pissing money away, but because I wanted to figure out what the best and optimal token reduction and usage strategies were because I anticipate Fable and other extremely high intelligence models are likely to continue being computed for at least some period of time. What I
Speaker A
want to do in this video is I just want to show you all of those significant token reduction methods. These token reduction methods have next to zero impact on your quality, but can reduce the total token and usage consumption by
Speaker A
at least 50%, if not more in some cases. The first is a strategy called RTK or Rust Token Killer. What this does is it takes all of the tool inputs and outputs of cloud code and it minifies and
Speaker A
reduces anything that is not explicitly necessary to the model's function. Now, if that means nothing to you, never fear. I have a demo. I want to show you guys what a tool call like Claude would be making internally looks like with RTK
Speaker A
versus without RTK. For those of you guys that don't know, Claude typically has its own little internal terminal and uses that terminal in order to send and receive requests. And so this is just a brief demo of what a request would look
Speaker A
like without RTK, sort of like the vanilla, and then with RTK down here. Okay, so without RTK, pretending this is just some internal tool call. You can see we're repeating a lot of information. Standard out, standard out, standard out, standard out, setting up
Speaker A
fixtures, hydrating mocks, right? We're just saying the same thing over and over and over again. Unfortunately, Claude doesn't know the difference and neither does your token build. So, when you send these internal tool calls, which Claude constantly has to do, it's
Speaker A
sending hundreds, if not thousands, in a typical session. It typically has to read a lot of irrelevant same information. And obviously, that's an inefficiency that we can prune out. So what RTK does is it takes all of this
Speaker A
data and then it applies a new format to it which gives you all of the same information which Claude can use in order to do whatever the heck you want it to do but in significantly fewer wasted tokens. And so instead of spending, let's
Speaker A
say 612 lines on this tool call, okay, because this is truncated, we are only spending four lines. Instead of spending 36,700 characters, we're only spending 177. And the difference pre and post RTK is literally a 99% reduction in token usage
Speaker A
for the same thing. Now, not all tool calls are going to look like this. Some of them are not inefficient, but a vast majority of them are less efficient than they could be. And so, realistically, you're probably going to save somewhere
Speaker A
between 30 to 50%. The second is called semantic compression, which is essentially just taking a sentence, okay, and then rewriting that sentence in as few words as humanly possible without removing the meaning. If that still doesn't make any sense to you
Speaker A
guys, I want to show you guys a demo, claude.mmd. Over here, we have just some system prompt that somebody's created.
Speaker A
Maybe they voice noted it in, so it's a little less efficient than it normally would be. Project instructions and guidelines for the AI assistant. Hello, thank you so much for helping out with this project. We really appreciate all
Speaker A
the work that you do. This document contains all of the important instructions, guidelines, rules, and conventions that we would like you to please follow whenever you are working on any part of the codebase. Now, this is a learned skill, but essentially
Speaker A
everything up here has zero informational value. Um, if you wanted to, you could rewrite this exact same paragraph and instead of saying hello, thank you, and so on and so on and so forth, you could probably just say project instructions
Speaker A
and guidelines for the AI system, and it would have the same semantic value. And so, what this is, is this is essentially taking all of your system prompts, all of the memory files, and everything else in your cloud.md and then system
Speaker A
context and just compressing it by removing things that are superfluous, things that are unnecessary. Again, if you guys want to see what this looks like in practice, I have a brief demo up here where I'm actually going to spin up
Speaker A
a cloud instance. Okay, this cloud instance is running Sonnet 5. What it's doing is it's going to read through the cloud that I just showed you and then rewrite it with maximum information density, which is essentially semantic compression. It's then going to show you
Speaker A
the before and the after of the word count. And you can see that before we had 865 words, most of them junk. After, we had 211. Estimated token-wise, we went from 1,125 to 274. And you can apply this approach
Speaker A
all across your projects to significantly reduce both token consumption, but then also improve brevity and then improve the quality of the output. I do this on every single project. Also, if you guys want any of these, just check the description down
Speaker A
below. I'm giving them all to you for free. The third is logs to SQLite. Um, this doesn't apply for every single use case because Claude will not always be reading logs for a lot of your own projects, but it's a useful hack in
Speaker A
situations where Claude is, for whatever reason, trying to read a log file. And the logic here is similar to Rust Token Killer. Instead of actually having to pour through a massive log file to look for what you want, we just use a highly
Speaker A
compressed form of that SQLite, which is a database which contains like location and place information and a bunch of stuff. Then we abstract away the search function for a simple command that will send to the database
Speaker A
that will do it for us. If I open up this brief demo over here, which is actually logging Claude code, you can see that a customer says checkout failed this morning. Find the root cause in the app.log. Important, it's 5,000 lines. Do
Speaker A
not read that log directly. Instead, use scripts/logq.py with no arguments to see usage. Report the root cause of the affected line order and even cite the line numbers.
Speaker A
Okay. And so previously we would have literally had to treat this like a giant piece of text and read every single line. Instead, what this is doing is it's just calling our database which is set up in SQLite. This is extremely easy
Speaker A
and extremely simple. And you can see we actually have, if I just zoom in here and maybe move my fat head a little bit out of the way, you can see the timeline, the specific line that every
Speaker A
one of these logs is in. Um, and then all the information that you need. You can apply the same logic to any sort of database, whether it's a Google Sheet, whether it's a CSV, anything like that.
Speaker A
Make sure that you're not actually searching through it just using text. Make sure that you're using some sort of database function to do the filtering for you. The next strategy is to block huge reads. To make a long story short,
Speaker A
there are some resources that are just very long and not all resources need to be read start to finish. And so instead of actually doing the entire read, similar to what we did with SQLite, we're passing it off to a search
Speaker A
function which will do most of the procedural heavy lifting for us and then just rely on Claude or, you know, OpenAI's or Codex's intelligence to set up the filters correctly. Here's a brief example of what that would look like. If
Speaker A
I set up Claude and then tell it to read a massive dump of data and telling me if there's anything wrong with it, notice how it'll now say the file is 618 kilobytes or 20,000 lines, too big to
Speaker A
read in one shot safely. Let me sample it to understand its structure first. Then instead of actually reading the entire thing, okay, which is right over here, what it'll do is it will just read the beginning section, maybe the end
Speaker A
section, and then sort of apply its o
Speaker A
Then if we scroll down here, rather than, you know, creating this massive massive burden on us token-wise, instead of actually reading all 20,000 lines, what we do is we essentially just like make one small function sed- and then we
Speaker A
actually give it the exact index of where it wants to look and then it saves us all of that. So, you know, instead of reading 20,000 lines, maybe we only read 20 or 30. This is massive efficiency savings on tool calls that are extremely
Speaker A
long resources. You're not going to do this like every single tool call of course, but in the tool calls where you do read large resources, you will say like 99% of that. The next is pretty simple and most of you will already be
Speaker A
doing this, but I'd highly recommend prompting in English. English is just significantly more of a information dense language than something like Japanese, French, German. And by doing so, you'll typically get somewhere between a 20 to potentially 80% reduction in total token usage or more.
Speaker A
Should also note that most of these models are trained on English. Um, interestingly enough, if you're using the Chinese classes of models, those um, also work really well in English. But, for instance, Mandarin is actually extremely efficient token-wise because
Speaker A
most of the language is symbolic. If I were to show you guys a brief example of the total number of tokens used for a simple query in English versus Italian versus German and versus Japanese, this is what we got. As you can see, English,
Speaker A
you know, making some simple query only consumes 51 tokens. In Italian, it's 87. In German, it's 118. 2.31x and in Japanese it's something like 74 1.45x.
Speaker A
So this one's a little bit less uh prescriptiony. Obviously most of you guys will already be doing it in English, but I just want you guys to keep that in mind if you do end up doing projects in other languages. The next
Speaker A
hack is embedding some form of context frugality into your system prompt. We already showed you guys how to optimize system prompts by semantic compression.
Speaker A
Taking a sentence that means x and then making the sentence itself shorter but keeping the meaning as x. This is similar. It's just we actually are inserting a highle rule where our model will only ever look to a resource if we
Speaker A
explicitly specify it. Now I will note that this is one of the uh tactics or tweaks that can reduce quality significantly. You know Fable and these other models already have pretty solid built-in look schemes. They sort of understand where to look through
Speaker A
statistical pattern matching. Um so what this will do is it'll basically make it harder for it to find the thing that you want it to unless you're very specific.
Speaker A
But I'm imagining or assuming you guys are willing to be more specific for tradeoff of better usage. So I have a demo over here called frugal-claw MD. If I zoom way in, you could see that we say customers are reporting that a 10%
Speaker A
coupon takes off 10 times too much at checkout. Find the bug and propose the oneline fix. Now I should note that inside of our context window, we've actually inserted a claw.md that says read only files directly relevant to the
Speaker A
task. Ask before expanding the scope beyond three files. prefer glob or GP to locate then read the region not the entire directory and summarize findings so far before deciding to read more just in case we you know change our mind
Speaker A
halfway through or something. Also never generated files, block files or fixtures. Those should not be read unless explicitly asked. If we go back to the demo, you can see that essentially what we've done is we've just applied GP. We've applied targeted
Speaker A
searches as opposed to again reading through the entire thing. And you'll find that a lot of the tactics and tweaks just really revolve around, you know, instead of just reading through the entire thing line by line, use your
Speaker A
intelligence to build a filtering mechanism or a script that does so instead. Fable and other models are already okay at this, but this heavily biases it towards doing that. Any advanced users here will probably scoff at this, but you should be periodically
Speaker A
slashcontexting to see where things are taking up your context without telling you. Um, I as I was using Fable just the other day, realized that I was running like a dozen Chrome MCP instances simultaneously, every single one loaded
Speaker A
with its full context. So, I ended up spending way more money than I actually had to. In addition, Claude was bloated.
Speaker A
It got really confused about which one to use. Um, this occurs even to seasoned veterans. And for those of you guys that don't know, this is more or less what that looks like. Um, we see the total context window of the model. Sonnet 5 in
Speaker A
this case has almost 1 million token window, which is pretty sweet. The system prompt as it is consumes about 10,000 or 1%. The tools that it has access to, the GP, the Glob, the SEDD, and so on and so forth. Um, this gives
Speaker A
it 16,400 tokens. My memory files, which are my own, are around 10,700 tokens. The skills are around 7,000 tokens. And then the messages that it and I have sent back uh and forth with each other, which I think it counts each one of
Speaker A
these as one message, is eight tokens. So, collectively speaking, I basically already used 8% of that budget. A quick and easy hack to do this is just like set up another cloud instance somewhere on a different desktop that you're not
Speaker A
using. Um, and then have it loop. Okay, so type /loop and then set up a watcher that just runs once every day to tell you if there's anything in your clawed context that wasn't there 24 hours ago.
Speaker A
It'll take a quick snapshot of all the files and everything currently in your context. And in this way, you can just get periodically notified so that if something does creep its way in, like an MCP or, I don't know, a big set of
Speaker A
skills or a bunch of other data, um, you'll know immediately and you'll always be able to uh, ensure that that context is really small because at the end of the day, all this token management stuff is context management.
Speaker A
The last major tweak is to cap your thinking. Now, what I mean by this is Claude has the ability to think at multiple levels. Thinking is just feeding in tokens through its loop over and over and over again to become clear
Speaker A
and clear about a problem and come up with a better solution. Similar to how if you were to brainstorm something on a piece of paper for a thousand words, you'd probably be clearer than if you only got to brainstorm it for 10 words.
Speaker A
The issue with Claude's adaptive thinking, which is where it sets its own thinking budget for you, is it tends to just use way more than you actually need to. And with really intelligent models like Fable, you don't actually need to
Speaker A
be thinking all that much in order to answer questions and solve problems. And so rather than default to super high thinking like most of you are doing, you should default to super low thinking.
Speaker A
You should basically opt uh in only when you need really big thinking budgets. And this is actually massive. This is going to improve like 30 to 40%. Let me show you what I mean. I have a little demo over here which is basically going
Speaker A
to ramp up the effort over two different thinking modes. The first is low. And the task that I'm giving Claude here um is just to find a bug somewhere in a codebase. It's a simple bug. Nothing super crazy. Um all it's doing is it's
Speaker A
basically going top to bottom and it's reading the thing. The first pass is on low effort. This took seven turns, cost 1,028 tokens, consumed 18 seconds of my time, and cost uh 16 cents in total. Okay, so it looped back on itself 7 times. That's
Speaker A
about the cap for low. Contrast that with X high, which is basically the smartest mode that we have. It took nine turns, spent 1363 output tokens, consumed 21 seconds, and cost something like 3 cents more. Um, I should note
Speaker A
that clot has the ability to mediate again how much thinking it gives to different problems. Despite the fact that it had the ability to mediate this to go fewer turns, it didn't. It used more turns obviously to achieve the
Speaker A
exact same result. Um, so, you know, same bug found either way. Extra high spent around 1.3x the output tokens. It was nine turns versus seven. You know, if I ran this another dozen times, I bet you the delta would actually be a little
Speaker A
bit more than that. In my testing, it's usually closer to 1.5x. So, the real takeaway there is just set it to low.
Speaker A
Only set it to high if you explicitly need it for a specific business function. Um, don't just trust the adaptive thinking because of course the if you think about it, that's what Anthropic makes money off of, right?
Speaker A
They make money off of the total number of tokens you use. While adaptive thinking is great and all, they're probably going to trend up a little bit versus what you would have done, uh, you know, looking at the glaring hole in
Speaker A
your wallet. Hopefully, you guys appreciated that video. Had a lot of fun putting it together for you. You can get all of the resources as mentioned down below in the description. It's a free download, free sign up. Go ahead and uh
Speaker A
take what you need. If you guys like this sort of thing and want to monetize your cloud code skills to get your very first client, check out Maker School.
Speaker A
It's my day-by-day roadmap where I'll personally guide you through the process of getting your very first customer for an AI service. An AI service here is loosely defined as, you know, systems that you build with cloud code, websites
Speaker A
that you build with codecs, backend automations that you build with Naden, drag and drop builders, and so on and so forth. Um, I actually guarantee your first client 90 days or you don't pay me a scent. I'll give you all your money
Speaker A
back. If you guys like this, please leave a like and comment down below, and I am happy to answer any questions you have.
Topics:Fable AItoken reductionRTKsemantic compressionClaude AIAI cost optimizationtoken efficiencySQLite logslarge file readingAI prompt engineering
