Speaker A
Today, I'm going to be showing you guys two different ways that we can run Claude Code for free.
Nate Herk shows two free ways to run Claude Code: a local open-source model via Ollama, or cloud-hosted free models via OpenRouter, explaining the harness-vs-engine analogy along the way.
Key Takeaways
- Claude Code can run for free by swapping the paid Anthropic model for free local (Ollama) or cloud (OpenRouter) open-source models.
- Open-source models are closing the performance gap with closed models but may need configuration to handle Claude Code's tool calling and context requirements.
- Closed frontier models like Opus 4.6 remain the best choice for high-stakes heavy work, while free models suit lighter tasks and experimentation.
- Managing context length and token usage is essential when working with open models that have smaller context windows.
What the video covers
- Claude Code is a harness (the 'car') wrapped around a model (the 'engine'); by default it uses Anthropic's Opus, Sonnet, or Haiku models, which are paid via API tokens.
- The video explains that you can swap the engine by pointing Claude Code at other models, either free cloud-hosted models on OpenRouter or self-hosted local models.
- Open-source models have their weights publicly available for download and modification, while closed-source models like Claude, GPT, and Gemini are only accessible through paid APIs.
- A benchmark chart (SWE-bench Verified) shows the performance gap between closed and open models is shrinking, with some open-weight models outperforming Claude Sonnet 3.7.
- Open-source models may misbehave in Claude Code because they weren't trained on its tools, have smaller context windows than its system prompt needs, or don't follow its JSON tool-calling protocol.
- Google's Gemma 4 is highlighted as offering the highest Elo score for its small size, making it practical to run on consumer hardware.
- Method one: install Ollama, download a model such as a Qwen variant from the Models page, and connect it to Claude Code through environment variables.
- Method two: create an OpenRouter API key and configure Claude Code to route requests to free OpenRouter models, including editing settings.local.json to set model names.
- Model selection can be guided by OpenRouter's rankings, and pricing per input and output token is shown on each model's page.
- Using open-source models in Claude Code is said not to violate Anthropic's terms of service, since Anthropic's agent harness is still used.
Chapters
- 00:00Introduction: Two Free Ways to Run Claude Code
- 01:09What Claude Code Is: Harness vs. Engine
- 03:33Open-Source vs. Closed-Source Models
- 06:57Performance Gap and Why Open Models Misbehave
- 08:08Gemma 4 and Choosing Model Size
- 09:15Terms of Service and Method 1: Setting Up Ollama
- 14:08Method 2: Configuring OpenRouter with Claude Code
- 19:00Choosing Models and Managing Context
Full Transcript — Download SRT & Markdown
Speaker A
The first way is going to be by running the LLM locally on our own machine, and then the second way is using OpenRouter.
Speaker A
Both of these ways are extremely easy, a lot easier than you would think, and I'm going to walk you through step-by-step both of them.
Speaker A
And my goal is to cover everything that you need to know in order to run this and answer any questions that you may have.
Speaker A
So, let's not waste any time and just get straight into the video.
Speaker A
All right, so before we get into the tutorials of the two methods, there is some important stuff that we need to cover.
Speaker A
The first one is about what actually is Claude Code.
Speaker A
And when I say, "How do you use Claude Code for free?" what does that mean?
Speaker A
So, this is the analogy that any AI model would pretty much give you, and it is the fact that Claude Code is the car and the chat model, the AI model, is the engine.
Speaker A
So, when we use Claude Code natively, we are using basically a harness that is wrapped around Opus or Sonnet or one of Claude's models that's really, really smart.
Speaker A
So, Claude Code tells the model how to organize its folders and how to use tools and how to, you know, execute plans and build projects for you and stuff like that.
Speaker A
And by default, we're using Opus, Sonnet, and Haiku.
Speaker A
So, when you're thinking about your context limit or when you're hitting your token limit, things like that, that is because you're paying Anthropic for access to the Claude Opus model or the Sonnet model or the Haiku model.
Speaker A
Now, what I'm going to show you guys how to do is we basically just open up the hood and we switch out the engine.
Speaker A
So, we can use the OpenRouter models in their cloud, which are free, or we can use self-hosted local models, which live on our desktops, on our laptops, whatever it is, and those are also going to be free.
Speaker A
And that means you don't run into any limits, or at least the way that the limits work are much different.
Speaker A
So, now that we understand that, the question is, okay, why can't you just run Opus locally for free?
Speaker A
Well, that is because there's a difference between open-source models and closed-source models.
Speaker A
An open-source model basically means that the model is published somewhere so that anyone can download it.
Speaker A
It is open. It's open-source.
Speaker A
We can see inside of it. We can edit it. We can run it. We can modify it.
Speaker A
And a closed-source model is closed.
Speaker A
So, Sonnet and GPT and Gemini models, those are kind of the top-tier leading closed-source models, and the only way that we can access them is through the company's API, which means in order to hit their API, we have to pay them for tokens.
Speaker A
So, if you think back to the car analogy, when we have an open-source model, that means, you know, the car lets us open the hood and we can actually look at the engine and change things.
Speaker A
But with a closed-source model, the hood is locked shut and we can't get in there and we can't do anything to the engine.
Speaker A
So, the question is, why are the majority of the people in the world using closed-source models?
Speaker A
That's because they're typically just better.
Speaker A
So, here's the chart.
Speaker A
You can see in blue we have closed-source models.
Speaker A
Up here we have Opus 4.6 and Sonnet 4.6.
Speaker A
And then in the green we have open-source models.
Speaker A
So, this green one right here is a Qwen model.
Speaker A
So, what's really cool is there's always been a gap between the performance of closed-source models and the performance of open-source models, but that gap is just shrinking and shrinking.
Speaker A
And I'm sure we're going to get to the point where they are just as competitive as closed-source models.
Speaker A
And by the way, this chart is based on SWE-bench Verified scores, which is a, you know, kind of computer engineering programming sort of benchmark for AI models.
Speaker A
Now, what's really cool is if we look at this, but we also overlay Sonnet 3.7 on top of this, we can see that Sonnet 3.7 in the orange here is not even in the top five, which means some of the open-weight models that we can access today are better than Claude Sonnet 3.7.
Speaker A
And if you remember, when that model dropped, everyone was freaking out.
Speaker A
It was the best model at one point.
Speaker A
So, just to put into perspective that the open-source models are really good, but still right now, if you want to do like heavy, heavy work and you've got something that you can't really mess up on, then Opus 4.6 is probably going to be the best or maybe GPT 5.4 with Codex.
Speaker A
And because of that, there's a few reasons why open-source models in Claude Code might seem to misbehave.
Speaker A
They might have not been trained on Claude Code's tools.
Speaker A
They might have a context window that's too small for Claude's system prompt.
Speaker A
And they might not follow the exact same JSON protocol that Claude Code expects.
Speaker A
So, once again, the car and the engine analogy, it might be like trying to put a motorcycle engine into a truck or maybe putting the wrong type of gas into your vehicle.
Speaker A
So, sometimes you do have to play around with the models to see if they can do the proper tool calling that Claude Code or your car actually expects.
Speaker A
And you know what inspired me to make this video?
Speaker A
It was the release of Google's new open-source model, Gemma 4.
Speaker A
And what's really cool to see is this is a comparison of some other open-source models that are pretty good.
Speaker A
On the x-axis, we have the size of these open-source models, and on the y-axis, we have their Elo score.
Speaker A
And what you can see is that these Gemma 4 models have the highest Elo score and the smallest size.
Speaker A
And in this context, a small size is better because that means that we can actually host it on our machine and our machine can handle it, because it's basically like, how many gigabytes is the model?
Speaker A
How many parameters does it have?
Speaker A
You need more hardware in order to run larger models.
Speaker A
All right, so now you should have a good understanding of open-source models, closed-source models, and how this is going to work.
Speaker A
The final thing is the question that people might be asking: is this against Anthropic's terms of service?
Speaker A
And the answer to that is no.
Speaker A
We're using their agent harness.
Speaker A
We're just plugging in a different local model or open-source model.
Speaker A
So, with all that out of the way, let's get into our first method, which is going to be running a local model.
Speaker A
We're going to be using Ollama, and then we're going to hook that up into Claude Code and see how it works.
Speaker A
All right, so step one here is to go to ollama.com.
Speaker A
Now, once you get here, what you're going to do is you're going to download Ollama, and you're going to do it for your operating system.
Speaker A
So, here on Windows, I would download for Windows.
Speaker A
Once everything is downloaded, we're going to have to pull in a model or download a model.
Speaker A
So, you're going to click on Models up in the top left, and now we just have to choose which model do we actually want to pull in and download.
Speaker A
So, if we go over to OpenRouter, and we're not jumping to step two yet or to method two yet, OpenRouter just has this really cool way where we can actually look at rankings between different AI models.
Speaker A
This page can be very, very slow.
Speaker A
There's a lot of data here, so just keep that in mind.
Speaker A
But you can see that for the programming category, there's a lot of models here that are open-source.
Speaker A
So, right here we can see there's a Qwen 3.6 that's pretty good.
Speaker A
There's a Mimo.
Speaker A
There's a MiniMax.
Speaker A
So, let's hop over to Ollama and see what we can do.
Speaker A
So, let's go ahead and choose this Qwen 3.5 model.
Speaker A
If I open this up, there's a few things that we can see, and we can see all of these different models.
Speaker A
So, these are different sizes.
Speaker A
We can also scroll down and we get some highlights.
Speaker A
So, we can see some benchmarks that they have, and these benchmarks will compare them to both other open-source, but also some other closed-source models, which is nice to be able to look at.
Speaker A
But what we're concerned about right now is how we actually download this.
Speaker A
So, you'll see that we have different model names, and we have differe
Speaker A
model if it's okay, and it said, "I don't have feelings, but my loss function is definitely in the red." So, you can see how us giving Qwen a little bit more context was helpful. Now, what if we wanted to do a
Speaker A
model from Ollama that was in the cloud? So, for example, let's say we wanted to use Min Max M 2.7. What we would do is we would just launch this command. So, we don't have to run this or download
Speaker A
it. We would just copy this command, and go back into my VS Code, and I would paste this in at the bottom down here.
Speaker A
And then when I run this, it's prompting me to sign in at ollama.com. And now it's saying that we need to connect our device here. And now that that has been connected, I can close this window. So, if I open back up over here, we can now
Speaker A
see that we have our Claude Code running, and we have Min Max M 2.7 on the cloud. But, because of the fact that you are not running this locally anymore, you do have a pricing barrier at some point. So, when you're just
Speaker A
getting started, you can definitely play around with those for a while, and you will be able to use them, but you do need to upgrade at some point if you need to run more than one model at a
Speaker A
time, and if you need more usage, and then same thing with the Max. So, that's the thing that you have to realize is that there's like there's really no such thing as free because if you want to run
Speaker A
a really good model locally, then you need the hardware to support it. If you want to run it on the cloud, you either have to pay for a subscription like Ollama, or you could pay for a VPS that's big enough in order to host that
Speaker A
model there. So, that's the trade-off. Um there's really no such thing as free. You got to find the balance between quality and price, and also thinking about like where are you making your investment, and are you going to max out
Speaker A
your usage, or are you going to pay per token, all these kind of things. How can you optimize your processes and also your price at the same time. And we can now use this model, which is probably going to be much faster and better than
Speaker A
this previous one up here with this local version of Qwen 3.5, which was only 9 billion parameters. So, I'm asking it that same question where I'm testing to see how quick and how good it is at reading through everything in
Speaker A
here. And you can see that came back way, way quicker. So, now I'm going to ask it to do something else. I'm going to see if it can run my morning coffee skill. And look at this, we can actually
Speaker A
see that it is indeed running the skill, and we get a little bit more visibility because I'm assuming that this Min Max model is much, much bigger than 9 billion parameters. So, the point I'm trying to make here is if you don't have
Speaker A
hardware that can support larger models, then you probably want to find a way to run larger models somewhere else because this is just such a better experience.
Speaker A
This almost feels like I'm actually using Sonnet in Claude Code right here. I mean, this literally spun up four different local agents, and it's actually running pretty fast. The only thing to think about here is this is no
Speaker A
longer 100% local because it's now going to Ollama's cloud, which is why you can understand now why it's such a big deal that Google Gemma 4 is so much smaller than all of the models that are its same
Speaker A
quality. All right, so before we get on to the method number two, which is using Open Router, when would you actually want to use these open source models?
Speaker A
There's a few different scenarios that I want to talk through. The first thing is if you have low stakes or high volume tasks. So, things like reading and summarizing files before passing to a smarter model, or grepping through a
Speaker A
code base to find relevant files and functions, so doing some searches, or generating repetitive code scaffolding.
Speaker A
The things where you really don't need the most top-tier model to be doing. We also have things like research and information gathering. So, web searches, summarizing emails, pulling docs, things like that. Organizing and classifying things, categorizing tasks, triaging
Speaker A
emails, organizing local files, or even your CRMs. And then if you want to go on like the actual coding side, it can definitely be some simple tests, or maybe improving some bugs or doing some reviews. But, I personally would just be
Speaker A
careful, and then always make sure that if you don't really know what's going on, you would be checking later with a more powerful model. Now, of course, the other times when you might want to do this is if Claude is down. So, if you go
Speaker A
to status.claude.com, and you're having issues with Opus or Sonnet, then maybe you switch to a local model for a little bit just so you don't feel unproductive for the next 2 hours. Or, of course, if you just hit your session limit, and you
Speaker A
have to wait another few hours, then maybe you open up your local version and just play around with that. Okay, so that was the local method. Now, let's talk about running it through Open Router. So, on Open Router's docs, they
Speaker A
even have an integration guide for basically how this works. And all we have to do is we just have to change a few things in our environment variables, which override the default permissions.
Speaker A
So, we're basically just telling Claude Code, "Hey, instead of pointing to Anthropic's API, you're going to point to Open Router, and I'm going to give you this key, and then you're just going to basically talk to these models
Speaker A
instead of Anthropic's models." So, if you don't have an Open Router account, go ahead and go to openrouter.ai and sign up. And what you're going to have to do is, in order to use these models, you're going to have to load your
Speaker A
account with like $5 or $10. And then after you do that, you're free to use all of these free models, and it won't actually eat into that 10 bucks that you put into Open Router. And basically, the reason why is because of rate limits.
Speaker A
So, if you don't have anything on your account, then you will get limited to 50 requests per day with your free models.
Speaker A
But, if you put $10 in, then you will get 1,000 requests per day. And by the way, this whole doc with all the setup information will be attached in my free school community. It's completely free.
Speaker A
The link for that is down in the description. Claude Code, and we basically launch up Claude. We will see that it says, "Okay, cool, we're using Opus, and we're using my Claude Max plan." So, we need to change this. And
Speaker A
the way we do that is in our project settings. So, in the dot Claude, in the settings.local.json, you can see that this is what I have.
Speaker A
I've got some permissions and stuff like that. I've got an environment variable, but we need to add a few more environment variables. All right, so I switched into a different project just so I could keep this clean, but I have
Speaker A
this other JSON file right here called free router. So, what I'm going to do is I'm just going to take all of this and copy it, and then I'm going to open up Claude, and I'm just going to say,
Speaker A
"Hey, can you please add this configuration to our settings.local.json?" And I would go ahead and paste that right in here. Now, in my case, there my API key is in there, so just be aware that if you do this,
Speaker A
Anthropic is reading that. But, the reason I'm doing this is because right after this video, I'm just going to delete this anyways. So, now let's talk about what this is doing. So, the first thing is it's pointing to Open Router
Speaker A
instead of Anthropic. And then in the Anthropic off token, this is where you put your Open Router API key. So, I know it says Anthropic, but this is where you put your Open Router one. So, you'd go into your Open Router account, you would
Speaker A
go over here to credits, and then you're going to go into your API key section, and you're going to create a new key. So, give it a name, hit create, and then when it gives you that value, you're going to copy that,
Speaker A
and then you're going to put that into the configuration that I just showed you right here, Anthropic off token. And then the Anthropic API key, you're going to leave completely blank. The Anthropic model and all of these other models,
Speaker A
you're going to put in a free model from OpenRouter. Now, the reason you have to do all of them and not just this one, like the OpenRouter documentation suggests, is because if you do it like that, it's actually going to, by default, call on
Speaker A
Anthropic paid models for Haiku and Sonnet. So, here in my logs, you can see I was testing this out and I did the free configuration, but then I noticed in my logs I was kept getting charged by by Haiku, and I was very confused.
Speaker A
Because whenever it was doing little tool calls or doing little file searches, it was using Haiku instead of using the free models that I had installed it to use. So, if you were to have just put in this configuration, and
Speaker A
you would have left out these variables down here, then it would have, by default, used Sonnet or Haiku for basically all of these things, and it would have charged you without you even knowing. So, make sure you put these
Speaker A
ones in there as well. And then you can see, after I figured that out, it was able to use all of these for tool calls and everything instead of using the paid models. The only reason these were paid
Speaker A
is because I did that intentionally. But yeah, this is what OpenRouter says, so just do it the way that I did it. So, now that we have this set up, if I actually open up a terminal, and I type
Speaker A
in Claude, what do we see here? We see that we are using OpenRouter free API billing usage rather than using our Max plan or our Team plan. So, what is OpenRouter free? Okay, well, this is very similar to Ollama. You've got all
Speaker A
these different models you can look at. When you click on a model, it'll show you the price per input tokens and the price per output tokens. But what you can do is if you come up here and you
Speaker A
just search free, you can see that there's a ton of models right here available that are free. And if you go down to the actual OpenRouter one, which is right here, the free models router, this one is completely free, but it
Speaker A
basically routes you to whichever model is like the most available at that moment. So, it kind of helps you from getting rate limited and things like that. Now, I'll show you how this works, but I'm also going to show you why I
Speaker A
wouldn't recommend using this because you just don't have any control over what model is getting called. So, here I'm asking it to make a file in this folder called OpenRouter test. You can see that we are able to get the
Speaker A
visibility as far as what tools it's using, which is great. We have over here the OpenRouter test file, and there's nothing in it, but it did create that, and it didn't take that long, either.
Speaker A
And now in my logs, you can see this is what we just did, April 3rd, 5:15. It called the OpenRouter free model three different times, and all of these times costed us zero dollars, as you can see.
Speaker A
Now, let's say instead I want to use this awesome Qwen 3.6 model with a million context and completely free, once again. Well, what I would do is I would copy this model name, so Qwen Qwen 3.6 plus colon free. Back in Claude
Speaker A
Code, I would just close out of this terminal real quick. I'm going to open up the settings.local.json, and I will basically just come in here, and I will paste that new model name into all of the instances where we said
Speaker A
OpenRouter free. And now, after I save this and I go ahead and open up a new terminal and launch Claude, we should see that we have in here this is running now with that new Qwen model, and I can say
Speaker A
hello, and it should be able to respond to us fairly quickly as Qwen. There we go. Hey, what can I help you with today? If I go back into my logs and I refresh, we should see that this
Speaker A
most recent call will be with Qwen. There we go. Those two came through as Qwen, and you can see that once again, it costed us nothing. So, let's see what else we can do with our open source models. Because if you remember, I
Speaker A
talked about some things where Anthropic has trained its Claude models to act a certain way with Claude Code, but not the all the other models have done that. So, can you go ahead and search the web and do some research on Google's new model
Speaker A
Gemma 4. So, what you'll notice when I asked it to search the web, it's doing web searches, and it's knowing how to use those tools, but it's running into an issue where the current model seems to not have web search access. So, it's
Speaker A
trying a different approach where it's basically just doing a fetch directly. So, what happened here is it tried to do fetches, and it looks like that worked, but I guess maybe it didn't because it said that it couldn't do any web
Speaker A
lookups, but it does have some information about the um actual model based on training data. So, what that means is you can still do web search with tools like, you know, Brave Search or Perplexity or Tavily, but it's a little
Speaker A
bit harder because it now might not natively have the web search ability. If you need to like look up API documentation or things like that, you just might have to give it more information manually or let it use more
Speaker A
Perplexity credits or something like that. So, just keep in mind, you want to be monitoring this kind of stuff because you do have certain limits on your account as far as how many calls you can make to free models per day, as well as
Speaker A
how many calls you can make to free models every single minute. So, something else you can keep in mind, though, is the ones where you won't be limited as much are models that are really, really cheap.
Speaker A
It's not going to work right now in Claude Code, but if you were using Gemma 4, 31 billion parameters, this is 14 cents per million input tokens and 40 cents on the output. And if I go ahead and compare this with Opus 4.6, you can
Speaker A
see that the cost is just a fraction. Down here, we have five bucks and 25, and we have 14 cents and 40 cents. So, you could very well just use this OpenRouter combination with Claude Code to basically get yourself Claude Code
Speaker A
for like 50 to 100x cheaper rather than just being completely free, and I think that's still a huge win. So, you understand these two different methods of using Claude Code for free, whether that is with a local model or whether
Speaker A
that's through Ollama Cloud or OpenRouter, but what's still really important is managing context and tokens because these open source models don't have as big of a context window as something like Opus does with a million, for the most part. So, what you need to
Speaker A
watch next is my 18 token management hacks, which I will tag right up here, and hopefully I see you guys over there.
Speaker A
Appreciate making it to the end of the video. See you in the next one.
Topics:Claude CodeOllamaOpenRouteropen-source LLMlocal AI modelsGemma 4Qwenfree AI codingAI automationagent harness
