Speaker A
So, we have seen this setup for the Claude code, and of course, we can use the Claude models. We can use Sonnet, Opus. The only thing is you have to pay for it. But, what if you have a machine
Learn how to use Claude Code for free by running local AI models with Ollama, avoiding paid services while managing hardware requirements.
Key Takeaways
- You can run Claude Code locally using open-source models via Ollama to avoid subscription fees.
- A capable GPU with sufficient VRAM is crucial for running larger models smoothly.
- Fine-tuned models like Gemma 4 work better with Claude Code than some default models.
- Local model performance may be slower but is manageable with proper hardware and context management.
- Ollama provides an easy way to download, manage, and launch AI models on your machine.
Summary
- The video explains how to run Claude Code using local AI models instead of paid cloud services.
- It introduces Ollama as a platform to download and run various open-source models locally.
- The presenter discusses hardware requirements, emphasizing the need for a GPU with sufficient VRAM.
- Different models like Gemma 4, Llama, and Qwen are tested, with fine-tuned versions recommended for Claude Code compatibility.
- Instructions are provided on how to install Ollama, download models, and launch Claude Code locally via terminal commands.
- The video covers performance considerations, such as speed and memory consumption when running models locally.
- It demonstrates practical usage by running Claude Code to explain projects and generate JavaScript files.
- Tips are shared on managing context and sessions to improve performance on larger projects.
- The presenter mentions integration options with extensions and manual configuration steps for enhanced usability.
- Overall, the video offers a cost-effective alternative to using Claude Code by leveraging local AI models.
Chapters
- 00:00Introduction to Claude Code and paid models
- 00:22Using Claude Code with local models
- 00:57Hardware requirements and model options
- 01:40Model sizes and compatibility
- 02:10Fine-tuned models and Ollama installation
- 02:39Downloading and managing models with Ollama
- 03:43Launching Claude Code locally
- 04:18Running Claude Code and performance insights
- 07:14Generating code and managing context
- 08:15Summary and final thoughts
Full Transcript — Download SRT & Markdown
Speaker A
which can run a local model? And again, I'm repeating this, your machine should be able to run the local model.
Speaker A
So, there are a lot of open-source models which we can use, and you can run it locally.
Speaker A
And you can connect your Claude code, or basically, you can have Claude code using the local models. Now, at this point, if you're doing that, you don't have to pay for the Claude services.
Speaker A
The only thing is it will be slow. But, if you can manage with the speed, you're good to go. Now, let me show you what are the options. So, one of the best options for running the local models is
Speaker A
Ollama. So, when you go to ollama.com, this is the official page, and there are a lot of models available here. So, if you click on models, you can download and use all these models locally. So, I have tried Gemma 4 and
Speaker A
most of the models here, with different Gemini options. I also tried Llama. But, you need to check what is your system configuration to do that. So, if you have a low-configured machine, specifically, if you have a GPU, you
Speaker A
need a GPU here. There are some models which work on CPU as well. So, if you look here, I'm using the RTX 4080. This is a laptop GPU, and it has 12GB dedicated GPU. So, if you have GPU memory, if you have
Speaker A
that, you're good to go for the models which I'm running. Otherwise, you have to look for the models, and maybe the lower version of it. So, for example, if you look for Gemma 4, there are different options here. So, we have
Speaker A
Gemma 4 latest, but if you can see, the size is 9.6GB. Not sure if you can fit in that. This will fit 4B, and even 12B works. And if you're using Mac, you can use any MLX version here based on the size.
Speaker A
Okay. Now, if you look at this model, not every model is capable of working with Claude code. So, even if you click here, you can see we have Claude code model Gemma 4, but I was facing
Speaker A
issues using that. It might work for you, or maybe you have to do some tweaks.
Speaker A
What you can also do is you can use some fine-tuned models, which you can try.
Speaker A
But, before you do that, the first thing you have to do is you have to install Ollama. So, when you go to the Ollama website, this is the way you can download. If you're using Windows, for Mac, depending upon which OS you're using,
Speaker A
you will get the option here to download. Now, on my machine, if I open terminal, I already have Ollama, and the way you can check that is by saying Ollama version, and you can see I do have Ollama installed. Now, once you have
Speaker A
Ollama, it's not like you'll have the model access directly. So, if you want to work with a particular model here, let's say if I go back to the models, and if I want to work with, let's say, Gemma 4 here, and you can
Speaker A
pick up any model. For example, let's say if I want to pick up 4B, so this is the command to run it. But, even before running it, it will download on your machine. So, I've done that already, and I have a few models that
Speaker A
are already downloaded. So, if I say Ollama list, these are the models I already have on this machine. So, you can see a lot of options here. And I found that this particular model, this fine-tuned for the Claude code, and it works. So,
Speaker A
you try with Gemma, and see if it is working or not. Qwen is also suggested by Ollama itself, but I'm going to use this.
Speaker A
And let me search about this here. So, on Ollama, you will find the fine-tuned model as well. So, this is what we are using here, which is for 4B, and this is the context 64K. So, you can't do
Speaker A
much with this locally, but you can manage. And it also runs fully on 16GB.
Speaker A
Okay, it was working well on my Mac. That's why it was working fine, because it is tuned for Mac.
Speaker A
But, you can try different models. You can test it out. Okay, now how do you run this? So basically, this is the command. So you can see it says Ollama launch Claude and then you can pass the
Speaker A
model name. So there are different options of doing this. Let me just go back to the Ollama page. So the first thing is you have to download that and the way you can do that is by going here and there's a
Speaker A
command. You can just run this and this will download and run this particular model. Now since I already have it here, I can directly run this. So I don't have to download again and again.
Speaker A
So that's one. Next you need to get the Claude. So you can just scroll down and you can see there are multiple options here which automate your work and Open Claude also works here. If I click on see more apps, one option is Claude
Speaker A
code. Just click on that. It will give you some steps to do that. So I will just zoom it a bit and if you go down, you can see it will suggest Qwen 3.5.
Speaker A
You try it out. I was happy with the fine-tuned model of Gemma 4. And this is how you do it. So you can say Ollama launch Claude. I will just copy this. And where should we do it? Of
Speaker A
course, I can do it in the terminal here or I will open VS Code and randomly I have selected one of the packages from the project which is front end and you can open any folder or maybe in the same
Speaker A
project which we are going to work on. And here you just have to pass that command which is Ollama launch Claude.
Speaker A
Okay? Now when you do this, it will launch it and it will ask you for which model. So basically, there are a lot of models available on this machine. So I'm going to select this one which is Gemma
Speaker A
4 4B Claude coder. I can go with this as well, but let's see if my system can handle it. But there's also the option of Qwen 3.6 agentic software engineering model with multimodal capabilities. The only thing is it is 28 GB.
Speaker A
My VRAM will die there. In this case, if you have a higher version, it will directly start using CPU.
Speaker A
And of course, we don't have good CPUs available to run these models. Now let me try this and run. So, you can see my Claude code is ready. You can even connect with this extension which you can download. So, you can connect
Speaker A
the extension, you just have to do some configuration and you can use this. But, I would suggest go to their website and do the remaining configuration by yourself. So, if you scroll down, you can do a manual setup here. You can set
Speaker A
the environment variables and then you can run. So, you can use that chat window as well. But, terminal works. So, if you are in the terminal, now you can start working. So, let me try. So, first I will say
Speaker A
explain the project. And while I do that, I will also show you my task manager. Let's see how much GB power it takes.
Speaker A
So, it's just going to kick off. Also, my camera or the recording is happening on the same machine. So, even that itself is consuming this and picked up.
Speaker A
You can see GPU is going a bit high. I hope it is not affecting the recording.
Speaker A
We will talk less. Okay, tokens per second speed looks good. Not bad actually compared to the local models.
Speaker A
Additionally, you need a good amount of memory as well to run this type of models because it is going to load in the memory as well.
Speaker A
Okay, so it is explaining the project. You can see this is what we have, right? The get task, create task. That looks good.
Speaker A
But, now if I try to ask it to create a JS file for calculator. Again, when you are working on a big project, it might struggle. But, there's a workaround as well. You can manage your context. You can manage your
Speaker A
specs, create a new session, or clear the current session to achieve that. Or maybe you can work with a specific method or a function at one time.
Speaker A
So, when I say create, this is going to be a long run. And sometimes it fails. Okay, so it thought for 13 seconds but no output. Let's try once again in the current folder.
Speaker A
You can set your thinking level as well. Okay, this time it is doing something.
Speaker A
Okay, so you can see it has created a file. I'll just minimize.
Speaker A
So we got the entire JavaScript file which has the code here and it will say accept. You can also add the test cases for this. So at this point you got the entire code. So it's your choice if you
Speaker A
want to use the cloud models or if you want to you run it locally.
Speaker A
But yes, running it locally also has some advantages. You don't have to pay for the models but this will be slow. Also, I would encourage you to explore other models as well. Maybe if you can try some other
Speaker A
fine-tuned models which you like and let me know your experience with the local models.
Topics:Claude Codelocal AI modelsOllamaGemma 4open-source AIAI model installationGPU requirementsAI coding assistantfine-tuned modelsfree AI tools
![[CC]FULL《炽热的夏天》EP01：寻找最合拍搭子！包上恩周柯宇礼物同频默契拉满｜#包上恩 #周柯宇 #炽… — Transcript](https://i.ytimg.com/vi/oHa4dHs7-2A/maxresdefault.jpg)
