Creator demos a fully offline AI assistant 'Jarvis' running local LLMs, local voice cloning and local memory on a Mac, and explains hardware and model choices.
Key Takeaways
- A capable personal AI assistant can run entirely offline on consumer hardware with local LLM, voice and memory
- Unified memory size is the main constraint determining which models can run well locally
- Local and cloud model backends can be switched on demand within the same assistant
- Open-source tools like Ollama and Voice Studio make the local setup accessible without ongoing API costs
What the video covers
- Demonstrates Jarvis, a personal AI assistant that can switch between cloud models (Anthropic API, ElevenLabs voice) and fully local models
- Runs local LLMs such as Qwen 3.5 (9B) and Gemma models (12B, 27B) via a local machine, with no subscriptions or API bills
- Shows switching voice from ElevenLabs to a local voice clone using the open-source Voice Studio platform
- Demonstrates tool access to Gmail, Google Calendar, Notion and local file retrieval from a 'second brain' knowledge base
- Explains hardware guidance: Mac Mini, Mac Studio or Windows machines, and how much unified memory is needed for different model sizes
- Gives a rough rule of thumb of about 0.6 GB of memory per billion parameters for model sizing
- Recommends matching model size to hardware budget, with larger models requiring high-memory machines
- Walks through installing local models with Ollama and downloading models via the interface
- Offers a free prompt pack and setup guide, with the full guide and classroom available through a paid community
- Mentions additional Jarvis features such as hand-gesture control, phone calls, text and Telegram interaction
Chapters
- 00:00Introduction: Jarvis Running Fully Offline
- 01:04Video Overview and What You'll Learn
- 02:11Live Demo: Switching Between Cloud and Local Brains
- 04:05Local Voice Cloning and Tool Integrations
- 06:30Hardware Requirements and Model Sizing
- 10:18Choosing Hardware: Mac Mini, Mac Studio and Pricing
- 12:58Installing Local Models and Voice Studio Setup
- 17:00Wrap-up and Community Resources
Full Transcript — Download SRT & Markdown
Speaker A
Jarvis, what machine are you currently running on? Your local Mac, sir. A rather powerful one, if I may say so. Powerful enough that I've stopped returning the cloud's calls. And before you ask, everything is local. Local brain, a local LLM, local
Speaker A
voice, local memory. Nothing you say leaves this room, and nobody is listening in. The only thing I share with the internet these days is a healthy distrust.
Speaker A
Switch to Gemma for 12 billion parameters. Gemma 4 fitted sir 12 billion parameters all of them local. Give me a moment the fan and I are getting acquainted. So my personal AI assistant Jarvis now runs completely on a separate computer fully
Speaker A
offline. So in this video I'm going to show you how Jarvis works. How you can build your own AI assistant like Jarvis and run it fully offline on a machine like a Mac Mini, a Mac Studio or a
Speaker A
Windows machine. We're talking local AI models, local voice and local memory. No subscription and no API bills at all.
Speaker A
And I'm also going to give you this full prompt pack and guide completely for free, which gives you all of the prompts and all of the guides you need to build your own Jarvis today. And then most importantly, I'm going to talk about
Speaker A
which hardware works best, what to buy, how much unified memory you're actually going to need to run these models, which AI models run best on which machine, and how to set all of it up. All right, let's get started. All right, so just so
Speaker A
you guys know what's going to happen in the video, feel free to skip around if you want to. I'm going to have the chapters below. So first of all I'm going to start with a demo so that way
Speaker A
you can see exactly how Jarvis performs when it's running locally on my machine with all local models and everything completely private right and then afterwards I'm going to talk about uh the difference between what type of machines essentially what hardware is
Speaker A
going to be required for running different AI models and then I'm going to talk about which models will fit on the hardware of your choosing depending on your budget. Like I said, there's going to be lots of differences here.
Speaker A
And then, of course, like I mentioned, I'm going to give you guys this uh full prompt pack, which is going to include all of the guides, including the hardware setup and everything else and all of the prompts you need to set up
Speaker A
your own Jarvis on your own machine. And then I'm also going to quickly point out how you can uh install local AI models using uh Olama and then also an open- source model for the voice cloning and voice interaction as well. Okay, so
Speaker A
that's going to be the whole video here. So, let's go ahead and start with a full demo first. Now, for those of you who have seen my previous video, this is Jarvis. And what you're seeing spinning here, these are all of my local files,
Speaker A
my local local brain, local second brain, right? And then Jarvis on the right hand side, as you can see, has um multiple options as far as the look and feel of it. And then it has access to a
Speaker A
bunch of different tools. So, he becomes a truly personal AI assistant. Now, by default, I've been running them on Opus 5.5 and then on 11 laps. But now, if I want to run them uh on my local machine,
Speaker A
I can just basically interact with them and just say something like, "Jarvis, switch to your local brain." So now Jarvis is thinking Quen 35 fitted sir 9 billion parameters running right here on your Mac. I'm smaller now, so do speak slowly.
Speaker A
So there you go. As you can see, Jarvis is now running. when I say local machine by default I've set that cuz I like really like Quen 3.5 the 9 billion parameter model uh uh so it automatically switches to that but I
Speaker A
have a bunch of other local models because I have a pretty powerful machine so I have a ton of different models that I've downloaded so something like a Gemma 4 12 billion parameters and higher uh models I'm going to talk about the
Speaker A
hardware setup in a little bit but again same thing it's very simple all I have to say is just tell him to jus go ahead and switch to Gemma for 12 billion parameters That's all I have to tell him and then
Speaker A
he understands and first of all finds out if he has access to that particular model and then of course he'll be able to switch his brain.
Speaker A
Right then sir consider me an upgrade less prone to existential crisis than the previous model and marginally more patient with your requests.
Speaker A
Actually you must have not heard me right but you can see uh I had him also switched to the Gemma 327 billion parameter model as well. So, I have a bunch of different Gemma models that I've downloaded, but you get the idea,
Speaker A
right? Like I'm I'm very I can very easily tell him to switch his uh models, his brain to different models. And I can always tell him to go back to the cloud uh brain as well. JS, go back to your
Speaker A
cloud brain. Opus 55 online, sir. Full strength, full price, full sentences. There you go. So, now that's how easy it is for me to switch back and forth between the cloud brain, which operates with APIs. Of course, um I use Antropic
Speaker A
APIs, so he can switch very easily. And the same thing, um right now he's running on 11 Labs for voice interactions. That's why you see how fast it is. But I can also switch his voice to run on a local uh um voice
Speaker A
clone using Voice Studio, which is a open- source platform that I'm going to talk about in a little bit. But same thing, I can say, Jarvis, switch to your local voice.
Speaker A
There you go. So now voice studio opens up because I had uh voice studio was turned off. So that's why it automatically opens that up. But if it's already if you have already voice studio open then how do I sound sir? This is my local
Speaker A
voice. There you go. So that's how uh easy it is for. So by the way I was going to point out that if you already have voice studio open. So we'll just quickly switch to that. But just because I had
Speaker A
turned off the application, that's why it took a couple of seconds to open the application and then it said now it's my local voice. Right? So that's how simple it is for me to have Jarvis move from cloud to local version completely. And
Speaker A
like I said, now right now because I can have him run everything locally on my Mac, then it means that it's completely private and it has no access to anything outside. And of course, he can do a lot
Speaker A
of things, right? So I have the ability for him to use all of the tool at his dis all of his tools at his disposal like the Gmail, Google calendars of the world, notion and then I can always give
Speaker A
him more tools and of course he can always uh I can always have him interact with my local my second uh brain here and pull up any of the files or anything that I want and it's very simple to do.
Speaker A
So I can say something like Jarvis pull up my free school community. So there you go. Now it's going to find out all of the folders.
Speaker A
I've pulled up all the information sir. You can read it yourself or if you'd like say read it out loud and I shall.
Speaker A
There you go. So now you can see it's pulling up uh all of the stuff that's related to my free school community. Of course, free school community is my business that I run where I provide you guys all of the free resources and also
Speaker A
I have a paid community. It's all part of my brand, right? So that's how you can how I can interact with Jarvis uh to my second brain. Of course, I used Opus.
Speaker A
just realized I used open opus 5.5 but this is very easily done with Quinn or other models as well. So that was a quick demo. I've done a bunch of different um you know I've done a lot of
Speaker A
demos on how Jarvis works. Uh uh you guys can check out my previous videos in my channel to see I mean right now he has a ton of different features right uh I can interact with them just by my
Speaker A
hands and stuff using hollow hands here. Uh I can he can make phone calls on my behalf. I can call um I can call him directly as well. I can interact with him via text, via Telegram, and so much
Speaker A
more. So, like I said, if you're interested in checking out all of the functionalities of Jarvis, check out my previous videos there. All right, so that was kind of a quick demo. Now, let's go ahead and jump in and talk
Speaker A
about uh the different um machines or the different hardware that you uh can set up and what type of models you run in. So, what I'm going to do is focus on Mac Mini or Mac Studio. And then on the
Speaker A
prompt pack, I also have a guide on uh oops, I have a guide on uh Windows machine using Nvidia card or raisin or uh if you have, for instance, the Nvidia Spark, right? That's a powerful machine there. So, there's lots of information
Speaker A
there for you guys to check out. But in this particular example, I'm just going to quickly talk about the Mac Mini and Mac Studio because these are a good representation of what uh kind of the smaller version of a machine, a local
Speaker A
machine that people have like a Mac Mini or a MacBook Pro like it kind of have the same type of configuration and then a Mac Studio which is a very very powerful machine. Right? So I'm going to compare these two and talk about the
Speaker A
unified memory and which models run best on what type of hardware and then you can make a decision on which route you want to go to. All right. So the Mac Mini and Mac Studio, all of these specs
Speaker A
are pulled directly from apple.com. So you guys can check out their website. But essentially what I want to focus on, the two most important things is uh the local storage and then most importantly the unified memory. Right? This is going
Speaker A
to be the main thing here that we're going to focus on because that's what determines what kind of AI model you can run locally on a machine. The latency and how fast that interaction occurs between you and your personal AI
Speaker A
assistant. As you saw uh in my demo that you saw earlier, the latency is very low because I have a very powerful Mac. So therefore, yours might be different depending on what you're running. So if you have like a Mac Mini, the latency
Speaker A
might be a little uh a little longer. So, it's not going to be as natural, but it's still going to be pretty good enough. But if you have like an Apple 5 or Apple M5 Ultra, like a studio with
Speaker A
the Apple M5 Ultra chip, then the interaction is going to be exactly or similar to mine there as well. All right. So, the Apple Mini, the M6 chip right here, right now, by default, it has 32 GB of unified memory. And then
Speaker A
the uh Mac Studio, it could go up to 256 gigabytes of unified memory, which is absolutely insane. If you get this particular uh Mac Studio, you can essentially run any U model, any open- source model, right? As I mentioned
Speaker A
right here, unified memory is the whole game. And specifically on Mac, right? The good thing about the Mac Studio and the Mac software itself is the Mac machine itself is that uh the CPU and the GPU and the neural engine they all
Speaker A
share the same memory which means that an AI model that you download and run it has to fit in that pool to run it run all of it right because the bigger the pool is obviously the bigger the brain
Speaker A
will be for your AI assistant like Jarvis. So for instance for a Mac Mini that the fact that it has 32 GB of unified memory uh compared to like a Mac studio which is eight times more you can
Speaker A
get up to two 256 GB of memory you can think of the rule of thumb as for a model uh about point so 0.6 6 GB per billion parameters. You could think of it that way, right? So if you have a 27
Speaker A
billion parameter model, then what you do is you just basically multiply that by6 and that's the machine you end up with. So let me explain a little further here. So here are the models, open- source models that uh run really well on
Speaker A
Mac Mini and then also on Mac Studio. So for the Mac Mini, if you have the 32 GB of unified memory, you can run something like Quinn 3.8 27 billion parameters pretty comfortably, right? Uh uh models like Gemma 4, the 26 billion parameters.
Speaker A
Uh the GPOSS 20 billion parameters or these 12 billion, 9 billion parameters, these are very comfortably run on a Mac Mini or a MacBook Pro. But if you want to run a bigger model, right, so something like a
Speaker A
Quen 3.8 8 flash which is 125 billion which is 125 billion parameter model then you need something like a Mac studio that has a unified memory of 256 GB or like I said if you want to run something like a 70 billion parameter
Speaker A
model uh you can get a max studio with 128 GB and of course the higher the unified memory the more the cost is going to be and this is something that you guys can compare on your own max uh
Speaker A
on apple.com depending on where you are let's say if you're in the US or Canada or Europe the prices are going to change. So you can go ahead on apple.com go to max studio and then pick the exact
Speaker A
unified memory that you want and then based on that you will see the uh the price change there. But essentially like I said if you get the 256 GB unified memory you can run majority of the models. The great thing is you can
Speaker A
always combine. So you can get two Mac Studios if you want to run like a uh one trillion parameter model, right? And then as far as the um the uh storage itself, the storage, you don't have to have that much storage. I mean the Mac
Speaker A
Studio automatically comes with a 1 TBTE of storage. And the good thing about storage is you can always get an external hard drive, right? So that's not that big of a deal big a deal. the biggest deal is or the biggest um factor
Speaker A
for uh local machine or running locally these AI models is your unified memory. So that's going to be the main focus if you want to run a bigger uh model there.
Speaker A
So like I said all of these details I just wanted to go over it so that way you guys can see u which model fits your need. But like I said, all of this information and details I've explained
Speaker A
here uh and exactly which model is going to run uh really well on uh which particular machine there. Okay. So like I said, you guys can check this out. And by the way, to gain access to this uh so
Speaker A
this there's going to be two links in the description and the comments the first comment. The first one is going to be to the AI workshop community. The second one is going to be to this free community. It's called AI workshop
Speaker A
light, right? So AI workshop light just go ahead and click on the second one.
Speaker A
It's a completely free community. Uh once you click on join you're going to you're going to come to the community right here. You're going to go to the classroom section. You're going to click on YouTube resources and you'll have
Speaker A
you'll have access to that uh prompt pack right there. That'll give you all of the guides including the prompts on how to build your own Jarvis cuz I have included the prompts right here. Where to go? There you go. So, the Galaxy, the
Speaker A
brain, and a bunch of other ones. So, that way you can uh build your own type of Jarvis. But, of course, if you want to have the finished Jarvis like mine here that you see, everything is already set up and pretty. I mean, this took me
Speaker A
like 3 months to build, that's going to be inside the paid community. That's going to be the first link. So, click on that. It's called AI workshop. This is a paid community. Once you come here, all you have to do is go to the classroom
Speaker A
section. And then we're right now on version uh 8 version 8.1. So, all you have to do is just download the zip files for your machine and you can run it up. You can run it in like 5 minutes
Speaker A
or so. The setup is pretty simple. But like I said, if you want if you don't want to pay or join the paid community, uh you can uh uh essentially create this from scratch using that prompt pack guide. Now, as far as the models, how to
Speaker A
download the models, all you have to do is just go to Olama. So, just go ahead and create a free account and you click on uh get started, go to models, and here you can search what a model. So for
Speaker A
instance a Quinn right you can search the entire Quinn family of models. So you can see Quinn 3.5 gives you all of the the multiple billion parameter models that you can access to. So the anywhere from8 billion parameters which
Speaker A
is a very small model all the way to 122 billion parameter model. Right? So, this is something that's going to be the Mac Studio uh world or if you're using like Nvidia um Spark or something like that or a powerful Windows machine uh with
Speaker A
lots of graphic cards. So, that's where you can uh uh download this bigger models. But to download it is very very simple. Whatever model you choose, right, scroll down to the bottom right here. So, let's say you want to choose
Speaker A
this um 9 billion parameter model. So all you have to do is click on that and then on your terminal just copy this command right here. Right? So uh oh to run that and to install it all you do is
Speaker A
just pull and the name of the model. Again all of this is inside the prompt pack I've given you guys. So if you just go right here yeah so in case you don't remember there all you have to do is just doma
Speaker A
pull quen 3.59 billion parameter and then you do run if you want to like run it to test it right. Same thing the um the command is the same exact way. So you can do pull and then the name of the
Speaker A
model. So since this is quen 3.59 billion so let's say you want to download like the 27 billion one, right?
Speaker A
So if I go to the bottom here, oops, let me go back here. You click on the 27 and you just copy this and keep everything else the same.
Speaker A
So very very simple there. So that's how you run the uh that's how you download the local models. And then as far as the cloning, so this is a voice studio, right? This is an open- source software uh that you can actually clone a voice,
Speaker A
design your own voice, transcribe, like a really good uh software that's completely open- source. So all you have to do is just search Voice Studio.
Speaker A
There you go. And you just click on this one uh free open source voice cloning and dubbing. And this is exactly how you can grab. And by the way, just uh if you want to just go to the GitHub, it's
Speaker A
probably simpler. So just go to GitHub, click on this one, and if you scroll down to the bottom, all you have to do is just copy. Where to go? Where's the installation here?
Speaker A
Oh, here you choose your uh machine. So if you just click on this, it will show you exactly what to do to install.
Speaker A
Again, same thing. I've added this on uh this prompt pack as well for you guys to just be able to just copy and paste this here. Okay, so that's how you can download the models and then also the
Speaker A
voice cloning there. And essentially that's pretty much it. You don't need anything else after that, right? You uh download the models that you need. Of course, the first thing is buying the machine or whatever machine you have and
Speaker A
then afterwards you download the local models and then download the voice cloning uh um software and then pretty much it, right? The rest if you want to build your own, like I said, follow the free prompt pack here. Use these prompts
Speaker A
to build your own Jarvis or like I said, if you want to be lazy, just go ahead and join the paid community so that way you can uh pick up Jarvis uh with this zip file right here. And of course, if
Speaker A
you run into any issues, we have a ton of people in this amazing community of ours. You can post your questions. We can answer right away. And by the way, like a lot of my community members are now uh building their own versions of
Speaker A
Jarvis. Somebody just landed their first client selling Jarvis to somebody. So, lots of great things happening there if you want to check that out. Anyways, well, hopefully you find you guys found that helpful. Let me know if you have
Speaker A
any questions in the comments below. Make sure you like, subscribe. Thanks for watching and I'll see you on the next one.
Topics:local AIoffline AI assistantJarvisOllamalocal LLMvoice cloningMac MiniMac Studiounified memoryGemma











