Speaker A
You know that feeling when you've spent 3 hours down a rabbit hole reading Reddit threads about running AI locally with the GPU benchmarks and the VRAMm requirements and the context windows and then you open your wallet, look at the
A creator tests Strata, a free MIT-licensed local AI engine that runs a 763B model on a 12GB GPU, and explains why viral speed claims mix up reading and writing speeds.
Key Takeaways
- Strata can run very large MoE models on a 12GB GPU by tiering experts across GPU, RAM, and SSD.
- Viral speed numbers often report prefill (reading) speed, which is far faster than generation (writing) speed.
- Generation on DeepSeek V4.1 Flash on 12GB hardware is roughly 4 to 6 tokens per second, which is usable but slow for heavy workloads.
- More system RAM (128GB+) and cheap SSD storage matter more than GPU VRAM for this approach.
What the video covers
- The creator, frustrated by 24GB VRAM requirements for local AI, tests whether Strata, a free open-source inference engine, can run large models on a 12GB consumer GPU.
- Strata launched on GitHub in late September, shipped 63 releases in 38 days, and gathered roughly 37,000 stars.
- It launched with Qwen 3.8 Flash Next (125B parameters) and later added DeepSeek V4.1 Flash (763B parameters, MIT license, 1M token context).
- Strata works by exploiting mixture-of-experts architecture: only a few of the 384 experts per layer are active for each token, so most of the model is idle at any moment.
- It places a model's hot core and most-used experts on the GPU, frequently used experts in system RAM, and rarely used data (including a 110GB engram table) on SSD, fetching it only when needed.
- A viral claim of over 100 tokens per second on an RTX 5070 referred to prefill (prompt reading) speed; the actual generation speed is about 4 to 6 tokens per second.
- Prefill is batched and efficient, while decode generates one token at a time and requires separate fetches from RAM or SSD, explaining the 20x+ gap.
- Cost analysis suggests SSD storage for the engram table is far cheaper than equivalent DDR5 RAM, and API pricing makes cloud generation much cheaper than local hardware for heavy output.
- Qwen 3.8 Flash Next is more usable, around 94 tokens per second in a Q2 build on suitable hardware.
- Community reports show older data-center cards like the MI50 can outperform some consumer cards on DeepSeek, and systems with 128GB+ RAM benefit most.
Chapters
- 00:00The GPU Problem and Why Strata Matters
- 01:35What Strata Is and Its Rapid Rise
- 03:13How Mixture of Experts Enables Tiered Memory
- 04:50The Viral Speed Claim Explained
- 07:42Hardware Costs and Price Comparisons
- 10:00Real-World Speeds Across Models and Setups
- 13:20Cloud API vs Local Economics
- 16:40Conclusions and Where This Is Heading
Full Transcript — Download SRT & Markdown
Speaker A
number and slowly close it again. That was me for almost a year. Every time I got excited about running a big model locally, I kept hitting the same wall, which was minimum 24 gigs of VRAM. And every time I looked at a card with 24
Speaker A
gigs, the price made me want to go lie down. And my machine is perfectly solid with a Ryzen 5, 64 gigs of RAM, and an RTX card with 12 gigs of VRAM. So, nothing flashy. But every forum, every
Speaker A
YouTube video, and every comment section told me the same thing, which was that 12 gigs isn't enough and I should go bigger and spend more. So, I waited and watched. And I told myself that maybe the next GPU cycle would be different.
Speaker A
And then in late September this year, something actually changed. A free engine called Strata dropped on GitHub with an MIT license and a desktop app for Windows, Linux, and Mac. And it claimed with receipts to run a 763
Speaker A
billion parameter model on a 12 gig card, which is exactly the card sitting in my machine. So I immediately lost my mind and had to figure out whether this was real and whether it was actually usable. And most importantly, I wanted
Speaker A
to know what the catch was because there's always a catch. This video is everything I found. From the good and the genuinely impressive to the wait, this is slower than I thought and the part nobody's putting in their
Speaker A
thumbnails. So, let's get into it. Now, first, what is Strata and why does it matter that it existed for only 38 days before people started losing their minds over it? Strata is a local inference engine, which is the software that
Speaker A
actually runs an AI model on your hardware. And it sits in the same category as Llama CPP or Olama, which you've probably heard of, but it's doing something architecturally different under the hood that we'll get to in a
Speaker A
second. It went up on GitHub on September 24th, and in the 38 days after that, it shipped 63 releases. I've had house plans that needed less attention than that. And it also collected around 37,000 GitHub stars, which in open-
Speaker A
source terms is the equivalent of everyone in a medium-sized city standing up and applauding at the same time. And the model it launched with was Quen 3.8 Flash Next at 125 billion parameters, which for context is already enormous.
Speaker A
And we're not talking about a cute little 7 billion parameter model you run as a party trick. We're talking about serious reasoning capability. Then in late October, they added the model that really made the internet go sideways, which was Deepseek V4.1 Flash, and it
Speaker A
has 763 billion parameters, an MIT license, a 1 million token context window, and a claim that it runs on a 12 gig card. And that last part is why people started posting screenshots at 11 at night and tagging their friends. But
Speaker A
one thing I've learned from years of being an excited early adopter is that when something sounds too good to be true, it's usually not false. It's just incomplete. And the complete story here is a lot more interesting than the
Speaker A
headline. Now, the part that actually blew my mind once I understood it isn't magic at all. It's just really clever engineering. You'd assume that to run a model, the whole thing has to live in your GPU's memory, and that's what I
Speaker A
assume, too. And that's what all those minimum 24 gigs VRAM requirements are based on. And for some models, it's more or less true. But modern large models, especially the ones called mixture of experts models, don't work that way. And
Speaker A
Deepseek V4.1 Flash is one of them. So instead of one giant brain that always uses all of itself, a mixture of experts model has hundreds of small specialized chunks, which are literally called experts. And for each word it generates,
Speaker A
it only uses a small handful of them. When we look at DeepSeek, it has 40 layers, each holding 384 experts. And for every single word it writes, it uses six of those experts per layer. So at any given moment, most of the model is
Speaker A
just sitting there doing nothing. And on top of that, DeepC carries something called the engram, which is basically two lookup tables totaling 196.6 6 billion parameters or a quarter of the entire model. And for each word, the model dips into those tables for a
Speaker A
handful of rows and then ignores the rest. So what Strata does is exactly what you're already thinking. It figures out which parts of the model get used constantly, which parts [music] get used sometimes, and which parts barely get
Speaker A
touched, and then it puts each part in [music] the cheapest place it can still be read from fast enough. The hot core of the model, which every single word needs, lives on your GPU. And the experts [music] that get used most
Speaker A
often, go on the GPU, too. As many as will fit. The next tier of frequently used experts, goes into your regular RAM. And the stuff that barely gets touched, including that entire 110 GB engram table, lives on your SSD and only
Speaker A
gets pulled when a specific word actually asks for it. So, we end up with three tiers. [music] The GPU, the RAM, and the SSD, with each bite in the cheapest place that's still fast enough for its job. And that's how we run a 763
Speaker A
billion parameter model on 12 gigs of VRAM because the card doesn't hold the model. It only holds the hot part of it.
Speaker A
I genuinely sat back when I understood this and just went, "Okay, that's actually brilliant." And it's like a chef who keeps their mostused knives on the counter, their occasionally used gadgets in a drawer, and their one purpose specialty tools in a box in the
Speaker A
pantry. And they aren't slower because the box is in the pantry since most of the time they never need what's in it.
Speaker A
Now, I have to stop and tell you about the number that went viral. And this is the part that almost made me make a different video and a much angrier one.
Speaker A
On October 25th, a post on X claimed that Deepseek V4.1 Flash was running on a 12 gig RTX 5070 at over 100 tokens a second and it got almost 100,000 views.
Speaker A
And another post the same day said 105 to 117. And people were quote tweeting it with fire emojis and local AI is here energy. I saw those posts and got excited. And I did the mental math of what over 100 tokens per second on a
Speaker A
model that size would mean for actually using it as a coding assistant. And I even started drafting a list of things I'd finally be able to do. And then I actually read Strata's own documentation. Strata's performance table for that exact card. The 12 gig
Speaker A
RTX 5070 shows two numbers. So, are you ready for them? It reads your prompt at about 110 tokens a second and it writes the answer at 4 to6. So, the viral number was the reading speed, which is the thing it does before it says a
Speaker A
single word back to you. And I'm not mad at the people who posted it because I probably would have made the same mistake if I'd been the one running the benchmark. But these are two completely different operations that perform in
Speaker A
completely different ways. and the gap between them is enormous. Now, reading your prompt is called prefill and it processes thousands of tokens all at once in a batch. So, when it fetches an expert from RAM or the SSD, it gets a
Speaker A
lot of use out of that fetch across the whole batch. Writing the answer is called decode and it's fundamentally different because it generates one token at a time and each token needs its own specific set of experts and each one is
Speaker A
its own round trip to wherever those experts are stored. On a 12 gig card running DeepSeek, a lot of those experts are in RAM or on the SSD and we're doing a fresh trip for each one. And that's
Speaker A
why we get 110 tokens per second reading and four to six writing. Two numbers that are more than 20 times apart. So, if you're in a comment section right now or you see someone sharing a screenshot with a big number on it, the first
Speaker A
question to ask is whether that's the reading speed or the writing speed. And those are two completely different conversations about two completely different things. Now, let's talk about what 4.7 tokens per second actually means for real use, because I think this
Speaker A
is where a lot of the excitement needs to get grounded. At that speed, a 1,000 token response takes about 3 and 1/2 minutes of staring at the screen and waiting. And that's roughly a long function plus an explanation. The kind
Speaker A
of thing we ask a coding assistant for all the time. I tried this and genuinely sat there and watched it. I have the patience of a person who once waited 25 minutes for a progress bar in 2007 because the alternative was starting
Speaker A
over. But 3 and 1/2 minutes of watching a blinking cursor is its own kind of suffering. Now, I want to be fair here because Quen 3.8 Flash Next, the 125 billion parameter model, runs at around 94 tokens a second on the same hardware.
Speaker A
That's genuinely fast. A 1,000 token answer in about 11 seconds, and it's a usable model on a 12 gig card, which is where Strata actually delivered on its original promise. But Deepseek is a different story, and its reading speed
Speaker A
only makes the writing speed feel worse by contrast. The first thing that happens is that your prompt reads in fast and you think, "Okay, here we go." And then it just crawls. There's also the prompt reading side to think about.
Speaker A
A real coding agent setup that knows your project structure, your tool definitions, and your current file might open with 30,000 tokens of context.
Speaker A
[music] And on the 5070 PC running DeepS in its best reading configuration, that's over 2 minutes before it writes a single character back every time we start a new task. One creator who actually ran an agent workflow with
Speaker A
Deepseek on this setup raised the context window, which we need to do for any serious work. And after that, a single simple question about a small game's codebase took 11 minutes and 30 seconds. And that was on a 12 gig card
Speaker A
plus a 24 gig card running together. An answer that slow to a simple question isn't an AI assistant anymore. It's a very polite email thread. I want to be really careful here not to sound like I'm trashing Strata because I'm not and
Speaker A
I'm genuinely impressed by what they built. But I think the right framing is that Strata made it possible to run DeepSeek on consumer hardware. And what possible feels like in practice depends a lot on your hardware and your
Speaker A
patience. And on a card that size, possible and pleasant aren't the same thing yet. Now, the part I spent the most time on is where the conversation about you [music] don't need an expensive GPU gets genuinely complicated. Strata's whole bet rests on
Speaker A
one idea, which is that gigabytes in RAM are cheap and gigabytes on the card are expensive. So, we keep as little as possible on the card and let the cheap RAM do the heavy lifting. That was an excellent bet when Strata was being
Speaker A
designed, but it's a harder bet today. So, let me run the actual price list as of late October because this is the math nobody's putting in their thumbnails. A used RTX 3060 with 12 gigs is about $310 on eBay, which is roughly $26 per GB of
Speaker A
VRAM. A 64 GB kit of DDR5 desktop RAM, going by the median price across 77 kits in stock right now, is about $1,200 or about $19 per gigabyte. And the thing that stopped me cold when I did this
Speaker A
math is that DDR5 RAM now costs about 70% of what a used 3060s VRAM costs per gigabyte. And a year ago, the same kind of kit was about $191 or $3 a gigabyte.
Speaker A
Today, it's $19, which is more than six times the price in 12 months. So, when we look at the build, most people in the Strata community are recommending I used 3060 plus 64 gigs of DDR5. That's $310 for the card and $1,200 for the RAM. So,
Speaker A
the RAM costs four times as much as the graphics card. And the expensive part didn't leave. It just moved one slot to the left. One viewer under a Strata video put it better than I could. In Canada, a 5070 Ti costs what a 4090 did
Speaker A
at launch. and not needing an expensive card is doing a lot of work in that sentence. Now, there's actually a genuinely cheap path here, and it's the SSD, which is where Strata keeps pushing as much as possible. A 2 TB PCIe 4 NVM
Speaker A
drive runs about $253 right now, which is 13 cents per gigabyte and 150 times cheaper per gigabyte than DDR5. And Strata is constantly finding ways to move more of the model onto it.
Speaker A
Deepseek's 110 GB Engram table comes to about $14 of SSD, whereas the same table sitting in DDR5 would cost about $2,100.
Speaker A
So every gigabyte that moves from RAM to the SSD saves us roughly $19. But the catch is that the SSD only works for bytes that don't need to be read constantly. And on this particular model, a lot of what matters gets read
Speaker A
constantly. The SSD tier is 10 times slower than DDR5 in bandwidth. And DDR5 is already four times slower than VRAMm.
Speaker A
So the slow tier is where we really feel the cost of running a model that's too big for our hot tiers. And if you're on DDR4 instead of DDR5, which has its own interesting math, a 64 gig DDR4 kit is
Speaker A
about $500 or roughly $7.80 per gigabyte. So you get more than twice the capacity per dollar. The trade-off is about half the bandwidth, which means the CPU half of every token takes about twice as long. you get more capacity and
Speaker A
slower operation. And depending on what you own, that might be the better deal. So, the right question isn't which GPU should I buy. It's what's my per GBTE cost across all three tiers and how many GB does the model I want to run need in
Speaker A
each one? That's the math that tells us what our actual options are. Now, there's a result in Strata's performance data that genuinely broke my brain a little, and I have to share it because it's both hilarious and deeply
Speaker A
instructive. When we look at Strata's performance table for Deepseek V4.1 Flash on their reference hardware, the RTX 5070 PC writes 4 to six tokens a second. That's a modern 12 gig card with a 6 core Ryzen 5, 64 gigs of DDR5, and a
Speaker A
3.6 GB per second SSD. Now look at an RTX 3060, a card from 2021 with about half the memory bandwidth of a 5070 and only 46 gigs of RAM instead of 64 running the same model on the same task.
Speaker A
[music] And that PC writes 9 to 11 tokens a second. So the machine with the cheaper older card is twice as fast. I can't stress enough how counterintuitive this is. I've spent years internalizing the idea that a better GPU means better
Speaker A
AI performance. And here's a card from 2021 with objectively worse specs producing a faster result on the same model. The reason is obvious in retrospect once we understand it. When most of the model isn't on the GPU, the
Speaker A
GPU isn't the bottleneck anymore. The CPU, the RAM, and the SSD are, and the GPU just finishes its share and waits.
Speaker A
Strata's own model guide says it plainly that on a 12 gig card, most of each token's time is spent waiting on the SSD and the CPU. The 3060 machine happened to have a better CPU, faster storage, or some combination that made the non-GPU
Speaker A
work faster. And the expensive newer card had nothing to speed up because the work had moved somewhere the card couldn't reach. The same thing showed up in a comparison between a $7,770 RTX5090, which is the flagship, and a $700 used
Speaker A
AMD data center card from 2018 called the MI50. and both have 32 gigs of memory and the 5090 is 11 times the price. On DeepSync, the Mi50 wrote 22 to 23 tokens per second, while the 5090 got about 15 to 25 depending on conditions.
Speaker A
So, the 2018 server card from eBay matched or beat the flagship consumer GPU that cost nearly $8,000.
Speaker A
Now, there are important caveats because they use different settings. The 5090 was running a longer context and these were single runs on a warm cache. The comparison isn't perfectly clean, but the ballpark is real. What actually matters for Deep Seek's decode speed
Speaker A
isn't the GPU chip, but the number of experts we can keep on the card. And that's decided by capacity, meaning gigabytes, not by how fast those gigabytes are. The MI50 kept about 2,700 of Deep Seek's 15,360 experts on the card, while the 5090 kept
Speaker A
about 1,450 because its owner had given a lot of the card's memory to a long context. And more experts on the card meant fewer trips to RAM and the SSD per token, and fewer trips meant faster tokens. [music]
Speaker A
So, the lesson isn't buy a cheap card. It's buy the right amount of capacity for the tier that's your actual bottleneck and stop paying a premium for speed in tiers that aren't for DeepS.
Speaker A
That means we do need a bigger GPU, just not an expensive one. Let me get concrete because after all of that, what you probably want to know is what you can actually run on the hardware you already own. Strata serves two main
Speaker A
models right now and they're genuinely different experiences. Quen 3.8 8 flash. Next at 125 billion parameters is the one that actually delivers on the headline. [music] In its Q2 build, it runs at about 94 tokens a second on the
Speaker A
5070 PC, which is noticeably fast because you ask it something, it thinks for a moment, and then it just writes.
Speaker A
At that speed, a 1,000 token answer takes about 11 seconds, and a 30,000 token coding prompt prefills in about 11 seconds as well. That makes it a working AI assistant at local speeds. The lookup table part of this model is 28.8 gigs
Speaker A
and lives on the SSD. And the active experts need about 34 gigs of RAM while the hot part that runs on the GPU is only about 3.5 gigs. [music] So a 12 gig card runs it comfortably and it runs
Speaker A
fast. Deepseek v4.1 flash is the 763 billion parameter 1 and the experience there depends almost entirely on your RAM situation because on 64 gigs with 12 gigs of VRAM were at 4 to six tokens a second but one user with 128 gigs a 5070
Speaker A
Ti and a fast SSD averaged 28.8 tokens a second on the smallest quantization and on a 32 gig card from 2018 we can hit 22 to 23. So, the model scales with capacity, and on most gaming PC setups,
Speaker A
it lands in the technically works but requires patience category. Now, a quick word about quality because this matters.
Speaker A
The recommended build for both models is a twobit quantization, which is two bits per weight and the most compressed version that still performs well. On benchmark tests, the Deepseek 2-bit version actually hits 100% of the original's average on several hard
Speaker A
reasoning tasks. But the caveat is that it might need more tokens to think its way to the same answer. And at 4.7 tokens per second, thinking is the most expensive thing it can do. And keep in mind how fast Strata is still changing.
Speaker A
It shipped 63 releases in 38 days. And some of those releases turned performance optimizations on and then off again after they caused quality regressions. In one case, a routing shortcut was added for 12 gig Nvidia cards that made code generation 37%
Speaker A
faster. But users reported it repeating itself in long sessions. And when the team checked, they found it failed 4.5 out of 10 [music] agent tasks instead of the normal 5.6. So they turned it off 2 days later. That's not a criticism. It's
Speaker A
just what real open- source development looks like at this speed. But it does mean the number you saw benchmarked two weeks ago might be from a setting that no longer exists in the same form. And now the part I kept not wanting to say
Speaker A
out loud because it kind of undercuts the whole premise of this video, but I think you deserve to hear it. If you specifically want to run Deepseek V4.1 Flash and you're on a 12 gig card with 64 gigs of RAM, just rent it. Deepseek's
Speaker A
own API charges 60 cents per million output tokens off peak and $1.20 at peak. The DDR5 kit you'd buy to run this model costs about $1,200.
Speaker A
And at the off peak API price, that $1,200 buys 2 billion output tokens. At 4.7 tokens per second, a local setup would need about 13 1/2 years of non-stop writing to produce 2 billion tokens. And even at the peak API price,
Speaker A
it's still almost 7 years. And the API is also running the real model, not the two-bit version using five of its six experts. I'm not telling you this to be discouraging. I'm telling you because the community conversation sometimes
Speaker A
gets so focused on whether something can run locally that it skips right past whether it should. The exception, and it's a real one, is privacy. If your code can't leave your machine, if you're under a client contract, or if you're
Speaker A
working with a codebase that can't touch an external API, none of that math applies, and local is the only option.
Speaker A
So cost per token is irrelevant. For everyone else, the question is the same as it's always been. Spend on capacity in the fastest tier you can afford per gigabyte. Skip the newest chip and match the hardware to the model's actual
Speaker A
access patterns. A 32 gig card from 2018, chosen for the right reasons, got further on DeepSeek than a 12 gig card from this year. That's a practical conclusion rather than the headline, but it's the one that tells us where to
Speaker A
actually put our money. Since RAM is the variable that matters most right now, I sorted the practical breakdown by what you might already own because the last thing we want is to buy hardware and then find out we didn't need to. If you
Speaker A
have 16 gigs, Strata can load Quinn 3.8 terminate flash next in a low RAM mode and pull experts off the SSD. One developer measured that at about 8 tokens a second on a 3060 with a single 16 gig stick which isn't fast but it
Speaker A
works and for learning tinkering or occasional use it's real. There's also a community fork called Battle Juice that one developer built to squeeze more out of this configuration and they reported 22 sustained tokens per second on the same box with a different quantization.
Speaker A
It's a oneperson project from recently and it has trade-offs, but the direction it points is clear. Engines are getting better at needing less RAM, so more of the model can move to that 13 cent per GB SSD. If you have 32 gigs, you're in a
Speaker A
sweet spot for Quen. And a Reddit post with almost 900 up votes ran the Q2 build on a laptop with an 8 gig card and 32 gigs of DDR5 and got up to 30 tokens a second. Strata also ships a coder
Speaker A
build that keeps 256 of each layer's 512 experts and needs about 30 gigs. Or there's Quen 3.635B, the smaller cousin, which strata measured at 59.7 tokens per second on a 3060. So at 32 gigs, we have real fast
Speaker A
local AI options. If you have 64 gigs, you're strata's target user because every Quinn 3.8 flash next build fits comfortably. And Deepseek runs too slowly on a 12 gig card, but it runs.
Speaker A
And on a bigger card, it becomes genuinely useful. And if you bought this RAM a year or two ago, congratulations, because those 64 gigs are now worth significantly more than what you paid for them. And Strata [music] is arguably
Speaker A
the best use they've ever had. If you have 128 gigs or more, DeepSync starts to make real sense. and one owner with a 5070Ti, a high-end Ryzen, and 128 gigs, averaged 28.8 tokens a second on the smallest quantization with a 4-bit KV
Speaker A
cache. And when it's spilled onto the SSD, it dropped to 19, [music] which is still more than four times what a 12 gig card gets. So, at that size, DeepS becomes an actual coding assistant rather than a patient's test. And if
Speaker A
you're thinking about buying RAM right now, check DDR4 prices first. At about $7.80 80 cents per gigabyte against DDR5's $19. The same money buys more than twice the capacity. We do lose bandwidth, which [music] matters. But if the choice is between 32 gigs of DDR5
Speaker A
and 64 gigs of DDR4 for the same price, the extra capacity wins for these workloads because the bottleneck is usually capacity and not speed. So, do you need an expensive GPU anymore?
Speaker A
Strata genuinely moved the answer because a 125 billion parameter model running at 94 tokens a second on a 12 gig card from 2021 is not something that was true before. And the fact that I can run Quinn 3.8 flash next on my machine
Speaker A
right now today without buying anything and have a fast local AI assistant [music] is new and really is worth something. The bigger model DeepS is more complicated. We can run it on a 12 gig card, but we have to be honest about
Speaker A
what that experience looks like on that card at this moment with this hardware where for some workloads it's fine and for others we'll spend more time waiting than working. What Strata actually did is shift the bottleneck where the GPU
Speaker A
stopped being the thing we had to spend on and RAM became the thing we have to spend on. And since RAM got six times more expensive this year, the price didn't leave, it just moved. But the direction Strata is moving in is right
Speaker A
because every release gets more of the model on the SSDs at 13 cents a gigabyte and squeezes more out of less RAM. And the developer own PC is a $300 range used card and a mid-range CPU. So they're optimizing for the hardware most
Speaker A
people actually own. Before you buy anything for this, make a simple list of what GPU you have and how many gigs, what RAM you have, and whether it's DDR4 or DDR5, and what SSD you have. and then look at what Strata's actual performance
Speaker A
tables say for your tier. Don't buy based on the number that went viral without checking which operation that number measured. And don't buy a faster card when the bottleneck is your RAM.
Speaker A
And don't buy more RAM when the model you actually want to run fits in what you already have. The expensive GPU really did stop being the price of entry. And the only problem is that the part that replaced it now costs almost
Speaker A
as much per GB, which is a much more interesting problem to have. If you found this useful, the like button is right there and it genuinely helps. And if you want more videos where we actually read the docs instead of just
Speaker A
reading the tweets, subscribe and you'll see the next one. And let me know in the comments what GPU and how much RAM you're running. I'm curious what setup most people watching this actually have because I think it might be different
Speaker A
from what most of the online discourse assumes.
Topics:Stratalocal LLMmixture of expertsDeepSeek V4Qwen 3.8VRAMtokens per secondGPU inferenceopen source AIRAM offloading
