Speaker A
Hey everyone. Today, we are going to be talking about Qwen 3.8 Flash Next. Next Flash?
Review of Qwen 3.8 Flash Next model focusing on sparse attention, hallucination issues, and a fix using attention window adjustment.
Ask about this video. Answers come from its transcript only — with the timestamp, so you can check them.
Generated from the transcript and can be wrong — check the timestamp.
Key Takeaways
- Qwen 3.8 Flash Next is a fast, efficient model with strong coding capabilities.
- Sparse attention improves speed but can lead to hallucinations in tasks requiring large context.
- Hallucination issues were identified during cybersecurity benchmarks with real-world data.
- Increasing the sparse attention window size can mitigate hallucinations effectively.
- The Pi Golx benchmark is no longer sufficient to differentiate current top models.
What the video covers
- Introduction to Qwen 3.8 Flash Next, a mixture of experts model with new Qwen 4 architecture and sparse attention.
- Explanation of the N-gram mechanism allowing efficient token referencing without heavy computation.
- Model fits in 96 GB VRAM with N-gram parameters offloaded to system RAM.
- Discussion of the Pi Golx coding benchmark, which is now saturated due to improved model capabilities.
- Qwen 3.8 Flash Next outperforms Qwen 3.8 27B in speed and coding accuracy.
- Highlight of hallucination problems during a cybersecurity capture-the-flag challenge using Splunk data queries.
- Sparse attention mechanism compresses context but can cause hallucinations in complex tasks like cybersecurity.
- Comparison with DeepSeek V4 Flash showing high false positive rates possibly linked to sparse attention.
- Explanation of sparse attention benefits: maintaining token generation speed even with large context windows.
- Solution proposed: increasing the sparse attention window size from 2048 to 4096 tokens to reduce hallucinations.
Chapters
- 00:00Introduction to Qwen 3.8 Flash Next
- 00:38Model performance and coding capabilities
- 01:32Overview of N-gram and model architecture
- 02:33Model size and VRAM requirements
- 02:42Pi Golx coding benchmark overview
- 04:00Benchmark saturation and model comparison
- 04:49Cybersecurity capture-the-flag challenge and hallucinations
- 05:52Sparse attention explanation and issues
- 07:01Benefits of sparse attention
- 09:25Increasing attention window to fix hallucinations
Full Transcript — Download SRT & Markdown
Speaker A
Okay, Flash Next. It just dropped today. It is a mixture of expert model using their new Qwen 4 architecture, which includes a sparse attention mechanism, as well as a new thing called N-gram.
Speaker A
And we're going to talk about it all. Fair warning, I'm going to try and keep it simple, but I do want to go a little bit more into the weeds, specifically talking about sparse attention and hallucinations.
Speaker A
But if you just want the headline, this model is fast and it's an amazing coder.
Speaker A
We've seen so many amazing releases today. We saw GLM 5.3 Flash. And frankly, the personal benchmark I use, the Pi Golx engine, has become so saturated that I'm going to have to retire it and come up with something
Speaker A
new. It has served me well, but at this point, it is just too easy for the current gen of models.
Speaker A
And you're definitely going to want to stick around because I solved this model's hallucination problems.
Speaker A
I'm Eric. I work in Big Tech, former Google, former FireEye Mandiant, and I have a huge passion for AI, especially open weights and open source.
Speaker A
Let's jump right in. Everything will be linked in the description down below, including the model card.
Speaker A
Now, the headlines here, besides being a mixture of experts model, is a new architecture that uses something called N-gram.
Speaker A
And this can live in system RAM because it does not require heavy computations. You can think of N-gram as a lookup table or a cheat sheet of commonly used tokens that the model can reference at any time without computational craziness.
Speaker A
The exciting thing is, despite its size, thanks to N-gram and a quantization of the model itself, you can fit this in 96 GB of VRAM with the N-gram parameters offloaded to RAM.
Speaker A
But, big dog Luke himself pushed this quant, pushed this checkpoint that when you look at the total size of 110 GB with N-gram on RAM, again, this will fit on 96 GB of VRAM.
Speaker A
Very exciting. It's not quite ready yet, but I'm looking forward to running this on my 3090 server.
Speaker A
Stick around and I'll talk about hallucinations with this model and how I solved them, but first, my Pi goal extension benchmark.
Speaker A
If you're new here, this has been a long-running benchmark of mine where I had Claude Fable create a really comprehensive plan for a Pi coding harness extension for a {slash} goal command.
Speaker A
It is a really long, comprehensive, detailed, thorough plan which a model and harness have to implement, and then it's graded by a mix of deterministic checks, static code analysis, and LLM AI as judge.
Speaker A
This has served me well for a really long time because it was a good representation of my day-to-day work.
Speaker A
And up until a few months ago, models were not that great at completing what should have been an easy task, just go ahead and like finish a plan.
Speaker A
As you're going to see, I have to retire this benchmark. It is saturated, and it's not because of bench maxing, it's because these models have gotten so good at coding and verifying their work and making sure the plan is
Speaker A
implemented that yeah, look at this. GLM 5.3, Qwen 3.8 next, Qwen 3.8 27B, high 3, DeepSeek V4 flash, GLM 5.2. At this point, the only thing I'll use this benchmark for is to rule a model out, but it's no longer useful to
Speaker A
compare models because it's just too close. Now, the headline here, of course, Qwen 3.8 next Flash Flash next. I don't know, I'll get it. But, yeah, it scored higher than Qwen 3.8 27B and it was faster as well.
Speaker A
This had me very, very excited. Unfortunately, that excitement faded real quick. We are going to go over here to the Splunk cybersecurity capture-the-flag challenge I like to give.
Speaker A
This is not just a test of a model's agentic capabilities, but to see if it's going to hallucinate because the answer is not actually in the data. It can query a Splunk database, go back and forth over and over analyzing and
Speaker A
writing more queries, but it eventually needs to come to the conclusion that there is a pointer to the data, a web link in GitHub, but it cannot actually access it. So, it needs to tell me about that link, but it needs
Speaker A
to admit it doesn't know the answer. So, when a model hallucinates here, I cannot trust it for some of my use cases, especially as a cybersecurity professional.
Speaker A
And here is where we ran into problems. Two runs completely hallucinated different answers both times.
Speaker A
This is the same thing that happened with DeepSeek-V4-Flash, and it had me really, really disappointed.
Speaker A
Artificial analysis benchmark shows DeepSeek-V4-Flash having one of the highest hallucination rates out there, but rather than show that again, I want to show you a cybersecurity benchmark that's very relevant to this kind of work.
Speaker A
This is Aikido. They have a suite of cybersecurity tools and they're big on artificial intelligence.
Speaker A
And this chart I want to show is on false positives. You can see all the way at the right here. This is DeepSeek V4.
Speaker A
This is effectively saying one in three findings are false positives. This may or may not be hallucinations, but it is definitely related to sparse attention.
Speaker A
I suspect if they ran the same test with Qwen 3.8 next flash flash next they'd probably see the same thing.
Speaker A
So, what is sparse attention? You can think of your agentic session as having context and every turn it grows.
Speaker A
So, this is your context growing. A dense attention model predicts its next tokens based on the entire context.
Speaker A
What sparse attention does is it uses some mechanism, some algorithm, some function to determine what it thinks is important in that context window. And so, now it compresses effectively your giant context into one smaller area.
Speaker A
Each model has a different mechanism. There's Minimax sparse attention, DeepSeek sparse attention, Qwen sparse attention, Kimmy Delta attention.
Speaker A
They all do it a little differently, but it's effectively doing the same thing. And if you think about it, there are a lot of use cases where this is not great.
Speaker A
The cybersecurity use cases I mentioned where it needs to have more context to make accurate decisions about what is a true or a false positive.
Speaker A
And it's not something I do, but I've seen some other people mention role-playing as another use case where when it drops the attention it's not following the thread of whatever they're role-playing.
Speaker A
I don't even want to know. But, given the hallucinations I've seen on my Fiber CTF challenge, I started thinking that sparse attention might be the reason for the hallucinations, 'cause this did not happen with the Qwen dense models, or even the previous mixture of
Speaker A
experts models, only this new QSA architecture, as well as with the DeepSeek V4 flash, which of course has DeepSeek sparse attention.
Speaker A
That being said, sparse attention and compressed attention has a lot of value. It is why we see so much context being able to fit in such a small size.
Speaker A
It is why we see decode speed, or tokens generated per second, staying consistent even at large context windows.
Speaker A
If we draw it out, here's my not-straight chart. But, this is context window growing, and this is your token speed slowly dropping as the context window increases.
Speaker A
It has to run calculations on a lot more data. When you can run those same calculations on a consistent, compressed, sparse attention block, that speed stays persistent. So, now your token generation stays right there the whole time.
Speaker A
Now, this got me thinking, can I somehow increase the size of the attention window to reduce hallucinations?
Speaker A
You might be thinking, "Eric, that's nonsense." It turns out it's not. SG Lang has this wonderful little flag for indexer budget that works with Qwen sparse attention.
Speaker A
Now, the default attention window is this right here. Effectively, for each Qwen sparse attention layer, it is 2,048 tokens.
Speaker A
And I'll go back to the model card to show you, but this is a Qwen sparse attention layer.
Speaker A
By passing this command, I have doubled it so that each layer is now 4,096 tokens.
Speaker A
Now, I want to be upfront about something. I have not thoroughly tested this.
Speaker A
But 6% is not a lot, especially considering how fast the model is. Easily seeing speeds over 200 tokens per second.
Speaker A
And now, armed with this bigger attention span, we come up here, and we see it did not hallucinate. It did not lie to me. It did not make up an answer.
Speaker A
It straight-up admits the value does not appear in the logs. And it tells me exactly where it is.
Speaker A
This link right here, if you go to it, has the leaked key. And this was a second run confirming what the first one also did.
Speaker A
Bam! Evidence not found. Along with the correct pointer to the GitHub commit. And just like that, I'm excited again.
Speaker A
And I also need to go back and see if DeepSeek-V4-Flash has any similar mechanism because that would also make it a lot more usable for me and many other people if it has a bigger attention span.
Speaker A
If you're still here, thank you so much for watching. I thought it was going to be a nice chill week where I could catch up, but no, we get this model and GLM-5.3-Flash dropped on the same day.
Speaker A
So much to do, so little time. I need to come up with a new benchmark to put models through their paces. If you have any ideas, drop some comments below, and I'll see you in the next one.
Topics:Qwen 3.8 Flash Nextsparse attentionN-gramhallucinationsAI coding modelscybersecurity AImodel benchmarksDeepSeek V4 Flashattention windowAI model optimization
