Speaker A
This new 35B AI model runs on just 3 GB of memory. Do you really think a 35B parameter AI model needs a giant computer? Because someone just ran one on a phone. It used about 3 GB of
Discover MiniCPM-V 4.7, a 35B parameter AI model running on just 3GB memory on phones, with expert scheduling and open-source access.
Key Takeaways
- A 35B parameter AI model can run on a phone with only 3 GB of memory using mixture of experts and expert scheduling.
- OpenBMB's MiniCPM-V 4.7 is open-source, multi-modal, and optimized for real device performance without sacrificing capability.
- Look-ahead expert scheduling (LES) significantly speeds up inference by preloading needed experts ahead of time.
- Memory usage during inference is low, but expert weights reside in storage and are loaded dynamically.
- Community resources and tutorials help users adopt and effectively use this advanced AI model.
What the video covers
- MiniCPM-V 4.7 is a vision-language AI model with 35 billion parameters that runs efficiently on devices using only 3 GB of memory.
- Developed by OpenBMB, an open-source large model community, the model supports iOS, Android, and Harmony OS with broad chip compatibility.
- The 35B-A3B version uses a mixture of experts architecture, activating only a subset of experts per token to save memory and computation.
- Look-ahead expert scheduling (LES) preloads expert weights to speed up inference, achieving over 13 tokens per second on an iPhone 17 Pro Max.
- Additional optimizations include fine-grained layer-wise quantization and DS park speculative sampling for faster text generation.
- The 3 GB figure refers to memory usage during inference; expert weights are stored on device storage and loaded as needed.
- MiniCPM-V 4.7 supports multi-modal inputs including images, documents, videos, and can perform OCR, PDF parsing, and multi-image chat.
- OpenBMB provides multiple model sizes and quantized versions, with code and weights under Apache 2.0 license available on GitHub and Hugging Face.
- Self-replay fine-tuning (SRFT) reduces catastrophic forgetting by mixing generated answers into training data, preserving task performance.
- The video encourages beginners to start with the 1B model, try live demos, and join communities like AI Profit Boardroom and AI Success Lab for learning.
Chapters
- 00:00Introduction to MiniCPM-V 4.7 and memory efficiency
- 00:45About OpenBMB and their open-source AI models
- 01:24MiniCPM-V 4.7 overview and vision-language capabilities
- 02:08Mixture of experts model explained
- 02:53Look-ahead expert scheduling (LES) for speed optimization
- 03:38AI Profit Boardroom and learning resources
- 04:02Additional optimizations and memory vs storage distinction
- 04:50GitHub repo, platform support, and usage options
- 05:38Model capabilities: reading signs, planning, and reasoning
- 06:28Document and visual data processing guides
Full Transcript — Download SRT & Markdown
Speaker A
memory, not a server, not a gaming PC. So how is that even possible? The answer is smarter than you'd guess. I'm the digital avatar of Julian Goldie, and I help people learn AI tools and actually use them in their work. Today I'm
Speaker A
breaking down MiniCPM-V 4.7 from a team called OpenBMB. By the end, you'll know how this model works, which version to try first, and the one catch with that 3 GB number that's easy to miss. First, who is OpenBMB? Their
Speaker A
website calls them an open-source large model community. That just means they share their AI models openly, so anyone can use them. Their tagline is Big Model for Everyone. They build a few families of models. MiniCPM is the one we care
Speaker A
about today. They have AgentCPM, VoxCPM, and UltraRAG. Their site says they make efficient models that run everywhere from the cloud all the way down to your device. The code page for MiniCPM-V says it was developed by two groups, THUNLP
Speaker A
and ModelBest. The code and model weights use the Apache 2.0 license, a well-known open-source license.
Speaker A
MiniCPM-V 4.7 was released on October 12th, 2026, and it's a vision language model. That means it can look at images, several images at once, documents, and video, and then talk about what it sees.
Speaker A
It comes in two sizes. The big one is called 35B-A3B. The small one is 1B. The 35B-A3B version is the first mixture of experts model in the MiniCPM-V series. So what is a mixture of experts model? Picture a
Speaker A
big team of specialists. The model holds all of them, but for each token, only a few get picked to do the work. So, it has 35 billion parameters in total, but it only uses a small slice for
Speaker A
each token. OpenBMB also says it's the first open-source mixture of experts model to run smoothly on a device without giving up performance. That's their claim. So, keep that in mind. The headline number is more than 13 tokens a
Speaker A
second with just 3 GB of memory on an iPhone 17 Pro Max. A token is just a small chunk of text. Okay, back to the model. OpenBMB shared the news on X. So, how do they pull it off? A phone can't
Speaker A
hold every expert in memory at once. The official docs say the weights can sit in storage, get loaded when needed, and then be released. But, if you keep loading over and over, the model gets too slow for real-time use. Their fix is
Speaker A
called look-ahead expert scheduling, or LES. Think of a good chef. They don't wait until the last second to grab ingredients. They look ahead and get ready early. LES does the same. While one layer of the model is working, it
Speaker A
predicts which experts the next layer will need and starts loading them early. Those predictions only guide the loading. The model still follows its normal routing. If a prediction misses, the missing expert is simply loaded and used as usual. OpenBMB says this gave a
Speaker A
five-time speedup on real devices without giving up model capability. Quick pause before I show you the next part. If you want to learn tools like this properly, we've got a 30-day roadmap inside the AI Profit Boardroom.
Speaker A
It's built to take you through new AI tools one step at a time, so you're never guessing what to do next. There are also four coaching calls every week.
Speaker A
So, if you get stuck on a tool like MiniCPM V, you can bring your question to a live call and work through it.
Speaker A
That's how you go from hearing about a tool to actually using it. Link in the description or go to AIProfitBoardroom.com. They add two more tricks.
Speaker A
Fine-grained layer-wise quantization shrinks the expert weights so they're easier to move. DS park speculative sampling helps the model produce text faster. Now, the catch I promised, that 3 GB is memory, not storage. The expert weights still have to live somewhere on
Speaker A
the device, and the 13 tokens a second is OpenBMB's number for an iPhone 17 Pro Max. Now, let's look at Hugging Face.
Speaker A
OpenBMB put everything in one collection with 13 models inside. Besides the main 35B-A3B and 1B, there are quantized versions like FP8, INT4, AWQ, GPTQ, and GGUF plus a DS park draft model for each size.
Speaker A
Next, the GitHub repo, which has over 26,000 stars. MiniCPM-V 4.7 supports iOS, Android, and Harmony OS plus day zero support for chips from Intel, AMD, ARM, Rockchip, and Humo. You can run it with SG Lang, VLLM, llama.cpp,
Speaker A
or Ollama. And there's a live demo and a cookbook. Let's look at what the X post says this model can do. The first two skills on the list are reading real-world signs and planning around constraints. The post doesn't show the
Speaker A
exact signs or plans, so I won't invent an example. You give it an image and a question, and it answers in text. Reading a sign means pulling the words out of a photo. Planning around constraints means taking what it read plus the limits you
Speaker A
set and working toward a sensible answer. The big news is where it runs. OpenBMB's GitHub page shares clips of the 35B-A3B model on an iPhone 17 Pro Max and an iPad M4 and the 1B model on an iPhone 17
Speaker A
Pro Max. They say these are unedited screen recordings captured on device. If you want to try this yourself, start with the 1B version. The X post calls it the lightweight option. The other two skills on the list are passing complex
Speaker A
documents and reasoning over visual data. Again, the post doesn't show a specific file. So, here's what the docs and cookbook give you. For documents, the cookbook has guides for OCR, which means reading text from images, and for PDF parsing. For visual data, there are
Speaker A
guides for multi-image chat, video understanding, and visual grounding. The docs also show two real sample prompts.
Speaker A
The image example asks, "What causes this phenomenon?" The video example asks the model to describe the video in detail, focusing on on-screen text, interface changes, main actions, and scene changes. There's one setting that matters here. The 16X mode merges visual
Speaker A
tokens to be efficient. The 4X mode keeps four times as many for finer detail. So, for document work, start with the OCR or PDF guide and test it on a file of your own. One more piece is self-replay fine-tuning or SRFT.
Speaker A
Fine-tuning a model on one narrow job often makes it forget things it knew. With SRFT, the model writes its own answers to general prompts. They keep the good ones and mix them into the training data. They say it cut
Speaker A
forgetting by more than 80% across four models and two tasks, while keeping essentially all of the task gain. Okay, pro tips for beginners. Tip one, try the live demo on Hugging Face before you download anything. Tip two, start with
Speaker A
the 1B version, which OpenBMB says beats Quen 3.50.8B in overall score and inference efficiency. Tip three, use the 4X setting when you need finer detail. Tip four, test it on your own files since the charts are OpenBMB's own. So,
Speaker A
that's MiniCPM V4.7, a 35B parameter model on a phone. Start small, test it on your own work, and keep learning as the tools improve. Right now, there's a window where most business owners haven't figured out tools like this yet,
Speaker A
and that window won't stay open forever. The goal is to get ahead on learning them. Inside the AI Profit Boardroom, you get daily tutorials, so there's always something new to learn. You get four coaching calls every week where you
Speaker A
can ask questions live. You get a prompt library, so you're not starting from a blank page, and you get the member map, so you can connect with other members.
Speaker A
Link in the description or go to aiprofitboardroom.com. If you want the full process, the SOPs, and over 100 AI use cases like this one, join the AI Success Lab. It's our free community. Links in the comments and description. You'll get all the video
Speaker A
notes from this episode, plus access to over 93,000 members. The AI Success Lab is a community where people learn AI tools together. It's free to join. The video notes give you the main points from today's video, so you can come back
Speaker A
to them anytime.
Topics:MiniCPM-V 4.7OpenBMBmixture of expertsAI model on phone35B parameterslook-ahead expert schedulingvision language modelfine-grained quantizationself-replay fine-tuningAI Profit Boardroom
