DeepSeek-V4-Flash 本地部署(相对)低成本方案终于来了
Unsloth 的GGUF 量化版已就位,还顺手把 llama.cpp 里那个让 V4 说胡话的 bug 给修了
给本地部署的大模型装上手脚,Unsloth 把MCP 这条路打通了
谷歌最新大模型 DiffusionGemma 续集:Unsloth 把它压到 18GB,单卡飙到 2000+ Token/s
这篇我就带大家把 DeepSeek-V4-Flash 从里到外捋一遍:它牛在哪、跑分什么水平、想在自己机器上跑起来要注意哪些坑
简介
vLLM 0.22大版本升级,DeepSeek V4 生产级优化,KV Cache 极致压缩
先说清楚 DeepSeek-V4 这一代到底是什么
它是一个 MoE(混合专家) 系列,一口气给了两个模型:
-
DeepSeek-V4-Pro:1.6T 总参数,激活 49B,冲的是开源天花板 -
DeepSeek-V4-Flash:284B 总参数,激活只有 13B,主打高效轻量
注意这个激活参数,Flash 每次前向只点亮 13B,这是它能被普通玩家惦记着本地部署的根本原因,两个模型都原生支持 100 万 token 的上下文长度
按官方技术报告(arxiv 编号 2606.19348)的说法,V4 这一代最关键的是三处架构与训练上的升级:
-
混合注意力架构:把 Compressed Sparse Attention(CSA,压缩稀疏注意力)和 Heavily Compressed Attention(HCA,重压缩注意力)揉到一起,专治长上下文的算力爆炸——在 100 万 token 场景下,V4-Pro 单 token 推理的 FLOPs 只要 V3.2 的 **27%**,KV cache 更是砍到只剩 10% -
Manifold-Constrained Hyper-Connections(mHC,流形约束超连接):给传统的残差连接上了个强化 buff,让信号在深层网络里传得更稳,同时不牺牲模型的表达能力 -
Muon 优化器:换掉老优化器,收敛更快、训练更稳
一张图看懂这三大升级到底强在哪:
一句话总结这套设计的野心:用更少的算力,把上下文喂到百万级,还要保住聪明程度
在 代码类硬指标上,V4-Pro-Max 几乎是屠榜的存在,LiveCodeBench、Codeforces、Apex Shortlist 全部登顶,一个开源模型能把编程能力做到这个份上,DeepSeek 这波属实是给国产开源长脸了
当然要诚实:在 SimpleQA、GPQA Diamond、HLE 这些纯知识和超难推理题上,Gemini-3.1-Pro 依然领先,V4 没有全面碾压,但作为开源模型,能站在这张牌桌上已经很了不起
三种思考模式
V4-Pro 和 V4-Flash 都内置了三档「思考强度」,这点挺人性化的,任务简单就别浪费算力空想
|
|
|
|
|---|---|---|
| Non-think |
|
|
| Think High |
|
|
| Think Max |
|
|
一个特别有意思的现象是:Flash 只要给够思考预算,推理能力能追平 Pro,官方原话是 Flash-Max 在给到更大 thinking budget 时,推理表现能逼近 Pro 版,只是参数规模摆在那,纯知识题和最复杂的 Agent 任务上会稍逊一筹
这意味着什么?小模型 + 多思考,能在很多推理任务上顶上大模型,对本地部署党简直是福音
更实在的是,这三档不是摆设,在 llama.cpp 里加个参数就能现切,默认开的是 Think High:
# 拉满推理,硬骨头就上这个
--chat-template-kwargs '{"reasoning_effort":"max"}'
# 常规推理
--chat-template-kwargs '{"reasoning_effort":"high"}'
# 彻底关掉思考,退回 Non-think
--chat-template-kwargs '{"enable_thinking":false}'
新版 llama.cpp 还给了更省事的 --reasoning on / --reasoning off 一键开关,嫌命令行麻烦的直接去 Unsloth Studio,右上角下拉菜单点一下就换档,比记参数舒服
思考预算的魔力:Flash 三档实测
前面说 Flash 靠思考预算能追平 Pro,这可不是嘴上说说,官方给了三档模式的完整成绩单,我挑几个最能说明问题的拎出来(均为官方 benchmark,指标为 Pass@1 / Rating / EM 等):
|
|
|
|
|
|
|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
88.1 |
|
|
|
|
|
94.8 |
|
|
|
|
|
91.6 |
|
|
|
|
|
85.7 |
|
|
|
|
|
|
|
看这几行数据我是真被震到了:同一个 Flash,光靠打开思考,HMMT 数学从 40.8 直接飙到 94.8,Apex Shortlist 从 9.3 干到 85.7,这已经不是量变是脱胎换骨
GPQA Diamond 更离谱,Flash-Max 的 88.1 贴脸 Pro-Max 的 90.1,用 13B 激活硬刚 49B,靠的就是把思考预算给足
一句话:Non-think 是它的下限,Think Max 才是它的真身,本地部署时舍得给上下文和算力,Flash 能还你一个惊喜
为什么要用 Unsloth 的 GGUF 版
模型再强,跑不起来也白搭,这就轮到 Unsloth 登场了
Unsloth API 来了,24GB 显存就能在 Claude Code、Codex、OpenClaw 里跑本地 Agent

DeepSeek 原始权重是 FP4 + FP8 混合精度(MoE 专家用 FP4,其余大部分用 FP8),想在自己的卡上跑,最省心的路子还是 GGUF + llama.cpp,Unsloth 干的就是这个活,而且干得漂亮:
-
修好了 llama.cpp 的乱码 bug:原版 llama.cpp 跑 V4,第二轮对话之后就开始胡言乱语(gibberish),Unsloth 定位并修复了这个问题,前提是你得用 llama.cpp 的 PR #25402 -
重做了 chat 模板:改进了 V4 的 chat jinja 模板,拿 4000 多组对话逐一验证,确认和官方 baseline 表现一致,这测试量相当扎实 -
Dynamic 2.0 动态量化:Unsloth 自家的 Dynamic 2.0 量化方案,精度号称超越其他主流量化 -
Q8 几乎无损,还不占地方:想要全精度无损效果就上 Q8(UD-Q8_K_XL),体积 162GB,关键是它只比 Q4(UD-Q4_K_XL)大了区区 7GB,这个性价比,能上 Q8 就别犹豫
Unsloth 官方放了几张量化对比图,我挑两张最能说明问题的。第一张是质量与体积的帕累托前沿,Unsloth 的 Q8、Q4 稳稳压在最优线上,同体积下精度更高:
第二张是逐层量化误差,Unsloth 的 MXFP4 专家权重做到了全程零误差,普通 Q4_K 则是每个权重都得四舍五入(约 5.2% RMSE),精度差距肉眼可见:
这里插一句:Unsloth 现在几乎是开源模型本地化的「售后保障」,从 gpt-oss、Qwen3、Llama 4 到 Gemma,哪个新模型出 bug 都能看到他们的修复补丁,这种生态位太重要了
本地怎么跑起来
给一套官方文档里的完整编译流程,核心就是必须切到 PR #25402 那个分支,否则前面说的乱码问题就会找上门
先用一张图把六步流程串起来,再看命令:
apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git fetch origin pull/25402/head:deepseek-v4-checkpointing-fix
git checkout deepseek-v4-checkpointing-fix
cd ..
cmake llama.cpp -B llama.cpp/build \
-DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first \
--target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp
这个 PR #25402 修的其实是 prompt caching(提示词缓存):在它之前,你得强制 --ctx-checkpoints 0 关掉上下文检查点,否则 V4-Flash 的缓存机制一启用就吐乱码,修复后把 V4 重新归类、处理好尾部 token 的重复问题,缓存这才能正常用起来
采样参数官方也给了建议:
-
常规使用: temperature = 1.0,top_p = 1.0 -
用 Think Max 模式:上下文窗口至少拉到 384K,把推理空间给足
关于 chat 模板要多提一嘴:这次发布没有带 jinja 格式的模板,官方改成提供一个 encoding 文件夹,里面是 Python 脚本和测试用例,教你怎么把 OpenAI 兼容格式的消息编码成模型输入,以及怎么解析输出,接入的时候记得去读那个文件夹的文档
如果嫌命令行麻烦,还有个更友好的选择——Unsloth Studio
这是个能本地跑和训模型的 Web UI,V4-Flash 可以直接在里面跑,还带 High 和 Max thinking 的开关,点一下就切换思考强度
安装也是一行命令的事:
# macOS / Linux / WSL
curl -fsSL https://unsloth.ai/install.sh | sh
# 启动,端口自己定
unsloth studio -p 8888
Windows 用户把命令换成 irm https://unsloth⋅ai/install.ps1 | iex 即可,同一条命令还能用来更新
实测建议与真心话
老实交代,这么大的模型我暂时没在本地完整拉起来实测,162GB 的 Q8 对显存的要求摆在那,Flash 版即便激活只有 13B,完整权重加载依然是重量级选手,这一点大家心里要有数、
但基于对 DeepSeek 历代和 Unsloth 生态的了解,我给几个务实的判断:
-
想尝鲜又缺卡:先去 Unsloth Studio 里试 Flash 版,配合思考模式开关体验推理能力,比硬啃命令行舒服 -
做长文档、长代码库分析:百万上下文 + 长上下文效率优化是 V4 最大的差异化卖点,这类场景值得重点押注 -
在意代码能力:Pro-Max 的编程跑分是真的猛,有条件上 Pro 的,代码任务体验会明显不一样 -
一定要用对分支:再强调一次,llama.cpp 务必切到 PR #25402,不然乱码 bug 会让你怀疑人生
DeepSeek-V4 这一代,把「百万上下文」从实验室名词变成了能落地的开源现实,而 Unsloth 帮我们把落地的门槛又往下砸了一截,国产开源这条路,越走越有底气了
DeepSeek-V4-Flash Local Deployment (Relatively) Low-Cost Solution Is Finally Here
Unsloth's GGUF quantized version is ready, and it also fixed the bug in llama.cpp that made V4 talk nonsense
Giving locally deployed LLMs hands and feet, Unsloth has opened up the MCP path
Google's latest large model DiffusionGemma sequel: Unsloth compressed it to 18GB, hitting 2000+ Token/s on a single GPU
In this article, I'll walk you through DeepSeek-V4-Flash inside and out: where it excels, what its benchmark scores look like, and what pitfalls to watch out for if you want to run it on your own machine
Introduction
DeepSeek's latest open-source V4 accelerated version, taking off immediately
vLLM 0.22 major upgrade, DeepSeek V4 production-grade optimization, extreme KV Cache compression
First, let's clarify what exactly the DeepSeek-V4 generation is.
It's a MoE (Mixture of Experts) series, offering two models at once:
DeepSeek-V4-Pro: 1.6T total parameters, 49B active, aiming for the open-source ceiling
DeepSeek-V4-Flash: 284B total parameters, only 13B active, focused on efficiency and lightweight performance
Note this active parameter count: Flash only lights up 13B per forward pass, which is the fundamental reason ordinary users can consider local deployment. Both models natively support a 1 million token context length
According to the official technical report (arXiv ID 2606.19348), the most critical changes in the V4 generation are three architectural and training upgrades:
Hybrid attention architecture: It fuses Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) to specifically tackle the compute explosion of long contexts—in the 1 million token scenario, V4-Pro's FLOPs for single-token inference are only **27%** of V3.2's, and KV cache is cut down to just 10%
Manifold-Constrained Hyper-Connections (mHC): It adds a strengthened buff to traditional residual connections, letting signals propagate more stably through deep networks without sacrificing model expressiveness
Muon optimizer: Replacing the old optimizer, it converges faster and trains more stably
One diagram shows exactly where these three upgrades excel:
One sentence summarizing the ambition behind this design: Use less compute to feed context into the millions while keeping intelligence intact
On hard coding benchmarks, V4-Pro-Max almost tops the charts: LiveCodeBench, Codeforces, and Apex Shortlist all take first place. An open-source model reaching this level of coding ability is a real win for domestic open source from DeepSeek
To be honest: on pure-knowledge and extremely hard reasoning benchmarks like SimpleQA, GPQA Diamond, and HLE, Gemini-3.1-Pro still leads, and V4 doesn't completely dominate everything. But as an open-source model, standing at this table is already remarkable
Three thinking modes
Both V4-Pro and V4-Flash come with three levels of "thinking intensity," which is quite user-friendly. For simple tasks, don't waste compute on idle deliberation
Mode
Features
Suitable scenarios
Fast, intuitive answers
Daily trivialities, low-risk decisions
Deliberate logical reasoning, slower but more accurate
Complex problems, solution planning
Push reasoning ability to the limit
The tough-nut problem of exploring the boundaries of model intelligence
A particularly interesting phenomenon is: as long as Flash is given enough thinking budget, its reasoning ability can catch up with Pro. The official statement is that when Flash-Max is given a larger thinking budget, its reasoning performance can approach the Pro version; it's just that the parameter scale is what it is, so it falls slightly short on pure knowledge questions and the most complex Agent tasks.
What does this mean? Small model + more thinking can stand in for large models on many reasoning tasks, which is a real blessing for self-hosting enthusiasts.
More practically, these three levels are not just for show. In llama.cpp, you can switch instantly by adding a parameter, and the default is Think High:
The new llama.cpp also provides a more convenient one-click switch with --reasoning on / --reasoning off. If you find the command line troublesome, just go to Unsloth Studio and click the drop-down menu in the top right corner to switch gears—much easier than remembering parameters.
The magic of the thinking budget: Flash's three modes tested
Earlier I said Flash can catch up with Pro thanks to the thinking budget—this isn't just talk. The official team provided a full report card for the three modes. I've picked out a few that best illustrate the point (all are official benchmarks; metrics include Pass@1 / Rating / EM, etc.):
Metric
Flash Non-think
Flash High
Flash Max
Pro Max (reference)
MMLU-Pro
83.0
86.4
86.2
87.5
GPQA Diamond
71.2
87.4
90.1
HMMT 2026 Feb
40.8
91.9
95.2
LiveCodeBench
55.2
88.4
93.5
Apex Shortlist
9.3
72.1
90.2
Terminal Bench 2.0
49.1
56.6
56.9
67.9
Looking at these rows of data, I was genuinely blown away: on the same Flash, just by turning on thinking, HMMT math jumped from 40.8 to 94.8, and Apex Shortlist went from 9.3 to 85.7. This is no longer a quantitative change—it’s a complete transformation.
GPQA Diamond is even more absurd: Flash-Max’s 88.1 is right up against Pro-Max’s 90.1, using 13B activated parameters to go toe-to-toe with 49B, purely by giving it enough thinking budget.
In one sentence: Non-think is its floor; Think Max is its true form. When deploying locally, if you’re willing to give it enough context and compute, Flash will surprise you.
Why use Unsloth’s GGUF version
No matter how strong the model is, it’s useless if you can’t run it. That’s where Unsloth comes in.
Unsloth API is here—24GB VRAM is enough to run local agents in Claude Code, Codex, and OpenClaw.
DeepSeek’s original weights use FP4 + FP8 mixed precision (MoE experts use FP4, most of the rest use FP8). To run it on your own card, the most hassle-free path is still GGUF + llama.cpp, and that’s exactly what Unsloth does—and does beautifully:
Fixed llama.cpp’s gibberish bug: the original llama.cpp running V4 starts producing gibberish after the second round of conversation. Unsloth located and fixed the issue, provided you use llama.cpp PR #25402.
Redid the chat template: improved V4’s chat jinja template and validated it against more than 4,000 conversation pairs, confirming identical behavior to the official baseline. That’s a solid amount of testing.
Dynamic 2.0 dynamic quantization: Unsloth’s own Dynamic 2.0 quantization scheme, with accuracy claimed to surpass other mainstream quantization methods.
Q8 is almost lossless and doesn’t take up much space: if you want full-precision lossless results, go with Q8 (UD-Q8_K_XL), which is 162GB. The key point is that it’s only 7GB larger than Q4 (UD-Q4_K_XL). At that value, if you can go Q8, don’t hesitate.
Unsloth officially released several quantization comparison charts. I picked the two that best illustrate the point. The first is the Pareto frontier of quality vs. size: Unsloth’s Q8 and Q4 sit firmly on the optimal line, delivering higher accuracy at the same size.
The second is per-layer quantization error: Unsloth’s MXFP4 expert weights achieve zero error throughout, while ordinary Q4_K has to round every weight (about 5.2% RMSE). The accuracy gap is visible to the naked eye.
A quick aside: Unsloth is now almost the “after-sales support” for localizing open-source models. From gpt-oss, Qwen3, Llama 4 to Gemma, whenever a new model has a bug, you can find their fix patch. This ecological niche is incredibly important.
How to run it locally
Here’s the complete compilation workflow from the official docs. The core point is that you must switch to the PR #25402 branch, or you’ll run into the gibberish problem mentioned earlier.
First, use a diagram to link the six-step process together, then look at the commands.
This PR #25402 actually fixes prompt caching: before it, you had to force-disable context checkpoints with --ctx-checkpoints 0, otherwise V4-Flash’s caching mechanism would spew gibberish the moment it was enabled. After the fix, V4 is reclassified and the duplicate tail-token problem is handled, so caching can finally be used normally.
The official docs also provide recommended sampling parameters:
Regular use: temperature = 1.0, top_p = 1.0
Use Think Max mode: pull the context window to at least 384K to give reasoning enough room.
One more note about the chat template: this release does not include a Jinja-format template. The official change is to provide an encoding folder containing Python scripts and test cases that teach you how to encode OpenAI-compatible format messages into model inputs and how to parse outputs. When integrating, be sure to read the documentation in that folder.
If you find the command line troublesome, there's a more user-friendly option — Unsloth Studio.
This is a Web UI that can run and train models locally. V4-Flash can run directly in it, and it comes with High and Max thinking toggles — click to switch thinking intensity.
Installation is just a one-line command:
Windows users can replace the command with `irm https://unsloth⋅ai/install.ps1 | iex` — the same command can also be used to update.
Practical test suggestions and honest thoughts
To be honest, I haven't pulled this huge model up fully for local testing yet. The 162GB Q8 puts a real demand on VRAM. Even though the Flash version only activates 13B, loading the full weights is still a heavyweight move — everyone should keep that in mind.
But based on my understanding of DeepSeek's generations and the Unsloth ecosystem, here are a few pragmatic judgments:
Want to try it but lack GPU: head to Unsloth Studio first and try the Flash version, using the thinking mode toggle to experience its reasoning capability — much more comfortable than slogging through the command line.
For long-document and long-codebase analysis: million-token context plus long-context efficiency optimizations is V4's biggest differentiating selling point — worth betting on for those scenarios.
If you care about coding capability: Pro-Max's programming benchmark scores are genuinely impressive. If you can get a Pro, the code-task experience will be noticeably different.
Make sure you use the right branch: once again, switch llama.cpp to PR #25402 — otherwise the garbled-text bug will make you question life.
With DeepSeek-V4, "million-token context" has gone from a lab term to a deployable open-source reality, and Unsloth has helped push the deployment barrier down even further. The domestic open-source path is gaining more and more confidence.
