WorkBuddy内置的本地模型来了WorkBuddy’s built-in local model is here!


AI free token/AI免费积分

本文来源:https://mp.weixin.qq.com/s/GvMIaiBSTzHRrFc0gSVU7Q
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

如果你刚好 需要安装个能力过得去的本地模型,来做隐私要求高的工作。
又刚好 在用表现不错的电脑,例如一台内存达到32G的mac。
还刚好 在用WorkBuddy。
那么WorkBuddy内置的本地模型值得一试。
图片
在「设置」—「模型」里面,可以看到模型列表下增加了一个「本地模型」,一键下载即可在WorkBuddy使用,不需要借助ollama或者llama.cpp之类的来承载。
这个Qwen3.6 35B-3AB是个MoE模型,35B参数只激活3B参数,提高本地的推理速度。同时,腾讯对模型开展了裁剪优化,大小只有15GB,比ollama上面的24GB版本少了1/3,这让它在WorkBuddy里面使用更加顺畅。
图片
实测模型速度观感还可以,生成速度稳定在 30–35 token/s,运行时候占用内存大概17G。首字延迟会比较高,主要是WorkBuddy平台提示和工具列表加载等很大(一般可以达到几万token),关闭掉不常用的工具可以改善。
和云端比,它当然没那么快、上下文也不如线上模型稳;但作为本机、不出网、零按量计费的思路,这个速度已经能日常用了,性价比极高。
图片

Source: https://mp.weixin.qq.com/s/GvMIaiBSTzHRrFc0gSVU7Q If you have any copyright questions, please write to: 13951431913#139.com or submit a comment on the right.

If you happen to need to install a reasonably capable local model for high-privacy work,

and you happen to be using a performing computer, like a mac with 32GB of RAM,

and you happen to be using WorkBuddy,

then the local model built into WorkBuddy is worth a try.

图片

In "Settings" — "Model", you can see a "Local Model" added to the model list, which can be downloaded and used in WorkBuddy with one click, without needing to use ollama or llama.cpp to host it.

This Qwen3.6 35B-3AB is an MoE model, 35B parameters activating only 3B parameters, improving local inference speed. At the same time, Tencent performed pruning optimization on the model, making it only 15GB in size, 1/3 less than the 24GB version on ollama, which makes it run smoother in WorkBuddy.

图片

The model speed feels decent in practical use, with a stable generation rate of 30–35 tokens/s, and a memory usage of about 17GB during operation. The first character latency is relatively high, mainly because the WorkBuddy platform prompt and tool list loading are very large (often reaching tens of thousands of tokens), turning off unused tools can improve this.

Compared to the cloud, it is certainly not as fast, and the context is not as stable as online models; but as a local, non-internet, zero-metering solution, its speed is already usable for daily tasks, offering excellent cost-performance.

图片

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry