
笔者正踏上一段此前一直犹豫不决的旅程:真正开始日常使用 AI。把最私密的数据交给云服务,一直是笔者最大的心理障碍,但如今在本机运行强大模型已越来越可行,可以亲自看看它们到底能做什么。这项技术真的有用吗?值得为一台电脑或者一台塞满昂贵内存的笔记本砸下大把钞票吗?笔者将试着帮你找到答案。笔者不是 AI 专家,这对我来说也将是一段学习之旅,但也许我能帮你判断这段旅程值不值得走——而我会一路把自己的实践过程记录下来。
苹果为新款 Mac 台式机宣传的一大卖点就是本地 AI 表现有多出色,而搭载最高 128GB 内存、面向智能体 AI 的全新 RTX Spark Windows 机型也即将上市。本地 AI 和智能体,现在就是这么火。但说真的,想到有一个只活在笔者桌上那台机器里、只听命于笔者、而不受 OpenAI、谷歌、微软或 Anthropic 这些「巨头」摆布的小跑腿,这种感觉跟信任云服务或订阅聊天机器人完全不同。别误会,笔者依然绝不会让它替我写作、替我编辑,或者生成难看的传单。但如今有了隐私这层保障,也许笔者终于能想出一些有用的「杂活」来「托付」给它。
作为对 M5 Ultra Mac Studio 持续测试的一部分,同时也在其他 Mac 和 Windows 系统上做实验,笔者安装了一些本地大语言模型并开始折腾。以下是目前的进展。笔者首先在 Mac Studio 上安装了开源的 Hermes Agent。Hermes 是一款自托管的 AI 智能体桌面应用,支持 macOS、Windows 和 Linux,如果坚持使用本地模型就完全免费。由于这台 M5 Ultra Mac Studio 拥有 256GB 统一内存,笔者几乎可以运行任何想要的模型。这也正是事情很快变得令人应接不暇的地方。
市面上的模型多得数不清,有些还有专门用途。一般来说,参数量达到数百亿的模型,能做的事情比只有几十亿参数的小模型更多。既然本地运行不必担心为 token 付费,笔者决定一开始就上大模型。通过 Hermes 简洁的引导界面和模型选择器,笔者选择了 Qwen 3.8 Flash Next——一个约 1250 亿参数、体积约 105GB 的模型。笔者还计划在 M6 Mac Mini、M5 MacBook Air、搭载 AMD Strix Halo 处理器的华硕 TUF Gaming A14,以及最终的 RTX Spark 上,花些时间体验更小的 Qwen 模型。
让 Hermes 和 Qwen 跑起来并没花太久,笔者甚至通过 Telegram 机器人实现了用手机操控它们。但随后,笔者又遇到了使用 AI 时那个老对手:输入框。「我该拿这东西做什么?」
为了先热热身,笔者遵循 YouTube 上的一些实用建议,让 Hermes 做一份每日晨报——先从一个 cron 定时任务开始,确保一切正常运转。它会扫描笔者的邮箱和日历,告知任何值得立即处理的事项,并附上一份简短的天气报告。它并不那么实用,而且你显然不需要一台 1.2 万美元的电脑来搭这个,但这是笔者可以逐步完善的东西。起初晨报总是生成失败,后来笔者终于弄明白:定时任务早上 7:30 触发时,macOS 不能处于睡眠状态。现在它已经像钟表一样准时运行,但笔者还得给它想出更多可以抓取的信息,让结果更有用。
一项更有用的任务是重新整理笔者的 Steam 游戏库。笔者在 Steam 上有超过 400 款游戏,最近开始给它们分类,以便心血来潮想玩新游戏时能快速找到。Steam 不会替你给游戏分类,你只能手动一款一款地做。而这正是一个智能体「跑腿」可以大显身手的地方。笔者问它:「你能重新整理我的 Steam 游戏库吗?」Hermes 通过查看已安装的 Steam 客户端就能看到笔者的大部分游戏,并很快给出了几种整理方案。笔者决定让它按类型整理,但保留笔者自己设置的几个专属分类(收藏、合作游戏、派对游戏、和妻子一起玩的游戏等)。
要让本地 AI 替你行动,你得给它一些本机权限。在游戏库这个案例中,笔者必须注册一个 Steam Web API 密钥并交给 Hermes。一旦获得访问权限,它只花了几分钟就把所有游戏分门别类整理完毕。而且事后撤销这个 API 密钥也很容易,因为笔者不再需要它。笔者的想法是让它替我把这项重活做一次,之后新买的游戏笔者自己归档就好。总体来说它干得不错,现在笔者的 Steam 库浏览起来方便多了。
笔者做的另一件简单的事是数据分析:让 Hermes 查看财务记录帮笔者算一些数字,还让它给笔者做了一份新笔记本的规格对比表格。这些都是小事,但都是笔者不会、也不愿托付给任何云服务的任务——笔者不想把这些敏感数据上传到云端,而新笔记本的信息还在 embargo 期内。把它们留在笔者自己的电脑上,正是笔者用或不用这个工具的分水岭。
笔者目前正在进行的一项更大的自动化工程,是写一个脚本来运行一系列笔记本基准测试。笔者所有的基准测试都是手动跑的,而守着一台笔记本跑十来项测试、还要重复三次取平均值,极其耗时。笔者希望能至少把其中一部分自动化。即便是一步步带着 Hermes 走完所有流程、让它吐出「氛围编程」出来的 Python 脚本,本身也是一项不小的工程,目前仍在进行中。但笔者希望能取得一些积极进展。
笔者与 Hermes 的磨合仍处于早期阶段,希望这些只是它能替笔者完成的有用工作的冰山一角。但同样清楚的是,即便是这些强大的本地模型,也无法变成无所不能的魔法助手。事情依然不总能如预期般运作——笔者的每日晨报已经坏过好几次了。
笔者把 Hermes 和本地 AI 仅仅视为一种工具,一个跑在笔者电脑上的程序。笔者不想要一个数字朋友。笔者不会对它说"谢谢"或"请",改变主意时也不会打"抱歉"。在使唤它的时候,笔者会像用鼠标点操作系统一样冷酷而淡然。在继续测试和学习本地 AI 的过程中,笔者希望自己能始终保持警觉与谨慎。让我们看看这条路会把我带向何方。
本文来源:https://www.theverge.com/ai-artificial-intelligence/1008484/learning-local-ai-hermes-qwen-model-mac-studio-testing-diary-progress
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
A big part of Apple’s pitch for its new Mac desktops is how good they are for local AI, and there’s an entire line of new RTX Spark Windows machines landing imminently with up to 128GB of RAM aimed at using agentic AI. Local AI and agents are just, like, so hot right now. But, in all seriousness, the thought of having a little gofer living solely in the box on my desk that answers only to me and not overlords from OpenAI, Google, Microsoft, or Anthropic hits a bit different than trusting cloud services or subscribing to a chatbot. Don’t get me wrong, there’s still no way I’m gonna have it write for me, edit for me, or generate ugly fliers. But now, with some privacy on my side maybe I can finally think of useful busy work to “trust” it with.
So as part of my ongoing testing of the M5 Ultra Mac Studio, as well as experimenting on other Mac and Windows systems, I’ve installed some local LLMs and I’m beginning to tinker. Here’s how things are going.
Next1/4Hermes Agent can be used with cloud or local models.I started by installing the open-source Hermes Agent on the Mac Studio. Hermes is a self-hosted AI agent desktop app that works on macOS, Windows, and Linux. It’s completely free to use if you stick to local LLMs. Since this M5 Ultra Mac Studio has 256GB of unified memory, I could run just about any model I want. This is how things quickly get overwhelming. There are more models out there than I can count, and some have specialized uses. Generally, models with many billions of parameters are able to do more than smaller ones with only a few billion. And since we don’t have to worry about paying for tokens when running a model locally, I figured I’d go big to start.
Using Hermes’ straightforward onboarding interface and model picker, I chose the Qwen 3.8 Flash Next model — a 125-billion parameter model that’s around 105GB in size. I also plan on spending some time with much smaller Qwen models on devices like an M6 Mac Mini, M5 MacBook Air, an Asus TUF Gaming A14 with AMD Strix Halo processor, and, eventually, RTX Spark. (Care to share what local models and hardware you use? You know where to leave your comments. 🫡)
It wasn’t long to get up and running with Hermes and Qwen, and even make them controllable via my phone by using a Telegram bot. But then, I was presented with my usual nemesis when it comes to using AI: the text box.
“What do I do with this thing?”
First up, to get things going, I followed some helpful YouTube advice and had Hermes make me a daily morning briefing — to just start small with a cron job and ensure things are working. It scans my email and calendar, tells me anything worth my immediate attention, and gives me a short weather report. It’s not all that useful, and you obviously don’t need a $12,000 computer to build it, but it’s something I can develop over time. The brief kept failing to generate at first, but then I eventually figured out that macOS couldn’t be asleep when the job fired up at 7:30am. Duh! It works like clockwork now, but I have to think of more info for it to pull for me to make the results more helpful.
A more useful task was reorganizing my Steam library. I have over 400 games on Steam, and I recently started categorizing them for when I want to play something new on a whim. Steam doesn’t categorize your games for you, so you instead have to do it manually, one-by-one. But here’s where an agentic lackey could fit right in. I asked, “Can you reorganize my Steam library?” Hermes could see most of my games by just viewing the already installed Steam client. It quickly came up with some organization options for me. I decided to have it organize my games by genre, but leave some of my own specialized categories(favorites, co-op titles, party games, games to play with my wife, etc.).

You have to give a local AI some permissions over your machine for it to take actions for you, and in the case of my game library I had to register a Steam web API key and give it to Hermes. Once it had that access, it only took a few minutes for it to churn through sorting all the games into categories. And the API key was easy to revoke right after, since I don’t need it further. The idea is for it to just do this heavy lifting for me once, and then I’ll file new games into the categories as I eventually buy them. It did a fine job overall, and now my Steam library is easily browsable.
Next1/2Starting the process of having Hermes organize my Steam library.Another simple thing I did was data analysis. I had Hermes take a look at financial records to crunch some numbers for me, and I also had it make me a spec comparison spreadsheet for a new laptop. These are small things, but they’re both tasks I cannot or will not trust with any cloud service — I don’t want to upload that sensitive data to the cloud, and in the case of a new laptop, that info was under embargo. Keeping them on my computer was the difference in me using this tool or not.
The bigger bit of automation I’m currently working on is a script to run a series of laptop benchmark tests. I run all our benchmarks manually, and tending to a laptop to run a dozen or so tests — three times over, to take the average — is incredibly time-consuming. I’d love to automate at least some of it. Even walking Hermes through all of the required procedures and getting it to spit out vibe coded Python scripts has been a bit of an endeavor, and it remains a work in progress. But I’m hoping to make some positive headway.
It’s still early days of me working with Hermes, and I’m hoping this is just the tip of the iceberg on the useful things I get it to do for me. But it’s also clear that even these powerful local models don’t make for a magical assistant that can do anything. Things still don’t always work as intended — my daily briefing already broke, multiple times.
I view Hermes and local AI as just a tool, a computer program to run on my computer. I don’t want a digital friend. I’m not saying “thank you” or “please,” or typing “sorry” when I change my mind on something. I’m remaining as stone-cold and indifferent when ordering it around as I am clicking a mouse around an operating system.
I hope to remain vigilant and cautious while I continue testing and learning with local AI. Let’s see where it takes me.