IT之家 9 月 23 日消息,小米 MiMo 大模型负责人罗福莉今日发文,宣布 MiMo-V3 即将使用全新架构。其核心 HySparse 2 今日发布,带来更少的预填充、更小的 KV 缓存、更出色的长上下文检索。
罗福莉表示,智能体(Agentic)推理是一种截然不同的工作负载。每一轮交互,一个简短的动作可能会返回需要开展预填充的长文本观察结果,同时上下文还在不断增长。这使得预填充成本、KV 缓存大小和检索准确率同时处于关键路径上。
另外还有两项改进:使用 Token 级别的挑选来替代块(block)级别的挑选,并用近期 Token 的强制窗口来替代独立的 SWA 分支,从而让本地和全局 Token 共享同一个 KV 缓存。由于所有的交叉解码器 KV 缓存现在都来自自解码器,因而预填充在自解码器完成时即可停止。
值得一提的是,昨日凌晨,小米刚刚发布并开源了全新的 Xiaomi MiMo-V2.6 系列,包含 Pro 与 Flash 两个原生全模态模型。小米称这是其探索 RSI(递归自我改进)路径的关键一步,借助规模化扩展强化学习算力,让模型在持续的探索与反馈中拓展智能边界。
IT之家附论文地址如下:
小米 18 Pro 系列手机新品发布会专题
本文来源:https://www.ithome.com/1/006/502.htm
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。
IT Home News, September 23 - Luo Fuli, head of Xiaomi's MiMo large model, announced today that MiMo-V3 will soon adopt a brand-new architecture. Its core HySparse 2 was released today, bringing less pre-filling, a smaller KV cache, and better long-context retrieval.
Luo Fuli stated that Agentic reasoning is a distinctly different workload. In each round of interaction, a brief action may return long text observation results that require pre-filling, while the context continues to grow. This puts pre-fill cost, KV cache size, and retrieval accuracy all on the critical path simultaneously.
There are also two additional improvements: token-level selection replaces block-level selection, and a forced window of recent tokens replaces the independent SWA branch, allowing local and global tokens to share the same KV cache. Since all cross-decoder KV caches now come from the self-decoder, pre-filling can stop as soon as the self-decoder finishes.
It is worth mentioning that early yesterday morning, Xiaomi just released and open-sourced the brand-new Xiaomi MiMo-V2.6 series, including two native omni-modal models: Pro and Flash. Xiaomi stated that this is a key step on its path to exploring RSI (Recursive Self-Improvement), leveraging scaled-up reinforcement learning compute to let models expand the boundaries of intelligence through continuous exploration and feedback.
IT Home appends the paper link as follows:
Xiaomi 18 Pro Series New Phone Launch Event Special
