英伟达提出 AI 内存全栈协同设计方案,Vera Rubin 平台推理吞吐最高提升 30 倍NVIDIA proposes full-stack AI memory co-design; Vera Rubin boosts inference throughput up to 30x


AI free token/AI免费积分

英伟达提出 AI 内存全栈协同设计方案,Vera Rubin 平台推理吞吐最高提升 30 倍

英伟达11月5日发布文章称,随着AI模型规模扩大、上下文窗口延长以及智能体应用普及,GPU内存压力持续上升,HBM供应趋紧、售价上涨。英伟达提出借助GPU、CPU、网络、存储及软件协同外观,提高内存使用效率,而非单纯增加GPU内存容量。

具体来看,英伟达借助NVFP4低精度格式压缩模型权重和KV Cache,将暂时不活跃的上下文转移至成本更低的存储层,并利用多GPU分布式部署释放HBM空间。

英伟达披露,在SemiAnalysis AgentX测试中,GB300 NVL72在特定工作负载下的单位成本推理吞吐量最高达到H200平台的10倍;Vera Rubin NVL72相较GB300 NVL72,单位成本推理吞吐量最高提高30倍,而单GPU HBM容量仅增加约3%。

(文章来源:财联社)

文章来源:财联社
原标题:英伟达提出AI内存全栈协同外观思路 Vera Rubin平台每GPU内存容量仅增约3% 推理吞吐量提高最高30倍

本文来源:https://wap.eastmoney.com/a/202611063923910691.html
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

NVIDIA proposes full-stack AI memory co-design; Vera Rubin boosts inference throughput up to 30x

NVIDIA posted an article on November 5 stating that as AI models scale up, context windows lengthen, and agent applications become more widespread, GPU memory pressure continues to rise, with HBM supply tightening and prices increasing. NVIDIA proposed a synergistic approach leveraging GPUs, CPUs, networking, storage, and software to improve memory usage efficiency, rather than simply increasing GPU memory capacity.

Specifically, NVIDIA uses the NVFP4 low-precision format to compress model weights and KV Cache, moves temporarily inactive contexts to lower-cost storage tiers, and leverages multi-GPU distributed deployment to free up HBM space.

NVIDIA disclosed that in the SemiAnalysis AgentX test, the GB300 NVL72 achieved up to 10 times the per-unit-cost inference throughput of the H200 platform under specific workloads; compared to the GB300 NVL72, the Vera Rubin NVL72 achieved up to 30 times higher per-unit-cost inference throughput while per-GPU HBM capacity increased by only approximately 3%.

(Source: Cailian Press)

Source: Cailian Press

Original Title: NVIDIA Proposes Full-Stack Synergistic Approach for AI Memory; Vera Rubin Platform Sees Per-GPU Memory Capacity Increase of Only About 3%, Inference Throughput Increases Up to 30 Times

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry