Gemini 4 对决 Gemma 4:免费 Gemma 紧咬不放,思考 token 才是真成本Gemini 4 vs Gemma 4: Free Gemma Stays Close as Thinking Tokens Drive the Real Cost


AI free token/AI免费积分

Gemini 4 对决 Gemma 4:免费 Gemma 紧咬不放,思考 token 才是真成本

本文来源:https://deniz.in/gemini-4-vs-gemma-4-head-to-head-free-gemma-stays-close-while-thinking-tokens-dr
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

dev.to 上一篇实测对比把七款 Google 模型放在相同条件下跑 GSM8K 数学题与 BFCL 函数调用测试,结论不在精度而在成本:免费的 Gemma 4 与旗舰 Gemini 4 精度只差约一个百分点,而最低单价的 Gemini 4 型号反而成为单次运行最贵的一个,原因是推理 token 无法关闭。

测试涵盖 Gemma 4 26B 与 31B、Gemini 3.5 Flash-Lite、3.8 Flash,以及 Gemini 4 Hydrogen、Carbon、Argon,统一走 Gemini API 的 OpenAI 兼容端点:同样 1,319 道 GSM8K 题与 400 道 BFCL 题,temperature 0,数学答案上限 2,048 token、函数调用 512,八并发并重试,脚本与配置公开在 gemma4-dev 仓库。

精度上 Argon 以 0.9841 领跑 GSM8K,Hydrogen 与 Carbon 为 0.9803,3.8 Flash 0.9788,Gemma 4 26B 0.9742、31B 0.9735,3.5 Flash-Lite 0.9629;BFCL 上 Gemma 4 31B 与 Argon 并列 0.9675。速度上 Hydrogen 在 Gemini 4 中最快,每道 GSM8K 题 2.84 秒,Argon 为 7.63 秒。

成本反差来自推理:Gemma 4 只在免费档可用,全程零花费;Gemini 各型号总账单在 0.92 至 5.02 美元之间。Hydrogen 单价最低(每百万输入 1.50 美元、输出 7.50 美元)却账单最高(5.02 美元),因为 1,319 道数学题和 400 次调用全部触发思考,思考占 GSM8K 账单的 55%。谷歌文档称 Gemini 4 无法禁用推理,且 completion 计数不含思考 token,需要自行做减法;Carbon、Argon 将在 12 月 1 日涨价到每百万 4/20 美元。作者的结论是:按 token 单价选模型会失准,实际花费是单价、答案长度与思考量三者的乘积,成本敏感场景 Gemma 4 是更稳的默认项。

Gemini 4 vs Gemma 4: Free Gemma Stays Close as Thinking Tokens Drive the Real Cost

本文来源:https://deniz.in/gemini-4-vs-gemma-4-head-to-head-free-gemma-stays-close-while-thinking-tokens-dr
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

Gemini 4 vs Gemma 4 head-to-head: free Gemma stays close while thinking tokens drive the real cost

A dev.to hands-on comparison ran seven Google models through identical GSM8K and BFCL tests. Free Gemma 4 nearly matched Gemini 4 on accuracy, while unswitchable thinking tokens made the cheapest Gemini 4 model the costliest per run.

A hands-on comparison published on dev.to has run Google's newly released Gemini 4 lineup and the open Gemma 4 models through identical tests, and the practical takeaway is less about accuracy than about cost. On the benchmarks used, the free Gemma 4 variants sit within roughly a point of the flagship Gemini 4 models, and the cheapest Gemini 4 model per token turned out to be the most expensive per run because its reasoning cannot be switched off.

The author ran seven models — Gemma 4 26B and 31B, Gemini 3.5 Flash-Lite, Gemini 3.8 Flash, and Gemini 4 Hydrogen, Carbon and Argon — through the Gemini API's OpenAI-compatible chat endpoint under matched conditions: the same 1,319 GSM8K math problems and 400 BFCL function-calling questions, temperature 0, token caps of 2,048 per math answer and 512 per function call, eight concurrent requests with retries, and a single shared start time. Scoring used the same script, and the setup is available in the author's public gemma4-dev repository.

According to the results, Argon led GSM8K at 0.9841, with Hydrogen and Carbon at 0.9803 and 3.8 Flash at 0.9788. Gemma 4 26B scored 0.9742 and Gemma 4 31B 0.9735, while 3.5 Flash-Lite trailed at 0.9629. On BFCL, Gemma 4 31B tied Argon at 0.9675, meaning the free model matched the flagship on function calling. Speed favoured Hydrogen among the Gemini 4 models at 2.84 seconds per GSM8K problem, against 7.63 seconds for Argon.

Gemma 4 is available on the API's free tier only, so its runs cost nothing; the Gemini runs cost between $0.92 and $5.02 in total. Hydrogen is the cheapest Gemini 4 model per token — $1.50 per million input and $7.50 per million output, against $2 and $10 for Carbon and Argon, per Google's pricing page as captured on 2026-11-14 — yet it produced the largest bill at $5.02, versus $3.46 for Argon.

The gap comes from reasoning tokens. Google's documentation, cited in the article, states that reasoning cannot be disabled on Gemini 4 models. Hydrogen thought on all 1,319 math problems and all 400 function calls, and its thinking accounted for 55% of the GSM8K bill. Scheduled increases will sharpen the effect: Carbon and Argon rise to $4 and $20 per million tokens on 2026-12-01, and 3.8 Flash to $1.50 and $7.50 on 2027-01-01.

The comparison probes the settings developers would reach for first. On the OpenAI endpoint, Gemma 4 and 3.5 Flash-Lite accept the lowest reasoning-effort level and emit zero thinking tokens. Hydrogen, 3.8 Flash, Carbon and Argon reject that lowest setting and only accept "none", which the endpoint treats as each model's minimum documented level — and which still leaves Hydrogen and 3.8 Flash thinking. Carbon and Argon emit no thinking tokens at that setting.

The native API offers no true off switch either, according to the tests: setting a thinking budget of zero on Hydrogen still returned 134 thinking tokens on a sample question, and Hydrogen produced reasoning even when prompted with "Say hi." Across the full run, 3.8 Flash thought on 1,293 of the math problems, Carbon on 268 and Argon on 66.

A billing detail compounds this. The endpoint's completion token count excludes thinking tokens, which must be derived by subtracting prompt and completion counts from the total — and every reasoning token is billed at the output rate.

For anyone choosing between these models, per-token price is a poor proxy for spend. What a model actually costs is the product of price per token, tokens per answer and thinking per answer, and those three factors can point in opposite directions. The results also make Gemma 4 a serious default for cost-sensitive work: it is free, it matched Argon on function calling, and it stayed within about a point of the Gemini 4 models on math. Teams already committed to Gemini 4 should measure reasoning tokens directly rather than trust the completion counter — particularly before Carbon and Argon's December price increase takes effect.

Two-verb probe of 137 x402 paid routes finds most reject POST with 404 or 405

Gemini 4 Carbon takes MathArena lead from GPT-6.1 Sol at 15 times the cost

Google's Gemini 4 agents and Antigravity /teamwork tackle open problems in math and science

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry