
AI 模型在处理部分语言的提示时所需的 token 数量,远高于处理英语提示。一个关注 AI 研究语言公平性的联盟 Coalition for Language Equity in AI Research 上个月发表的研究将此称为“隐性 token 税”(hidden token tax)——这意味着使用小语种的用户要为 AI 支付更高的成本。
研究发现,即便在互联网最流行的 20 种语言中,token 使用量也存在差异:最昂贵语言所用的 token 数量,是最便宜语言的 1.5 至 5 倍。当研究人员将范围扩大到 200 种语言时,差异进一步扩大——某一 AI 模型的 token 成本差距甚至拉大到 17 倍。
研究指出,位于班加罗尔的初创公司,可能要为每一次提示与补全支付附加费,而其位于奥斯汀的竞争对手却无需承担这笔费用。token 税的影响远不止于成本:token 数量越多,延迟越高,有效的上下文容量也越低。在计算和连接本就有限的地区,这些惩罚最为严重。token 不公平,正在限制人们参与 AI 经济的能力。
本文来源:Solidot(相关链接:clear-languages.org/blog/the-tokenizer-tax、restofworld.org/2026/ai-tokenizer-tax-non-english-languages-cost-inequity)。若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

AI models require far more tokens to process prompts in some languages than they do for English. A study published last month by the Coalition for Language Equity in AI Research calls this the "hidden token tax" — meaning users of smaller languages pay more for AI.
The research found that even among the 20 most popular languages on the internet, token usage varies: the most expensive language uses 1.5 to 5 times as many tokens as the cheapest. When researchers expanded the scope to 200 languages, the gap widened further — for one AI model, the token-cost disparity grew to 17x.
The study notes that a startup based in Bangalore may pay a surcharge on every prompt and completion, while its competitor in Austin pays nothing. The token tax’s impact goes well beyond cost: more tokens mean higher latency and lower effective context capacity. The penalties are most severe in regions where compute and connectivity are already limited. Token inequity is restricting who can participate in the AI economy.
Source: Solidot (related: clear-languages.org/blog/the-tokenizer-tax; restofworld.org/2026/ai-tokenizer-tax-non-english-languages-cost-inequity). For copyright issues, please contact us.