robots.txt 成 AI 训练闸门:四大巨头共推退出标准,数据治理权上移robots.txt becomes an AI-training gate: four giants back a unified opt-out standard


AI free token/AI免费积分
robots.txt 成 AI 训练闸门:四大巨头共推退出标准,数据治理权上移

通信世界报道,2026 年 11 月 1 日,据 IT之家消息,OpenAI、谷歌、微软与 Anthropic 宣布将采用一项新的 AI 数据抓取退出标准。

按照该标准,内容发布方可通过一个统一信号(集成在网站的 robots.txt 文件中),一次性向多家 AI 企业表达其网站内容是否可用于模型训练,而不必逐家配置规则。

标准由 Anthropic 技术负责人向 IETF 提出。Anthropic 已率先采纳,OpenAI 计划未来数月落实,谷歌和微软预计于明年支持;Meta 方面表示支持建立通用信号,但未明确具体落地计划。

关键不在多了一个"拒绝抓取"选项,而在于 AI 训练数据的治理接口,开始从企业间双边规则,向机器可读、可复用的公共协议层迁移,重新定义了内容分发、流量识别与平台合规的边界。

但需明确:统一退出信号解决的是"表达"问题,不是"授权"问题——它更像技术层面的"请勿进入"标识,而非法律层面的授权合同。

本文来源:https://www.toutiao.com/article/7522739117425308758/
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

Communication World reported on November 1, 2026, citing ITHome, that OpenAI, Google, Microsoft, and Anthropic announced they will adopt a new AI data scraping opt-out standard.

According to this standard, content publishers can use a unified signal (integrated into the website's robots.txt file) to simultaneously indicate to multiple AI companies whether their website content can be used for model training, without needing to configure rules for each company individually.

The standard was proposed to the IETF by Anthropic's technical lead. Anthropic has already adopted it, OpenAI plans to implement it in the coming months, and Google and Microsoft are expected to support it next year; Meta expressed support for establishing a universal signal but has not clarified its specific implementation plans.

The key lies not in the addition of a "do not scrape" option, but in the fact that the governance interface for AI training data is transitioning from inter-enterprise bilateral rules to a machine-readable, reusable public protocol layer, redefining the boundaries of content distribution, traffic identification, and platform compliance.

However, it must be clarified: the unified opt-out signal addresses the issue of "expression," not "authorization"—it acts more like a technical "Do Not Enter" sign rather than a legal authorization contract.

robots.txt becomes an AI-training gate: four giants back a unified opt-out standard

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry