中文EN RSS

蚂蚁开源大模型内生式安全护栏SingProbe,适配29个主流模型Ant unveils in-house security guardrail SingProbe for large models, compatible with 29 mainstream models


AI free token/AI免费积分
蚂蚁开源大模型内生式安全护栏SingProbe,适配29个主流模型

2026国家网络安全宣传周期间,蚂蚁AI安全实验室宣布开源大模型内生式安全护栏SingProbe,同步开放相关代码、模型和实测基准,为开发者给出与模型生成过程同步运行的安全检测能力。

当下,SingProbe已适配Ling-3.0系列、GLM-5.2/5.3、Qwen、DeepSeekV4等29个主流开源大模型,并接入SGLang、vLLM推理框架,兼容开发者在现有推理流程中集成安全防护。

蚂蚁开源大模型安全内生护栏SingProbe Infra,已适配29个主流开源模型

随着大模型进入日常问答、办公辅助、客户服务等业务情形,开发者既需要关注回答质量,也需要及时发现不安全内容和编造信息。

例如,AI在健康咨询中可能给出不恰当的用药建议,在资料问答中可能编造不存在的文献。如何兼顾风险检测、响应速度和部署成本,成为大模型应用落地的关键难题。

当下,大模型应用通常借助独立的外置护栏模型审核输入或输出。这种方式通用、直接,但需要额外处理待审核内容,增加计算和部署成本。如果等待完整回答生成后再检查,风险信号可能来得太晚;如果提高检查频率,又会进一步增加运行负担。

运行时探针是学界探索兼顾安全检测成效与运行效率的一条工艺路径,但从研究走向实际应用,还需要模型适配、推理框架集成等工程兼容。

SingProbe借助适配多种开源模型、接入主流推理框架,补充了相关部署能力,让开发者更方便地将内生式安全护栏接入实际业务。

运行时探针是学界探索兼顾安全检测成效与运行效率的一种护栏思路。

但由于缺乏配套基础设施兼容,相关研究成果在实际落地中仍面临障碍,应用端仍更多使用外置护栏。SingProbe借助给出模型适配、推理框架集成等工程兼容,让开发者更方便地将内生式安全护栏接入实际业务。

SingProbe如同一个模型内的“安全探头”,借助复用大模型推理过程中已经产生的内部信息,持续输出使用者意图、回答安全和幻觉风险分数。

模型一边生成回答,探针一边给出风险信号,应用平台无需等待整段回答结束,就可以据此采取告警、中止回答、重新生成等措施。

据研发团队在Ling-3.0-flash模型生产环境中的实测,SingProbe在解码阶段引入的额外开销低于0.5%。

团队实测表明,flash版本在回答安全分类和流式安全检测任务上超过所选公开基线,在幻觉检测任务上与参考基线基本持平,展现出兼顾检测能力与运行效率的潜力。

此次同步开放的流式安全实测基准SingStreamBench,进一步关注风险浮现和被发现的具体时刻。它不仅检验护栏能否识别风险,还考察是否过早触发、发现是否及时,协助开发者评估安全防护成效及其对正常回答的影响。

在风险识别之外,团队还探索了如何利用风险信号开展按需纠偏。作为一项情形验证,SingProbe-Med在医疗内容生成过程中,仅在风险达到触发条件后,对局部高风险内容开展干预。

在AntAngelMed-100B的医疗实测中,完整干预可以纠正基线模型中25.03%原本出错的回答,展示了内生风险信号用于生成过程安全控制的潜力。

借助此次开源,开发者可以获取代码与集成说明、模型及实测资源,开展部署、测试和进一步适配。SingProbe既可独立使用,也可与外置护栏配合。

后续,团队将逐步扩展对更多开源模型的兼容,并邀请开发者、研究者和产业伙伴参与试用反馈,共同完善大模型生成过程中的安全防护能力。

项目代码与集成说明已在GitHub开放,相关模型及SingStreamBench实测基准可借助Hugging Face获取。

本文来源:https://zhidx.com/p/594440.html
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

Ant unveils in-house security guardrail SingProbe for large models, compatible with 29 mainstream models

During the 2026 National Network Security Week, Ant AI Security Lab announced the open-source release of SingProbe, an intrinsic safety guard for large models. This release includes related code, model, and benchmark tests, providing developers with safety detection capabilities that run synchronously with the model generation process.

Currently, SingProbe is compatible with 29 mainstream open-source large models, including the Ling-3.0 series, GLM-5.2/5.3, Qwen, and DeepSeekV4. It has also been integrated into the SGLang and vLLM inference frameworks, allowing developers to incorporate safety protection into their existing inference workflows.

蚂蚁开源大模型安全内生护栏SingProbe Infra,已适配29个主流开源模型

As large models are increasingly used for daily Q&A, office assistance, and customer service, developers need to ensure not only the quality of the responses but also the timeliness of detecting unsafe content and fabricated information.

For example, an AI might give improper medication advice in health consultations or invent nonexistent literature in knowledge Q&A. Balancing risk detection, response speed, and deployment costs is a key challenge for applying large models in real-world scenarios.

Currently, most large model applications rely on standalone, external guard models to review inputs or outputs. While this is a direct approach, it requires additional processing time for the content to be reviewed, increasing computational and operational costs. Checking only after a full response has been generated may result in risk signals arriving too late, while frequent checks increase the computational burden.

In-stream probes are a research direction that aims to balance the effectiveness of safety detection with running efficiency. However, transitioning from research to practical application requires engineering compatibility, such as model adaptation and integration with inference frameworks.

By supporting various open-source models and integrating with popular inference frameworks, SingProbe fills this gap, making it easier for developers to deploy intrinsic safety guards in their actual business operations.

In-stream probes represent a novel approach to safety guards that simultaneously considers detection effectiveness and running efficiency.

However, due to a lack of compatible infrastructure, these research results have struggled to be widely adopted, with the industry still primarily relying on external guards. SingProbe addresses this by providing model compatibility and framework integration, enabling developers to more easily incorporate intrinsic safety guards into their workflows.

SingProbe acts like a "safety sensor" within the model, continuously outputting scores for user intent, response safety, and hallucination risk by leveraging the internal information generated during the model's inference process.

As the model generates a response, the probe simultaneously outputs risk signals. This allows the application platform to take actions such as issuing warnings, halting the response, or triggering a re-generation without waiting for the full response to complete.

According to the development team, the extra overhead introduced by SingProbe during the decoding phase is less than 0.5% in the production environment for the Ling-3.0-flash model.

Practical tests show that the flash version outperforms selected public baselines in response safety classification and in-stream safety detection tasks, while performing on par with reference baselines in hallucination detection, demonstrating its potential to balance detection capability with running efficiency.

The team has also released SingStreamBench, an in-stream safety benchmark that focuses on the timing of risk emergence and detection. This benchmark tests whether the guard is triggered prematurely and whether the detection is timely, helping developers assess the effectiveness of their safety protection and its impact on regular responses.

In addition to risk identification, the team is exploring how to use these risk signals for on-demand correction. As a proof of concept, SingProbe-Med intervenes in medical content generation only when the risk criteria are met, rather than applying a blanket correction.

In medical tests using AntAngelMed-100B, full intervention successfully corrected 25.03% of the errors in the baseline model's answers, highlighting the potential of using intrinsic risk signals for safety control during the generation process.

With this open-source release, developers can access the code, integration instructions, models, and test resources to deploy, test, and further customize SingProbe. The tool can be used independently or in combination with existing external guards.

The team plans to expand compatibility to more open-source models in the future and invites developers, researchers, and industry partners to test the system and provide feedback to improve safety capabilities during the large model generation process.

The project code and integration guide are available on GitHub, while the model and SingStreamBench are accessible via Hugging Face.

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry

←上一篇← Prev下一篇→Next →