从 OmniRoute 到 AIRoute:一个面向国内合规增强的 AI 网关是如何诞生的From OmniRoute to AIRoute: How an AI Gateway for Enhanced Chinese Compliance Was Born


AI free token/AI免费积分

从 OmniRoute 到 AIRoute:一个面向国内合规增强的 AI 网关是如何诞生的

本文将从海外热门项目 OmniRoute 讲起,分析国内 AI 网关生态的不足,最后引出一个专门为国内场景打造的开源项目 —— AIRoute。重点介绍它能做什么、能解决哪些实际问题。演示地址:

https://jaccen.github.io/AIRoute/


一、OmniRoute 是什么?

如果你在 GitHub 上搜索 “AI Gateway”,OmniRoute 绝对是绕不开的一个项目。它是一个基于 Next.js 构建的本地 AI 路由网关,提供一个统一的 OpenAI 兼容端点(/v1/*),可以将流量智能路由到多个上游 Provider。

OmniRoute 的核心能力:

维度
详情
Provider 数量
237+ 个 LLM Provider(含 90+ 免费 Provider)
路由策略
17 种(priority、weighted、cost-optimized、context-relay、fusion 等)
Token 压缩
RTK + Caveman 堆叠压缩,节省 15-95% Token
内置 MCP
95 个工具,3 种传输,30 个 Scope
A2A 协议
JSON-RPC 2.0,6 个技能
记忆
FTS5 + 向量
防护栏
PII、注入、视觉检测
多平台
Web · Desktop · Termux · PWA
国际化
42 种语言

简单来说,OmniRoute 的定位是 “一个端点连接所有 AI 工具和 Provider”。它的 Combo 机制(模型链式路由)是一大亮点——当一个 Provider 配额耗尽或故障时,会自动滑动到下一个模型,实现"永不断线"。

在海外开发者社区,OmniRoute 已经成为 Claude Code、Codex、Cursor、Cline、Copilot 等编码工具的首选 AI 网关。


二、国内 AI 网关生态的现状与不足

国内的 LLM 网关生态也有自己的主力选手:One-API / New-API。

  • One-API
    (songquanpeng/one-api):Go 语言编写,支持 30+ 家国内外 Provider,统一 OpenAI 协议,天然支持国产模型(通义、文心、讯飞、智谱、Kimi、豆包、DeepSeek 等),是国内做"多供应商 + 计费 + 团队额度"的首选。
  • New-API
    :One-API 的社区增强版,增加了 Midjourney、Suno、Rerank 等多模态支持。

但深入使用后,你会发现国内方案存在几个明显的短板:

1. 路由能力薄弱

One-API/New-API 本质上是 “中转 + 计费” 工具,路由策略基本只有简单的权重轮询和失败切换。缺少 OmniRoute 那样的 17 种路由策略、Auto-Combo 引擎、成本优化路由、上下文感知路由等高级能力。

2. 缺乏 Token 压缩能力

OmniRoute 的 RTK + Caveman 压缩可以节省 15-95% 的 Token 消耗,这对成本敏感的场景(尤其是工具调用密集的编码 Agent)价值巨大。国内方案完全没有这方面的布局。

3. 合规能力缺失

国内企业在使用 AI 模型时,面临独特的合规要求:

  • 数据出境
    :涉密数据不能随意发送到海外 Provider
  • 内容安全
    :需要符合国内的内容审核要求
  • PII 脱敏
    :个人信息保护法(PIPL)要求对敏感信息进行脱敏
  • 审计追踪
    :金融、政务等场景需要完整的调用审计链

OmniRoute 虽然有 Guardrails,但它是面向西方合规体系设计的(GDPR 等),对国内的《数据安全法》《个人信息保护法》《网络安全法》等没有针对性适配。

4. 国内 Provider 集成深度不够

OmniRoute 虽然声称支持 237+ Provider,但对国内 Provider(DeepSeek、通义千问、文心一言、智谱等)的集成深度远不如 One-API。国内 Provider 的 API 格式、认证方式、特殊参数都有本土特色,需要专门适配。

5. 部署方式缺乏本土化支持OmniRoute 的部署方式主要是 Docker / 本地 Node.js,对国内常用的 ACK(阿里云容器服务)、CCE(华为云容器引擎)、Fly.io 等部署方式缺乏原生支持。


三、AIRoute:面向国内合规增强的 AI 网关

正是基于以上痛点,AIRoute 项目应运而生。

AIRoute 的定位:基于 OmniRoute v3.8.46 Fork,专为国内合规场景增强的 AI 网关。

它保留了 OmniRoute 强大的路由引擎和多 Provider 支持,同时针对国内生态做了深度增强。


四、AIRoute 能做什么?—— 能力全景

4.1 智能路由:让请求自动找到最优路径

AIRoute 继承了 OmniRoute 的 17 种路由策略,覆盖了从简单到复杂的各种路由场景:

路由策略
解决什么问题
priority
按优先级依次尝试,主 Provider 挂了自动切到备用
weighted
按权重随机分配,适合多 Provider 均衡负载
cost-optimized
自动选择当前最便宜的 Provider,省钱第一
context-relay
长对话自动接力,上下文在 Provider 间无缝传递
fusion
多模型并行回答,一个裁判模型综合出最优答案
lkgp
贴住上次成功的 Provider,减少抖动
auto
9 因子实时打分,全自动选择最佳 Provider

实际场景:你配置了 DeepSeek + 通义千问 + 文心一言三个 Provider,选择 cost-optimized 策略。AIRoute 会实时对比三家的价格,自动把请求发给当前最便宜的那家。当某家配额用完,无缝切换到下一家,你的业务完全无感知。

4.2 Token 压缩:省 15-95% 的 Token 费用

这是 AIRoute 继承自 OmniRoute 的核心能力。内置 10 种压缩引擎,按顺序叠加运行:

引擎
做什么
Session-Dedup
跨轮次去重,丢弃重复内容
CCR
大块内容归档,按需取回
RTK
工具结果智能过滤、去重、截断
Headroom
JSON 数组无损压缩(~30%+)
Relevance
基于查询的句子相关性评分
Caveman
规则压缩(输出 ~65-75%)
LLMLingua-2
ML 语义剪枝(MobileBERT ONNX)
Lite
空白 + 图片 URL 裁剪
Aggressive
摘要 + 老旧轮次渐进老化
Ultra
启发式 Token 裁剪 + 小模型兜底

实际场景:你在用 Claude Code 做一个大型项目的代码重构,每次工具调用返回大量文件内容。传统方式一个会话可能消耗 50 万 Token,开启 AIRoute 的压缩后,实际发送给模型的可能只有 10 万 Token。按 Claude 3.5 Sonnet 的价格算,一次会话就省了 20」

AIRoute 的解法:开启 Token 压缩。RTK + Caveman 堆叠压缩,工具调用密集的编码 Agent 场景可以节省 50-80% Token。按 Claude 3.5 Sonnet 价格,每月能省 (openssl rand -hex 32)# 3. 部署kubectl apply -f deploy/ack/airoute.yaml# 4. 访问kubectl port-forward svc/airoute 20128:20128 -n airoute

 

本地开发

  •  
  •  
  •  
  •  
  •  
  •  
# 安装npm install -g airouteAIRROUTE_REGION=cn STORAGE_ENCRYPTION_KEY=$(openssl rand -hex 32) airoute# 访问# Dashboard: http://localhost:20128# API: http://localhost:20128/v1

在你的应用中使用

  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
# Python(OpenAI SDK 直接用)from openai import OpenAIclient = OpenAI(    base_url="http://localhost:20128/v1",    api_key="your-airoute-key")# 一切照旧,AIRoute 自动路由response = client.chat.completions.create(    model="auto",  # 或指定具体模型    messages=[{"role": "user", "content": "你好"}])

十、与同类项目的对比

维度
OmniRoute
One-API/New-API
AIRoute
Provider 数量
237+
30+
237+(继承 OmniRoute)+ 国内深度适配
路由策略
17 种
简单权重
17 种(继承 OmniRoute)
Token 压缩
RTK + Caveman (15-95%)
无
RTK + Caveman (15-95%)
MCP 支持
95 个工具
无
95 个工具
A2A 协议
6 个技能
无
6 个技能
PII 脱敏
无
无
18 类敏感信息自动脱敏
国密支持
无
无
SQLCipher (SM4) + SM2/SM3 证书
数据导出管控
无
无
支持
国内 Provider 适配
基础
深度
深度 + 区域路由
部署方式
Docker / Node.js
Docker / 本地
Docker / ACK / CCE / Fly.io / Native
计费能力
基础
完善
继承 OmniRoute + 增强
开源协议
MIT
MIT
MIT
维度
OmniRoute
One-API/New-API
AIRoute
—
—
—
—
Provider 数量
237+
30+
237+(继承 OmniRoute)+ 国内深度适配
路由策略
17 种
简单权重
17 种(继承 OmniRoute)
Token 压缩
RTK + Caveman (15-95%)
无
RTK + Caveman (15-95%)
MCP 支持
95 个工具
无
95 个工具
A2A 协议
6 个技能
无
6 个技能
PII 脱敏
无
无
18 类敏感信息自动脱敏
国密支持
无
无
SQLCipher (SM4) + SM2/SM3 证书
数据导出管控
无
无
支持
国内 Provider 适配
基础
深度
深度 + 区域路由
部署方式
Docker / Node.js
Docker / 本地
Docker / ACK / CCE / Fly.io / Native
计费能力
基础
完善
继承 OmniRoute + 增强
开源协议
MIT
MIT
Apache License 2.0

十一、适用场景

场景
AIRoute 如何帮助你
金融 / 政务
国密加密 + 数据不出境 + 完整审计链 + PII 脱敏,满足等保和监管要求
企业内部
多模型统一入口 + 团队额度管理 + 成本控制 + 用量审计
编码 Agent
Claude Code / Cursor / Cline 国内加速 + Token 压缩省费 + Auto-Combo 防限流
RAG 应用
百度搜索 Provider + 向量存储 + 记忆系统,一站式 RAG 方案
混合云部署
核心业务走自建推理,非敏感业务走海外 Provider,区域路由自动分流
多 Provider 管理
一个端点管理 200+ Provider,自动切换,零停机

十二、总结

OmniRoute 是一个功能极其强大的 AI 网关,但它的设计初衷是面向全球开发者,对国内的合规要求和 Provider 生态缺乏深度支持。国内的 One-API/New-API 虽然解决了"多 Provider 统一入口"的问题,但在路由能力、Token 压缩、合规增强等方面存在明显短板。

AIRoute 的出现,正是为了弥合这一差距。它继承了 OmniRoute 强大的路由引擎,同时针对国内场景做了深度增强:

  • 省钱
    :Token 压缩节省 15-95% 费用
  • 省心
    :17 种路由策略 + Auto-Combo,Provider 挂了自动切
  • 合规
    :PII 脱敏 + 国密加密 + 数据导出管控 + 审计日志
  • 好用
    :一个端点接入所有 Provider,5 分钟部署完成

如果你正在寻找一个既强大又合规的 AI 网关方案,不妨试试 AIRoute。


项目地址:https://github.com/jaccen/AIRoute

基于:OmniRoute v3.8.46 Fork

技术栈:Node.js 22+ / TypeScript / Next.js 16 / SQLite / PostgreSQL


本文首发于 CSDN,转载请注明出处。

 

From OmniRoute to AIRoute: How an AI Gateway with Domestic Compliance Enhancements Was Born

This article starts from the overseas popular project OmniRoute, analyzes the shortcomings of China's AI gateway ecosystem, and finally introduces an open-source project built specifically for domestic scenarios — AIRoute. We will focus on what it can do and what practical problems it can solve. Demo address:

https://jaccen.github.io/AIRoute/

I. What is OmniRoute?

If you search for "AI Gateway" on GitHub, OmniRoute is definitely a project you cannot bypass. It is a local AI routing gateway built on Next.js, providing a unified OpenAI-compatible endpoint (/v1/*), which can intelligently route traffic to multiple upstream providers.

The core capabilities of OmniRoute:

Dimension

Details

Provider Count

237+ LLM Providers (including 90+ free providers)

Routing Strategy

17 types (priority, weighted, cost-optimized, context-relay, fusion, etc.)

Token Compression

RTK + Caveman stacked compression, saving 15-95% tokens

Built-in MCP

95 tools, 3 transmission methods, 30 scopes

A2A Protocol

JSON-RPC 2.0, 6 skills

Memory

FTS5 + Vector

Guardrails

PII, injection, visual detection

Multi-platform

Web · Desktop · Termux · PWA

Internationalization

42 languages

Simply put, OmniRoute's positioning is "one endpoint connects all AI tools and providers." Its Combo mechanism (model chaining routing) is a major highlight — when one provider's quota is exhausted or fails, it automatically slides to the next model, achieving "never disconnected."

In the overseas developer community, OmniRoute has become the preferred AI gateway for coding tools such as Claude Code, Codex, Cursor, Cline, and Copilot.

II. Current Status and Shortcomings of China's AI Gateway Ecosystem

China's LLM gateway ecosystem also has its own leading players: One-API / New-API.

(songquanpeng/one-api): Written in Go, supports 30+ domestic and international providers, unified OpenAI protocol, natively supports domestic models (Tongyi, ERNIE, iFlytek, Zhipu, Kimi, Doubao, DeepSeek, etc.), and is the top choice in China for "multi-provider + billing + team quota."

: One-API's community enhanced version, adding multi-modal support such as Midjourney, Suno, and Rerank.

But after in-depth use, you will find that domestic solutions have several obvious shortcomings:

1. Weak routing capabilities

One-API/New-API is essentially a "relay + billing" tool, with routing strategies limited to simple weighted round-robin and failure switching. It lacks advanced capabilities such as OmniRoute's 17 routing strategies, Auto-Combo engine, cost-optimized routing, and context-aware routing.

2. Lack of Token compression capability

OmniRoute's RTK + Caveman compression can save 15-95% of token consumption, which is of great value in cost-sensitive scenarios (especially tool-call-intensive coding agents). Domestic solutions have no layout in this regard.

3. Lack of compliance capabilities

When Chinese enterprises use AI models, they face unique compliance requirements:

: Classified data cannot be freely sent to overseas providers

: Must comply with China's content review requirements

: The Personal Information Protection Law (PIPL) requires desensitization of sensitive information

: Financial, government, and other scenarios require a complete call audit chain

OmniRoute has Guardrails, but it is designed for the Western compliance system (such as GDPR), with no targeted adaptation for China's Data Security Law, Personal Information Protection Law, Cybersecurity Law, etc.

4. Insufficient deep integration with domestic providers

Although OmniRoute claims to support 237+ providers, its integration depth with domestic providers (DeepSeek, Tongyi Qianwen, ERNIE Bot, Zhipu, etc.) is far inferior to One-API. Domestic provider APIs have localized features in format, authentication methods, and special parameters, requiring dedicated adaptation.

5. Deployment methods lack localization support. OmniRoute's deployment is mainly Docker/local Node.js, lacking native support for deployment methods commonly used in China such as ACK (Alibaba Cloud Container Service), CCE (Huawei Cloud Container Engine), and Fly.io.

III. AIRoute: An AI Gateway with Enhanced Domestic Compliance

It is precisely based on the above pain points that the AIRoute project emerged.

AIRoute's positioning: Based on OmniRoute v3.8.46 Fork, an AI gateway enhanced specifically for domestic compliance scenarios.

It retains OmniRoute's powerful routing engine and multi-provider support, while making in-depth enhancements for the domestic ecosystem.

IV. What Can AIRoute Do? — Capability Overview

4.1 Intelligent Routing: Let Requests Automatically Find the Optimal Path

AIRoute inherits OmniRoute's 17 routing strategies, covering various routing scenarios from simple to complex:

Routing Strategy

What Problem It Solves

Try in priority order, automatically switch to backup when main provider goes down

Random allocation by weight, suitable for multi-provider load balancing

Automatically select the cheapest provider, cost savings first

Long conversations automatically relay, context seamlessly passed between providers

Multiple models answer in parallel, one judge model synthesizes the optimal answer

Stick to the last successful provider, reduce jitter

9-factor real-time scoring, fully automatic selection of the best provider

Practical scenario: You configure three providers — DeepSeek, Tongyi Qianwen, and ERNIE Bot — and choose the cost-optimized strategy. AIRoute will compare the prices of all three in real-time and automatically send requests to the cheapest one. When one provider's quota is used up, it seamlessly switches to the next, and your business remains completely unaware.

4.2 Token Compression: Save 15-95% on Token Costs

This is the core capability inherited by AIRoute from OmniRoute. It has 10 built-in compression engines that run in sequence:

Engine

What It Does

Session-Dedup

Cross-turn deduplication, discard duplicate content

CCR

Archive large blocks, retrieve on demand

RTK

Intelligently filter, deduplicate, and truncate tool results

Headroom

Lossless compression of JSON arrays (~30%+)

Relevance

Query-based sentence relevance scoring

Caveman

Rule-based compression (output ~65-75%)

LLMLingua-2

ML semantic pruning (MobileBERT ONNX)

Lite

Whitespace + image URL trimming

Aggressive

Summarization + progressive aging of old turns

Ultra

Heuristic token trimming + small model fallback

Practical scenario: You are using Claude Code for code refactoring on a large project, and each tool call returns a large amount of file content. In traditional ways, a session may consume 500,000 tokens. After enabling AIRoute's compression, only 100,000 tokens may actually be sent to the model. Calculated at Claude 3.5 Sonnet's price, one session saves 20

AIRoute's solution: Enable Token compression. RTK + Caveman stacked compression can save 50-80% of tokens in tool-call-intensive coding agent scenarios. At Claude 3.5 Sonnet pricing, you can save per month.

Local Development

Using it in your application

X. Comparison with Similar Projects

Dimension

OmniRoute

One-API/New-API

AIRoute

Provider Count

237+

30+

237+ (inherited from OmniRoute) + deep domestic adaptation

Routing Strategy

17 types

Simple weight

17 types (inherited from OmniRoute)

Token Compression

RTK + Caveman (15-95%)

None

RTK + Caveman (15-95%)

MCP Support

95 tools

None

95 tools

A2A Protocol

6 skills

None

6 skills

PII Desensitization

None

None

Automatic desensitization of 18 types of sensitive information

National Cryptography Support

None

None

SQLCipher (SM4) + SM2/SM3 certificates

Data Export Control

None

None

Supported

Domestic Provider Adaptation

Basic

Deep

Deep + regional routing

Deployment Method

Docker / Node.js

Docker / Local

Docker / ACK / CCE / Fly.io / Native

Billing Capability

Basic

Complete

Inherited from OmniRoute + enhanced

Open Source License

MIT

MIT

MIT

Dimension

OmniRoute

One-API/New-API

AIRoute

—

—

—

—

Provider Count

237+

30+

237+ (inherited from OmniRoute) + deep domestic adaptation

Routing Strategy

17 types

Simple weight

17 types (inherited from OmniRoute)

Token Compression

RTK + Caveman (15-95%)

None

RTK + Caveman (15-95%)

MCP Support

95 tools

None

95 tools

A2A Protocol

6 skills

None

6 skills

PII Desensitization

None

None

Automatic desensitization of 18 types of sensitive information

National Cryptography Support

None

None

SQLCipher (SM4) + SM2/SM3 certificates

Data Export Control

None

None

Supported

Domestic Provider Adaptation

Basic

Deep

Deep + regional routing

Deployment Method

Docker / Node.js

Docker / Local

Docker / ACK / CCE / Fly.io / Native

Billing Capability

Basic

Complete

Inherited from OmniRoute + enhanced

Open Source License

MIT

MIT

Apache License 2.0

XI. Applicable Scenarios

Scenario

How AIRoute Helps You

National cryptography encryption + data not leaving borders + complete audit chain + PII desensitization, meeting classified protection and regulatory requirements

Multi-model unified entry + team quota management + cost control + usage auditing

Claude Code / Cursor / Cline domestic acceleration + Token compression for cost savings + Auto-Combo to prevent rate limiting

Baidu search Provider + vector storage + memory system, one-stop RAG solution

Core business runs on self-built inference, non-sensitive business uses overseas providers, regional routing automatically分流

One endpoint manages 200+ providers, automatic switching, zero downtime

XII. Summary

OmniRoute is an extremely powerful AI gateway, but its design初衷 is for global developers, lacking deep support for China's compliance requirements and provider ecosystem. Although China's One-API/New-API has solved the "multi-provider unified entry" problem, it has obvious shortcomings in routing capabilities, Token compression, and compliance enhancements.

AIRoute's emergence is precisely to bridge this gap. It inherits OmniRoute's powerful routing engine, while making in-depth enhancements for domestic scenarios:

: Token compression saves 15-95% in costs

: 17 routing strategies + Auto-Combo, automatic switching when providers fail

: PII desensitization + national cryptography encryption + data export control + audit logs

: One endpoint connects to all providers, deployed in 5 minutes

If you are looking for an AI gateway solution that is both powerful and compliant, why not try AIRoute?

Project address: https://github.com/jaccen/AIRoute

Based on: OmniRoute v3.8.46 Fork

Tech stack: Node.js 22+ / TypeScript / Next.js 16 / SQLite / PostgreSQL

First published on CSDN, please indicate the source when reprinting.

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry