aisix
Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock & more — routing, guardrails, caching, rate limits, observability.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
It has changed since its scan on 11 Nov 2026.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/api7/aisix with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/api7/aisix'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/api7--aisix)HTML
<a href="/listings/api7--aisix"><img src="/api/listings/api7--aisix/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
One container. No control plane, no database, no configuration store — the gateway reads every dynamic resource from one declarative `resources.yaml`.
# config.yaml
resources_file: /etc/aisix/resources.yaml
proxy:
addr: "0.0.0.0:3000"
admin:
enabled: false # a declarative gateway needs no admin listener
observability:
metrics:
prometheus:
enabled: true
addr: "0.0.0.0:9090"# resources.yaml
_format_version: "1"
provider_keys:
- display_name: openai-main
provider: openai
api_key: ${OPENAI_API_KEY} # interpolated from the environment
models:
- display_name: my-model
provider: openai
model_name: gpt-4o-mini
provider_key: openai-main
api_keys:
- display_name: local-dev
key_env: CALLER_API_KEY # hashed at load; the plaintext is never stored
allowed_models: ["my-model"]export OPENAI_API_KEY="YOUR_PROVIDER_KEY" export CALLER_API_KEY="YOUR_CALLER_KEY" docker run -d --name aisix \ -v "$(pwd)/config.yaml:/etc/aisix/config.yaml:ro" \ -v "$(pwd)/resources.yaml:/etc/aisix/resources.yaml:ro" \ -e OPENAI_API_KEY -e CALLER_API_KEY \ -p 3000:3000 -p 127.0.0.1:9090:9090 \ ghcr.io/api7/aisix:latest # proxy → :3000, metrics + status → :9090 # ^ the metrics/status listener is unauthenticated; # keep it on loopback or a private network
Then call the gateway exactly like OpenAI:
curl http://localhost:3000/v1/chat/completions \
-H "Authorization: Bearer $CALLER_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"my-model","messages":[{"role":"user","content":"hello"}]}'Edit `resources.yaml` and send `SIGHUP` (`docker kill -s HUP aisix`) to apply changes with no restart — an invalid file is rejected whole and the last good configuration keeps serving. Check a file before booting with `aisix validate --resources resources.yaml`.
Full walkthrough: the [Gateway Quickstart](https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart) · every field: the [resources file reference](https://docs.api7.ai/ai-gateway/reference/resources-file). For a multi-replica cluster, point the gateway at etcd instead — `resources_file` and `etcd` are mutually exclusive.
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 353 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- <div align="center">
- # AISIX OSS
- ### The open-source, Rust-native AI gateway for LLMs and AI agents
- **One OpenAI-compatible API in front of every model.** Route, govern, secure, cache, and
- observe all your LLM and AI-agent traffic from a single control point — shipped as one
- static binary with low per-request overhead. Run it in your infrastructure for free,
- forever.
- *Built by the original creators of [Apache APISIX](https://apisix.apache.org/).*
- [](LICENSE)
- [](https://www.rust-lang.org/)
- [](https://docs.api7.ai/ai-gateway/)
- [](https://discord.gg/dUmRZ7Rvf)
- [](https://api7.ai/ai-gateway)
- [**Start free**](https://api7.ai/ai-gateway?utm_source=github&utm_medium=readme&utm_campaign=ai-gateway) ·
- [**Documentation**](https://docs.api7.ai/ai-gateway/) ·
- [**Quickstart**](https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart) ·
- [**AISIX Enterprise and AISIX Cloud**](https://api7.ai/ai-gateway?utm_source=github&utm_medium=readme&utm_campaign=cloud) ·
- [**Roadmap**](ROADMAP.md)
- <br>
- <img src="assets/aisix-architecture.svg" alt="AISIX OSS architecture — one OpenAI- or Anthropic-compatible API in front of OpenAI, Anthropic, Gemini/Vertex, Bedrock, Azure OpenAI, and DeepSeek, with API key auth, rate and token limits, guardrails, caching, routing and failover, and observability in between" width="100%">
- </div>
- ---
- **AISIX OSS** is the open-source product built around the Rust-native AISIX gateway. The
- gateway puts a single, OpenAI-compatible API in front of every LLM provider — OpenAI,
- Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, DeepSeek, and any OpenAI-compatible
- endpoint. It gives platform teams one place to route, govern, secure, and observe LLM
- traffic, with first-class SSE streaming and low gateway overhead.
- It runs as a **single static binary** — low cold-start, lock-free config reads, and hot
- configuration reloads with no restarts: declare resources in one `resources.yaml` and
- reload on `SIGHUP`, or point the gateway at etcd for a multi-replica cluster. Run the
- gateway in your infrastructure, or connect it to **AISIX Enterprise** or
- **[AISIX Cloud](https://api7.ai/ai-gateway?utm_source=github&utm_medium=readme&utm_campaign=cloud)**
- for centralized management with team governance, budgets, audit, and a dashboard.
- > **AISIX OSS (this repo)** is the open-source product. It runs without a control plane
- > using declarative configuration or etcd. The same gateway binary serves as the data
- > plane in the commercial products: customers host the control plane with **AISIX
- > Enterprise**, while API7 hosts it with **AISIX Cloud**. In all three products, the
- > gateway runs in your environment and calls providers directly; live AI traffic does not
- > pass through either control plane. The proxy API is identical throughout.
- > **[Talk to us about AISIX Enterprise or AISIX Cloud →](https://api7.ai/contact?utm_source=github&utm_medium=readme&utm_campaign=cloud)**
- ## ⚡ Quickstart
- One container. No control plane, no database, no configuration store — the gateway reads
- every dynamic resource from one declarative `resources.yaml`.
- ```yaml
- # config.yaml
- resources_file: /etc/aisix/resources.yaml
- proxy:
- addr: "0.0.0.0:3000"
- admin:
- enabled: false # a declarative gateway needs no admin listener
- observability:
- metrics:
- prometheus:
- enabled: true
- addr: "0.0.0.0:9090"
- ```
- ```yaml
- # resources.yaml — every entry is an id plus the resource's document
- _format_version: "2"
- provider_keys:
- - id: 0b6f3c1e-2f0a-4c5e-9d55-7d1a8b0c2e11
- display_name: openai-main
- provider: openai
- api_key: ${OPENAI_API_KEY} # interpolated from the environment
- models:
- - id: 6efd592d-a73b-573d-8e2c-e9e103692f92
- display_name: my-model
- provider: openai
- model_name: gpt-4o-mini
- provider_key_id: 0b6f3c1e-2f0a-4c5e-9d55-7d1a8b0c2e11
- api_keys:
- - id: 4a1c9e2b-7d3f-4b6a-8c5e-1f2a3b4c5d6e
- display_name: local-dev
- key_hash: ${CALLER_API_KEY_SHA256} # sha256 hex of the caller key; the plaintext is never stored
- allowed_model_ids: ["6efd592d-a73b-573d-8e2c-e9e103692f92"]
- ```
- ```bash
- export OPENAI_API_KEY="YOUR_PROVIDER_KEY"
- export CALLER_API_KEY="YOUR_CALLER_KEY"
- export CALLER_API_KEY_SHA256="$(printf %s "$CALLER_API_KEY" | sha256sum | cut -d' ' -f1)"
- docker run -d --name aisix \
- -v "$(pwd)/config.yaml:/etc/aisix/config.yaml:ro" \
- -v "$(pwd)/resources.yaml:/etc/aisix/resources.yaml:ro" \
- -e OPENAI_API_KEY -e CALLER_API_KEY_SHA256 \
- -p 3000:3000 -p 127.0.0.1:9090:9090 \
- ghcr.io/api7/aisix:latest # proxy → :3000, metrics + status → :9090
- # ^ the metrics/status listener is unauthenticated;
- # keep it on loopback or a private network
- ```
- Then call the gateway exactly like OpenAI:
- ```bash
- curl http://localhost:3000/v1/chat/completions \
- -H "Authorization: Bearer $CALLER_API_KEY" \
- -H "Content-Type: application/json" \
- -d '{"model":"my-model","messages":[{"role":"user","content":"hello"}]}'
- ```
- Edit `resources.yaml` and send `SIGHUP` (`docker kill -s HUP aisix`) to apply changes with
- no restart. The file is loaded exactly as etcd is: a resource that does not load is dropped
- and reported on `/status/config` while the rest serve, and a file that cannot be read at all
- keeps the last configuration serving. Check a file before booting with
- `aisix validate --resources resources.yaml`.
- Full walkthrough: the
- [Gateway Quickstart](https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart) ·
- every field: the [resources file reference](https://docs.api7.ai/ai-gateway/reference/resources-file)
- (still being updated for format `"2"`; each entry's fields are the resource's document, see
- [`schemas/resources/`](schemas/resources/)).
- For a multi-replica cluster, point the gateway at etcd instead — `resources_file` and
- `etcd` are mutually exclusive.
- ## ✨ Why AISIX
- - **One API, every model.** Speak the OpenAI *or* Anthropic wire format in; the gateway
- translates to whichever provider each model points at. Point an OpenAI or Claude SDK at
- one `base_url` and switch models without changing code.
- - **A real gateway, in Rust.** Single static binary, low cold-start, lock-free config reads
- on the hot path, native streaming.
- - **Open source, free forever.** Apache-2.0 licensed and built to run in your
- infrastructure. Choose AISIX Enterprise or AISIX Cloud when you want centralized
- management through a control plane and dashboard.
- - **Production controls built in.** Routing & failover, rate limits, guardrails, caching,
- and observability ship in the box. (Budgets and spend caps are AISIX Enterprise and
- AISIX Cloud features — the gateway enforces the control plane's decisions.)
- ## 🧩 Features — available today
- Covered by 183 end-to-end scenario files (496 cases) that run against real gateway processes.
- - **OpenAI-compatible proxy** (`:3000`) — `chat/completions`, `completions`, `responses`,
- `embeddings`, `rerank`, `images/{generations,edits}`, `audio/{speech,transcriptions,translations}`,
- `videos` (submit → poll → fetch), `files`, `batches`, `fine_tuning/jobs`, `realtime`,
- `GET /v1/models`, plus a root-level `/passthrough/:provider/*` escape hatch. Native SSE streaming,
- tool/function calling, JSON mode, vision/multimodal input, and reasoning-content support.
- - **Anthropic Messages API** — `POST /v1/messages` as a first-class route, working against
- **any** configured upstream: requests and responses (including streaming) are translated
- both ways when a model points at a non-Anthropic provider.
- - **Routing & failover** — virtual/routing models with six strategies: `round_robin`
- (smooth weighted round-robin), `consistent_hash` (session affinity keyed by header /
- cookie / API key / client IP), `failover`, plus metric-based `least_cost`,
- `least_latency`, and `least_busy`. Per-target `priority` tiers (active/backup pools),
- retry budgets, cooldowns, tag-conditional targets, and per-attempt timeouts.
- - **Ensemble models** — fan one request out to a panel of models concurrently, then have a
- judge model synthesize a single answer, with a minimum-successful-responses threshold.
- - **Semantic routing** — one virtual model that dispatches by the *meaning* of each
- request: it embeds the prompt, scores it against per-route example utterances, and routes
- to the best match (or a default). See the
- [semantic routing docs](https://docs.api7.ai/ai-gateway/routing/semantic-routing).
- - **Rate limiting & concurrency** — RPS/RPM/RPH/RPD + TPM/TPD + concurrency caps,
- AND-combined across caller keys, models, and policy scopes (`api_key` / `model` / `team` /
- `member` / `team_member`). Counters are per-process by default, or shared across replicas
- with the Redis backend.
- - **Guardrails** — content-policy enforcement on input and output, in-process or through a
- provider: keyword/regex, built-in PII detection and redaction, Presidio, Lakera, OpenAI
- Moderation, AWS Bedrock Guardrails, Azure AI Content Safety (Prompt Shield + text
- moderation), and two Alibaba Cloud services. A block returns `422 content_filter`;
- monitor mode records what would have happened without blocking.
- - **Caching** — exact-match response cache with per-policy TTL and model/key scope matchers;
- memory and Redis backends; cost-saved telemetry on every hit. Separately, **automatic
- prompt caching** can be enabled per direct Anthropic model to inject cache breakpoints, so
- callers get provider-side prompt discounts without changing their requests.
- - **MCP gateway** — front registered upstream MCP servers at `/mcp` with gateway-held
- credentials, per-server tool namespaces, and per-caller access. It serves every
- Streamable HTTP revision from `2025-03-26` through stateless `2026-07-28` without
- downstream sessions. Upstreams use `initialize` by default or `server/discover` with
- `protocol_version: "2026-07-28"`. CI runs the official MCP suite's applicable tools-only
- protocol scenarios. Also exposes a REST API as MCP tools from its OpenAPI description.
- - **A2A agent gateway** — front A2A (Agent-to-Agent) agents at `/a2a/:agent`, serving each
- agent's card with URLs rewritten to the gateway, over JSON-RPC 2.0.
- - **Inbound authentication** — caller API keys (SHA-256 hashed, model allowlists, expiry,
- rotation), or OIDC/JWT bearer tokens validated against registered providers (Entra ID,
- Okta, Google Workspace, or any OIDC issuer) with JWKS caching.
- - **Observability** — Prometheus `/metrics`, structured per-request access logs, usage
- events, OTLP/GenAI span export (Langfuse, Honeycomb, Grafana Cloud, or any OTLP receiver),
- plus dedicated Datadog and Aliyun SLS log exporters and object-storage (S3/GCS/Azure Blob)
- telemetry.
- - **Declarative configuration** — one `resources.yaml` carries all ten resource collections
- (provider keys, models, caller keys, guardrails, MCP servers, A2A agents, cache policies,
- observability exporters, rate-limit policies, OIDC providers) as the same documents the
- gateway reads from etcd, loaded by the same code. `aisix validate` checks a file offline;
- `SIGHUP` reloads it.
- - **Operational endpoints** — `/livez` and `/readyz` on the proxy listener; `/status/config`,
- `/status/ready`, `/status/models`, and Prometheus `/metrics` on a dedicated metrics
- listener (`:9090`). The admin listener (`:3001`) additionally serves a **read-only**
- resource surface, OpenAPI 3 with a Scalar UI, and a playground. Resources are managed
- declaratively — through the `resources_file` (reloaded on SIGHUP) or direct etcd
- writes — not through the admin listener; its former write endpoints were removed.
- ## 🔌 Supported providers
- AISIX dispatches through **five native adapter families** — distinct wire-protocol bridges,
- not one generic relabel. Whatever the upstream protocol, the client-facing API stays
- OpenAI-shaped.
- | Adapter family | Reaches | Wire shape · auth |
- |---|---|---|
- | `openai` | OpenAI **+ any OpenAI-compatible vendor** — DeepSeek, Groq, Mistral, Together, Fireworks, Perplexity, vLLM, Ollama, or self-hosted OpenAI-compatible endpoints | OpenAI chat completions · Bearer |
- | `anthropic` | Anthropic Claude | Anthropic Messages · `x-api-key` |
- | `bedrock` | AWS Bedrock — Anthropic, Meta Llama, Mistral, Cohere, Amazon Titan/Nova, AI21 | Bedrock Converse + `/invoke` · SigV4 |
- | `vertex` | Google Vertex AI (Gemini) | Vertex `:generateContent` · OAuth2 |
- | `azure-openai` | Azure OpenAI | Azure deployments · api-key / Entra ID |
- Plus specialized handling for vendor quirks (e.g. DeepSeek reasoning content) and dedicated
- **rerank / embeddings** vendors (Cohere, Jina). Details in
- [adapter protocol families](https://docs.api7.ai/ai-gateway/providers/adapters).
- ## ☁️ Choose an AISIX Product
- Same gateway binary, same proxy API — in every form the gateway runs in your environment.
- **AISIX Enterprise** adds a customer-hosted commercial control plane. **AISIX Cloud** adds
- an API7-hosted commercial control plane.
- <table>
- <tr>
- <td width="50%" valign="top">
- <img src="assets/console-overview.png" alt="AISIX Enterprise and AISIX Cloud overview — requests, latency p50/p99, error rate and cost today, with a 7-day request-and-cost trend and gateway health" width="100%"><br>
- <sub><b>Overview</b> — traffic, latency, error rate & spend at a glance</sub>
- <br><br>
- <img src="assets/console-models.png" alt="AISIX Enterprise and AISIX Cloud models — alias an upstream LLM per provider with model IDs and per-model rate limits" width="100%"><br>
- <sub><b>Models</b> — one alias per upstream: OpenAI, Anthropic, Bedrock, DeepSeek…</sub>
- <br><br>
- <img src="assets/console-guardrails.png" alt="AISIX Enterprise and AISIX Cloud guardrails — pre-input and post-output content policies that block on violation" width="100%"><br>
- <sub><b>Guardrails</b> — pre-input & post-output policies, block on violation</sub>
- </td>
- <td width="50%" valign="top">
- <img src="assets/console-playground.png" alt="AISIX Enterprise and AISIX Cloud playground — pick a model, set prompts, and read the response with live token and cost metering" width="100%"><br>
- <sub><b>Playground</b> — test any model with live token & cost metering</sub>
- <br><br>
- <img src="assets/console-observability.png" alt="AISIX Enterprise and AISIX Cloud observability exporters — fan out telemetry to OTLP, Datadog and object storage, with per-target delivery health" width="100%"><br>
- <sub><b>Observability</b> — fan out traces & logs to OTLP, Datadog, object storage</sub>
- <br><br>
- <img src="assets/console-budgets.png" alt="AISIX Enterprise and AISIX Cloud budgets — organization and per-environment spend caps with hard-stop and warn-only policies" width="100%"><br>
- <sub><b>Budgets</b> — hard-stop spend caps with warn-only tiers</sub>
- </td>
- </tr>
- </table>
- <p align="center">
- <em>The AISIX Enterprise and AISIX Cloud dashboard — overview metrics, multi-provider models, guardrails, budgets, and observability exporters across all your gateways.</em>
- <br><br>
- <a href="https://aisix-demo.api7.ai/"><b>▶ Try the live dashboard demo — aisix-demo.api7.ai</b></a>
- </p>
- | | AISIX OSS (this repo) | AISIX Enterprise | [AISIX Cloud](https://api7.ai/ai-gateway?utm_source=github&utm_medium=readme&utm_campaign=cloud) |
- |---|---|---|---|
- | License | Apache-2.0 | Commercial | Commercial |
- | Control plane | None | Customer-hosted | API7-hosted |
- | Configuration | Declarative `resources.yaml`, or etcd for a cluster | Dashboard + AISIX Cloud Admin API | Dashboard + AISIX Cloud Admin API |
- | Tenancy | Single instance / namespace | Organization-scoped environments, teams, and members | Organization-scoped environments, teams, and members |
- | Provider keys | `${VAR}` references in `resources.yaml`; plaintext credentials in etcd | Encrypted at rest; operator-triggered in-place rotation | Encrypted at rest; operator-triggered in-place rotation |
- | Budgets | — (rate and token limits only) | Per key, provider, environment, organization, and team | Per key, provider, environment, organization, and team |
- | RBAC and audit | Local admin key; no organization layer | Organization roles, invitations, and audit history | Organization roles, invitations, and audit history |
- | Operations | You operate the gateway | You operate the control plane and gateways | API7 operates the control plane; you operate gateways |
- → **Want centralized governance, budgets, and a dashboard?**
- **[Talk to API7](https://api7.ai/contact?utm_source=github&utm_medium=readme&utm_campaign=cloud)** about
- AISIX Enterprise or AISIX Cloud, or **[book a demo](https://api7.ai/contact?utm_source=github&utm_medium=readme&utm_campaign=demo)**.
- ## 🏗️ Architecture
- A single Cargo workspace; the `aisix-server` crate builds one binary named `aisix` that
- wires the crates together.
- ```text
- crates/
- ├── aisix-core Config, snapshot, resource model, errors
- ├── aisix-etcd Config providers (etcd, resources.yaml) + watch supervisor
- ├── aisix-gateway Hub & bridge, SSE parser, provider trait
- ├── aisix-proxy /v1/*, /mcp, /a2a handlers, routing, middleware
- ├── aisix-admin Read-only resource surface + playground + OpenAPI
- ├── aisix-provider-* openai · anthropic · azure-openai · bedrock · vertex
- ├── aisix-mcp MCP gateway — server registry, tool ACL, transports
- ├── aisix-a2a A2A agent gateway — agent cards, JSON-RPC bridge
- ├── aisix-ratelimit fixed-window + token accounting + concurrency (local | redis)
- ├── aisix-cache memory + redis backends
- ├── aisix-redis shared Redis connection for cache + rate limits
- ├── aisix-guardrails pre/post content-policy hooks
- ├── aisix-obs tracing, metrics, access log, exporters
- └── aisix-server the `aisix` binary — bootstrap + CLI
- ```
- ## 🗺️ Roadmap
- Highlights on the [roadmap](ROADMAP.md); tracked live in
- [issues](https://github.com/api7/aisix/issues):
- - Semantic (embedding-similarity) response caching
- - More observability sinks — Langsmith, Helicone, Slack alerts
- - Prompt templates managed as gateway resources
- - Llama-Guard as a guardrail provider
- Shipped since this list was last written: the MCP gateway, the A2A agent gateway,
- OIDC/JWT inbound auth, Redis-backed distributed rate limiting, and the Lakera, Presidio,
- PII, and OpenAI Moderation guardrails — see **Features** above.
- ## 🛠️ Development
- Prerequisites: the Rust toolchain pinned in `rust-toolchain.toml`. Docker is only needed
- for the tests that exercise etcd, Redis, or provider emulators.
- ```bash
- cargo check --workspace
- cargo fmt --check
- cargo clippy --workspace -- -D warnings
- cargo test --workspace
- # Coverage (matches the CI gate)
- cargo llvm-cov --workspace --lcov --output-path lcov.info
- # Run locally against a resources.yaml (no etcd needed). Copy the Quickstart's two files
- # and change resources_file to the local path, e.g. resources_file: ./resources.yaml
- cargo run -p aisix-server --bin aisix -- --config config.local.yaml
- # Check a resources file without starting a listener
- cargo run -p aisix-server --bin aisix -- validate --resources resources.yaml
- ```
- ## 💬 Community
- - **Discord** — [discord.gg/dUmRZ7Rvf](https://discord.gg/dUmRZ7Rvf)
- - **Issues & discussions** — [github.com/api7/aisix/issues](https://github.com/api7/aisix/issues)
- - **Contributing** — [CONTRIBUTING.md](CONTRIBUTING.md)
- - **Website** — [api7.ai/ai-gateway](https://api7.ai/ai-gateway?utm_source=github&utm_medium=readme)
- If AISIX is useful to you, a ⭐ helps other engineers find it.
- ## 📄 License
- [Apache 2.0](LICENSE).
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.