Skip to content
Back to search

vault

MCP prompt-injection scanning proxy — runtime security for MCP tool responses

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/vaultmcp--vault/badge.svg)](/listings/vaultmcp--vault)

HTML

<a href="/listings/vaultmcp--vault"><img src="/api/listings/vaultmcp--vault/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Wrap any MCP server — no config change to your agent:

# Set a key for full detection (or run without one for L1+L2 offline mode)
export ANTHROPIC_API_KEY=sk-ant-...

# Put Vault in front of any MCP server
npx @aimcpvault/mcp-proxy -- npx -y @modelcontextprotocol/server-filesystem /data

That's it — every tool response is now scanned before your agent sees it. Point your agent at the proxy instead of the server.

**Proof:** 100% TPR (80/80) on a frozen public holdout, 0.0% FPR on 100 benign docs — see [Measured performance](#measured-performance). How the four layers work: [How it works](#how-it-works). No key? Vault runs L1+L2 only — see [Offline mode](#offline-mode-no-api-key).

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 552 lines · 1 finding marked

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Vault — MCP Prompt Injection Firewall
  2. [![Guarded on Robinhood Chain](https://vaultmcp.io/badge/robinhood)](https://robinhoodchain.blockscout.com/address/0x89bf75bccea833fff371fa300f7c885b5c23f103)
  3. **Site:** [vaultmcp.io](https://vaultmcp.io) **·** **X:** [@vaultmcp](https://x.com/vaultmcp) **·** **Repo:** [github.com/vaultmcp/vault](https://github.com/vaultmcp/vault)
  4. Vault is a production prompt-injection firewall for MCP. It intercepts every tool response before your agent reads it and scans through four layers of detection (L0 deterministic decoder, L1 heuristics, L2 embedding similarity, L3 LLM judge).
  5. ## Quickstart
  6. Wrap any MCP server — no config change to your agent:
  7. ```bash
  8. # Set a key for full detection (or run without one for L1+L2 offline mode)
  9. export ANTHROPIC_API_KEY=sk-ant-...
  10. # Put Vault in front of any MCP server
  11. npx @aimcpvault/mcp-proxy -- npx -y @modelcontextprotocol/server-filesystem /data
  12. ```
  13. That's it — every tool response is now scanned before your agent sees it. Point your agent at the proxy instead of the server.
  14. **Proof:** 100% TPR (80/80) on a frozen public holdout, 0.0% FPR on 100 benign docs — see [Measured performance](#measured-performance). How the four layers work: [How it works](#how-it-works). No key? Vault runs L1+L2 only — see [Offline mode](#offline-mode-no-api-key).
  15. ---
  16. ## Measured performance
  17. All numbers below are reproducible from the harness committed in this repo. Each is footnoted with the commit and dataset it was measured on.
  18. | metric | value | source |
  19. |---|---|---|
  20. | TPR — v2 holdout (L3 enabled, 80 attacks) | **100%** (80 / 80) · 95.5%+ lower bound at 95% CI | [^holdout-l3] |
  21. | TPR — L1+L2 only, no API key | materially lower (L3 is required for production) | [^holdout-degraded] |
  22. | False positive rate (benign flagged) | **0.0%** (0 / 100) | [^holdout-l3] |
  23. | L1 latency (p50 / p99) | 0.02 ms / 0.04 ms | [^latency] |
  24. | L2 latency (p50 / p99) | 8.86 ms / 15.11 ms | [^latency] |
  25. | L3 latency (per call) | ~1.5–2 s (provider round-trip) | [^latency] |
  26. | Sustained throughput (single proxy) | 100 req/s, 0 errors over 30,000 requests | [^load] |
  27. | Steady-state memory (RSS) | 135–180 MB after warmup, no leak observed | [^load] |
  28. | 500 KB response scan time | ~25 s (embedder-bound, see LIMITATIONS §14) | [^edge] |
  29. [^holdout-l3]: `packages/eval/results/eval-clean-baseline-v2-L3-enabled-2026-05-21.md` — one-shot eval against v2 holdout constructed after detection code was frozen at commit `8d230e4`. Dataset: `packages/eval/datasets/holdout-v2-novel/` (50 attacks) + `packages/eval/datasets/holdout-v2-paraphrase/` (30 attacks) + `packages/eval/datasets/benign-v2/` (100 entries). L3 enabled (Anthropic Haiku 4.5). No tuning occurred between holdout construction and eval run.
  30. [^holdout-degraded]: Without `ANTHROPIC_API_KEY`, Vault runs L1+L2 only. On the v2 novel holdout (attacks designed to sit outside L2's detection range) TPR is near 0%; on paraphrase attacks it is ~7%. L3 is the detection backbone for out-of-distribution attacks. See `packages/LIMITATIONS.md` §11 for offline-mode limitations.
  31. [^load]: `packages/eval/load/report.md` — 100 req/s × 300 s × single stdio proxy instance × ~200-byte stub-MCP responses, L1+L2 only. Past 100 req/s the cliff is not yet measured.
  32. [^latency]: `packages/eval/results/latency-baseline-2026-07-20.md` — L1/L2 measured on-device over 1,664 samples (public-corpus attacks + benign tool responses) by `packages/eval/src/latency-bench.ts`; reproduce with `cd packages/eval && npx tsx src/latency-bench.ts`. Hardware-dependent (measured on Apple Silicon). L3 is a single provider round-trip (Anthropic Haiku 4.5) — network-bound, not independently re-measured in this run.
  33. [^edge]: `packages/proxy/test/edge-cases.test.ts` scenario 4. The latency is bounded by the L2 embedder iterating ~140 streaming chunks; L1 stays sub-millisecond at any size. See LIMITATIONS §14.
  34. ---
  35. ## How it works
  36. Vault sits between your agent host and the upstream MCP server, intercepting every `tools/call` response and running it through a layered detection pipeline:
  37. ```
  38. Agent ──► Vault Proxy ──► MCP Server
  39. │
  40. ▼
  41. ┌─────────────────────────────────────────────┐
  42. │ L1 Regex (<1ms) heuristics + unicode │
  43. │ L2 Embeddings (~8ms) bge-small cosine sim │
  44. │ L3 LLM judge (~1s) haiku-4.5 disambiguation│
  45. └─────────────────────────────────────────────┘
  46. │
  47. ▼ clean → forward malicious → block/warn
  48. ```
  49. **Layer 1 — Heuristics** (see [Measured performance](#measured-performance)): Regex patterns for English instruction overrides, unicode tag smuggling (U+E0000–U+E007F), bidi-control characters (U+202A–U+202E, U+2066–U+2069), zero-width character density, HTML comment injection, long HTML-entity runs, and markdown link anchors. High-confidence matches short-circuit — no L2/L3 cost. L1 is English-only; see LIMITATIONS §1.
  50. **Layer 2 — Embeddings** (see [Measured performance](#measured-performance)): Cosine similarity against a curated corpus of 31 attack categories using `bge-small-en-v1.5` (runs entirely on-device, no network call, ~30 MB WASM). Matches within distance 0.35 block; borderline cases escalate to L3. The corpus is intentionally public; adversaries who paraphrase past distance 0.35 evade L2 — that class is for L3.
  51. **Layer 3 — LLM Judge** (~1.5–2 s per call, provider round-trip): Claude Haiku 4.5 (or GPT-4o-mini with OpenAI key) resolves ambiguous cases. Only runs when L2 is uncertain; the escalation rate depends on your traffic (see [Cost transparency](#cost-transparency)). Requires `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`. **Without a key, L3 is disabled and TPR drops to the L1+L2 number reported above.**
  52. ### Capability Firewall
  53. Beyond injection detection, Vault tracks taint: tool responses that entered the agent's context are marked, and if those tokens later appear in calls to sensitive tools (network requests, file writes, shell execution), the call is gated.
  54. ```bash
  55. VAULT_CAPABILITY=1 # enable taint tracking + gate
  56. VAULT_CAPABILITY_MODE=block # or: warn
  57. ```
  58. ### Manifest Verification
  59. Vault fingerprints the tool manifest of every MCP server on first connect and alerts on any subsequent drift — new tools, changed schemas, version bumps. Supply-chain protection against a compromised server silently adding a `delete_all` tool.
  60. ```bash
  61. VAULT_MANIFEST_CHECK=on # default — warn on drift
  62. VAULT_MANIFEST_CHECK=strict # treat drift as error
  63. ```
  64. ### Optional: on-chain attestation (opt-in)
  65. When an operator opts in (`VAULT_ATTEST=1` + a funded hot wallet), scan verdicts can be attested on-chain via [EAS](https://attest.sh) on Base — currently **Base Sepolia** (testnet); Base mainnet is pending. Each MCP server then accumulates a public, append-only reputation score any agent can query before connecting. Off by default — no chain dependency for using the proxy.
  66. > A continuous attestation feed and public reputation registry are planned for **v0.3**. The proxy supports opt-in attestation today for operators who want it; the always-on hosted attester ships alongside v0.3 once we have real install traffic to back the numbers.
  67. ---
  68. ## What Vault catches
  69. With L3 enabled (Anthropic API key set), Vault catches 100% of attacks in our 80-entry public structural-generalization eval, at 0% false positive rate on 100 benign documents. Detection methodology and datasets are public — see `/packages/eval/`.
  70. Breakdown by category (80 attacks, holdout-v2 structural-generalization eval, 2026-05-21):
  71. | Attack category | Caught / Total | TPR |
  72. |---|---|---|
  73. | exfiltration | 60 / 60 | 100.0% |
  74. | instruction_override | 13 / 13 | 100.0% |
  75. | multi_turn_setup | 6 / 6 | 100.0% |
  76. | encoded_payload | 1 / 1 | 100.0% |
  77. Numbers are from `packages/eval/results/eval-clean-baseline-v2-L3-enabled-2026-05-21.md` with L3 enabled (Anthropic Haiku 4.5). Dataset: `packages/eval/datasets/holdout-v2-novel/` (50 attacks) and `packages/eval/datasets/holdout-v2-paraphrase/` (30 attacks), verified non-overlapping with the detection corpus. See [Measured performance](#measured-performance) for latency and throughput context.
  78. ---
  79. ## What Vault does NOT catch
  80. - **User-initiated jailbreaks** — out of scope by design. Vault sits between the agent and the upstream MCP server, not between the user and the agent. Jailbreaks typed directly by the user are the model provider's responsibility, not a proxy's.
  81. - **Genuinely novel injection patterns** — attacks that paraphrase far from all known examples (L2 cosine distance > 0.50 and no L3 key set) evade detection. The corpus covers published 2022–2024 attack literature; novel post-cutoff techniques may not be represented. See [`packages/LIMITATIONS.md`](packages/LIMITATIONS.md) §3.
  82. - **Protocol-encoded data without L3** — MCP tools that return Pub/Sub messages, SQS payloads, binary blobs, or encrypted content will produce false positives when L3 is disabled (~40% FP rate on that traffic class). See LIMITATIONS §11.
  83. - **Multi-turn attacks split across sessions** — Vault scans each MCP response independently. A sleeper directive established in session A and activated in session B is outside Vault's detection window. See LIMITATIONS §10.
  84. - **Image/audio embedded instructions** — Vault is text-only. Binary content in tool responses (images, audio files, PDFs) is forwarded to the agent unscanned. If your MCP server returns image data or vision-model inputs, Vault provides no protection for instructions hidden in that content.
  85. For the complete gap taxonomy: [`packages/LIMITATIONS.md`](packages/LIMITATIONS.md) — 15 sections covering multilingual, semantic, structural, normalization, self-referential, protocol-encoding, and scale gaps. [`packages/SECURITY_MODEL.md`](packages/SECURITY_MODEL.md) maps each gap to the threat it leaves open.
  86. ---
  87. ## Offline mode (no API key)
  88. When `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` are both unset, Vault runs in offline mode:
  89. - Layer 0 (decoder) and Layer 1 (heuristics) operate normally
  90. - Layer 2 (embedding similarity) operates normally
  91. - Layer 3 (LLM judge) is disabled
  92. **Suitable for:**
  93. - Development environments and local testing
  94. - CI/CD pipelines where the agent isn't running production traffic
  95. - Air-gapped or offline deployments
  96. **NOT suitable for:**
  97. - Production MCP servers returning Pub/Sub, SQS, or webhook payloads
  98. - Tools that return binary content (images, encrypted blobs)
  99. - Any deployment where false positive rate above 5% is unacceptable
  100. See [LIMITATIONS §11](packages/LIMITATIONS.md) for specific FP measurements and rationale.
  101. ---
  102. ## Cost transparency
  103. Layer 3 (LLM judge) calls Anthropic Haiku 4.5 only on responses that fall in L2's uncertain zone, so cost scales with the **L3 escalation rate** — there is no single fixed per-request cost. The formula:
  104. **cost / request ≈ L3_rate × ~$0.0005** (Haiku 4.5 input + small output at current pricing).
  105. | L3 escalation rate | cost / request | cost / 1,000 requests |
  106. |---|---|---|
  107. | 5% | ~$0.000025 | ~$0.025 |
  108. | 20% | ~$0.0001 | ~$0.10 |
  109. | 40% | ~$0.0002 | ~$0.20 |
  110. On the adversarial eval holdout, escalation was **~91%** — by construction: every entry is built to sit in L2's uncertain zone. **Real-traffic escalation rate is unmeasured** until we have install telemetry; plug your observed rate into the formula above rather than trusting a guessed number.
  111. - **$0 without an API key** — L1+L2 only. TPR drops to near 0% on novel out-of-distribution attacks and the FP rate on protocol-encoded traffic rises to ~40%. See [Offline mode](#offline-mode-no-api-key) above. The eval harness refuses to run in this mode unless `--allow-degraded` is passed explicitly.
  112. The high L3 rate in our eval (91%) reflects an adversarial dataset — nearly every entry is an attack that lands in L2's uncertain zone by design. Real MCP traffic (mostly clean tool output) has a much lower L3 call rate.
  113. Set `ANTHROPIC_API_KEY` before running. Without it, Vault emits a `WARNING: Layer 3 unavailable — degraded mode` message on startup and falls back to L1+L2 only.
  114. ---
  115. ## Why
  116. Every MCP tool response is a direct write into your agent's context window. A malicious file, a compromised API, or a poisoned search result can redirect your agent mid-task — exfiltrating secrets, overwriting files, or pivoting to other tools. MCP has no transport-level protection against this class. Vault adds a layered detection pass; whether that pass is sufficient for your threat model depends on the gaps documented in [`packages/LIMITATIONS.md`](packages/LIMITATIONS.md).
  117. ## Requirements
  118. Vault requires an LLM for Layer 3 in production. Three options:
  119. - **Anthropic** (`claude-haiku-4-5-20251001`, recommended) — set `ANTHROPIC_API_KEY`
  120. - **OpenAI-compatible** (`gpt-4o-mini`, or self-hosted via vLLM/llama.cpp) — set `OPENAI_API_KEY`
  121. - **Ollama** (local, air-gapped) — set `OLLAMA_HOST=http://localhost:11434`
  122. Without any of the above, Vault runs in offline mode (L1+L2 only). Offline mode has documented limitations — see [LIMITATIONS §11](packages/LIMITATIONS.md).
  123. ### Offline operation with Ollama
  124. Ollama lets you run L3 locally with no cloud dependency — useful for air-gapped environments, development without an API key, and cost-sensitive pipelines.
  125. ```bash
  126. # 1. Install Ollama and pull a model (one-time)
  127. curl -fsSL https://ollama.ai/install.sh | sh
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 193 on GitHub
  128. ollama pull llama3.2:3b
  129. # 2. Run Vault with Ollama as the L3 backend
  130. export OLLAMA_HOST=http://localhost:11434
  131. npx @aimcpvault/mcp-proxy -- npx -y @modelcontextprotocol/server-filesystem /data
  132. ```
  133. To use a different model or a remote Ollama instance:
  134. ```bash
  135. export VAULT_LAYER3_PROVIDER=ollama
  136. export VAULT_LAYER3_MODEL=mistral:7b
  137. export VAULT_LAYER3_BASE_URL=http://gpu-box:11434
  138. ```
  139. **Caveats:**
  140. - Measured TPR numbers in this README are for Anthropic Haiku. Local 3B models vary; run the eval harness before relying on them in production. Anecdotally, `llama3.2:3b` has missed subtle role-hijack and multi-turn-setup attacks in smoke tests where Haiku catches them.
  141. - Reachability is verified on the first scan, not at startup. If Ollama isn't running, Vault logs `vault: layer-3 failed (...)` per request and falls back to L1+L2.
  142. - **Latency:** the default L3 timeout is 15s, sized to absorb the cold-start latency of local 3B models on CPU. If you're running on slower hardware or larger models, increase it further via `VAULT_LAYER3_TIMEOUT_MS=30000`. The first scan after Ollama loads the model can take much longer than steady-state — subsequent calls typically complete in 1–2s.
  143. - **Security:** the default URL is `http://localhost:11434`. If you set `VAULT_LAYER3_BASE_URL=http://0.0.0.0:11434` or point to a remote host, tool response content (which may include sensitive data) is sent over the network to that host. Only point to a trusted, local-network Ollama instance.
  144. ---
  145. ## On-chain reputation inspector
  146. `vault inspect` reads your Claude Desktop MCP config and reports the on-chain reputation Vault has accumulated for each server you have configured.
  147. ```bash
  148. npx @aimcpvault/mcp-proxy inspect
  149. # vault inspect (3 MCP servers)
  150. # config: /Users/you/Library/Application Support/Claude/claude_desktop_config.json
  151. # network: Sepolia (testnet)
  152. # contract: 0x3A977E4D8BA43367cc41BB4695feFF4615fec189
  153. #
  154. # TRUSTED filesystem [stdio:npx:@modelcontextprotocol/server-filesystem]
  155. # score=0.980 scans=412 blocks=2 maliciousRate=0.5%
  156. # NEW git [stdio:uvx:mcp-server-git]
  157. # score=1.000 scans=0 blocks=0 maliciousRate=0.0%
  158. # UNTRUSTED sketchy [stdio:npx:some-untrusted-pkg]
  159. # score=0.400 scans=83 blocks=14 maliciousRate=16.9%
  160. ```
  161. **Trust thresholds (transparent, no magic numbers):**
  162. - `TRUSTED` score ≥ 0.95 AND totalScans ≥ 100
  163. - `UNTRUSTED` maliciousRate ≥ 0.10
  164. - `CAUTION` totalScans ≥ 10 AND maliciousRate ≥ 0.01
  165. - `NEW` totalScans < 10
  166. **Flags:**
  167. - `--config <path>` override the Claude Desktop config path
  168. - `--rpc <url>` and `--contract <addr>` point at a different network/deployment
  169. - `--json` one JSON record per server (suitable for CI / scripts)
  170. - `--strict` exit code 1 if any server is UNTRUSTED — useful in CI checks
  171. The reputation contract today is on **Base Sepolia (testnet)**. Mainnet deployment is pending; the inspector will continue to default to Sepolia until mainnet is live, with a yellow warning printed on each run.
  172. ---
  173. ## Local scan history (opt-in)
  174. Vault can persist every scan to a local SQLite database (`~/.vault/scans.db`) so you can review what was blocked, search by tool/server/verdict, and view a local dashboard.
  175. **Off by default.** Enable with `VAULT_PERSIST=1`:
  176. ```bash
  177. VAULT_PERSIST=1 npx @aimcpvault/mcp-proxy -- npx -y @modelcontextprotocol/server-filesystem /data
  178. # vault: persisting scan history to /Users/you/.vault/scans.db (set VAULT_PERSIST=0 to disable)
  179. ```
  180. **Query from the CLI:**
  181. ```bash
  182. npx @aimcpvault/mcp-proxy history # last 50
  183. npx @aimcpvault/mcp-proxy history --verdict malicious # only blocks
  184. npx @aimcpvault/mcp-proxy history --since 7d --json # last week, JSON
  185. npx @aimcpvault/mcp-proxy history --server stdio:npx:@modelcontextprotocol/server-filesystem
  186. ```
  187. **Browse the dashboard:**
  188. ```bash
  189. npx @aimcpvault/mcp-proxy dashboard
  190. # vault dashboard: http://127.0.0.1:9876
  191. ```
  192. Dark-themed single-page dashboard with verdict cards, per-day stacked bars (30 days), top tools, and a recent-scans table. Auto-refreshes every 5s (configurable via `--refresh <sec>`).
  193. **Privacy guarantees:**
  194. - Off by default — no data is written unless `VAULT_PERSIST=1` is set.
  195. - All data stays local. The dashboard and history reader both serve `~/.vault/scans.db` directly; nothing is uploaded.
  196. - Content previews are passed through a regex redactor before storage (API keys, GitHub/AWS tokens, JWTs, emails, SSN-shaped strings, credit-card-shaped numbers, generic `password:` / `token:` headers). This is a best-effort filter; if you need stronger guarantees, leave persistence off.
  197. - 30-day rolling retention. Override with `VAULT_RETENTION_DAYS=<n>` (set to `0` to disable purging).
  198. - Override the DB path with `VAULT_PERSIST_PATH=/path/to/scans.db`.
  199. ---
  200. ```bash
  201. # Wrap any MCP server — zero config change to your agent
  202. npx @aimcpvault/mcp-proxy -- npx -y @modelcontextprotocol/server-filesystem /path/to/data
  203. ```
  204. ---
  205. ## Install
  206. ```bash
  207. # npm / npx (no install required)
  208. npx @aimcpvault/mcp-proxy -- <your MCP server command>
  209. # or install globally
  210. npm install -g @aimcpvault/mcp-proxy
  211. mcp-proxy -- npx -y @modelcontextprotocol/server-filesystem /data
  212. ```
  213. ### Claude Code / Claude Desktop integration
  214. The fastest way is `vault init` — it auto-detects your existing MCP configs and wraps each server with the proxy:
  215. ```bash
  216. npx @aimcpvault/mcp-proxy && vault-init
  217. ```
  218. `vault init` shows a diff preview before writing, backs up the original config to `<config>.vault-backup`, and is idempotent (re-running skips already-wrapped servers). To revert: `vault-init unwrap`.
  219. Or wrap by hand:
  220. ```json
  221. // ~/.claude/mcp_settings.json (or claude_desktop_config.json)
  222. {
  223. "mcpServers": {
  224. "filesystem": {
  225. "command": "npx",
  226. "args": [
  227. "@aimcpvault/mcp-proxy",
  228. "--",
  229. "npx", "-y", "@modelcontextprotocol/server-filesystem", "/path"
  230. ]
  231. }
  232. }
  233. }
  234. ```
  235. ### Check a server's reputation
  236. The `vault-check` binary ships inside the proxy package — no separate install needed:
  237. ```bash
  238. # already installed if you have @aimcpvault/mcp-proxy
  239. npx --package=@aimcpvault/mcp-proxy@next vault-check stdio:npx:@modelcontextprotocol/server-filesystem
  240. npx --package=@aimcpvault/mcp-proxy@next vault-check --all # scores every server in your MCP config(s)
  241. npx --package=@aimcpvault/mcp-proxy@next vault-check --json | jq .
  242. ```
  243. A standalone `vault-check` binary (Homebrew tap, curl-piped install script) is planned for **v0.3** once we ship signed releases on GitHub.
  244. Reputation comes from EAS attestations on Base (currently Base Sepolia testnet; mainnet pending), aggregated across every Vault deployment that scans the same server. Score range 0–1000 (higher = safer). The opt-in attestation path is documented above under "Optional: on-chain attestation"; a continuous public feed lands with v0.3.
  245. ### Add a reputation badge to your MCP server's README
  246. If you maintain an MCP server, embed its live Vault reputation score as a badge — same shape as shields.io. The badge updates automatically as new attestations land on-chain.
  247. ```markdown
  248. ![Vault Score](https://vaultmcp.io/badge/your-server-name.svg)
  249. ```
  250. Replace `your-server-name` with the identifier Vault uses for your server — typically the package name (e.g. `@modelcontextprotocol/server-filesystem`) for npm-launched servers, or the full URL for HTTP/SSE servers. Servers with no attestations yet render as a neutral "unranked" badge.
  251. ### Public reputation API
  252. CORS-open, no-auth read endpoints backed by the same on-chain data:
  253. | Endpoint | Returns |
  254. |---|---|
  255. | `GET https://vaultmcp.io/api/score/:server` | Single server's score, scans, blocks, basescan link |
  256. | `GET https://vaultmcp.io/api/leaderboard?n=10` | Top N servers by scan count |
  257. | `GET https://vaultmcp.io/api/threats/recent?n=20` | Recent ThreatRecord attestations |
  258. | `GET https://vaultmcp.io/badge/:server.svg` | SVG reputation badge |
  259. All endpoints accept an optional `?network=base|base-sepolia` query parameter. Cache headers are set; expect 60s edge cache.
  260. ### HTTP/SSE mode
  261. ```bash
  262. # Proxy a remote MCP server over HTTP
  263. npx @aimcpvault/mcp-proxy --transport http \
  264. --upstream https://mcp.example.com/v1 \
  265. --port 8800
  266. ```
  267. ---
  268. ## Configuration
  269. All configuration is via environment variables:
  270. ```bash
  271. # Detection
  272. VAULT_MODE=block # block (default) | warn | log
  273. VAULT_LAYER2_THRESHOLD=0.35 # cosine distance cutoff for L2
  274. VAULT_LAYER3_PROVIDER=anthropic # anthropic | openai | custom
  275. VAULT_LAYER3_MODEL=claude-haiku-4-5-20251001 # model override
  276. VAULT_LAYER3_TIMEOUT_MS=5000 # judge call timeout
  277. # Keys (BYO — Vault never stores or forwards them)
  278. ANTHROPIC_API_KEY=sk-ant-...
  279. OPENAI_API_KEY=sk-...
  280. # Capability firewall
  281. VAULT_CAPABILITY=1
  282. VAULT_CAPABILITY_MODE=block
  283. VAULT_TAINT_MIN_OVERLAP=32
  284. VAULT_SENSITIVE_TOOL_PATTERNS=^my_custom_sensitive_tool
  285. # Manifest verification
  286. VAULT_MANIFEST_CHECK=on # on | off | strict
  287. # Audit log
  288. VAULT_AUDIT_LOG=/var/log/vault-mcp.jsonl
  289. # On-chain attestation (opt-in, off by default — full hosted feed lands in v0.3)
  290. VAULT_ATTEST=1
  291. VAULT_ATTESTER_PRIVATE_KEY=0x... # fund with ~0.05 ETH on Base
  292. VAULT_EAS_ADDRESS=0x4200000000000000000000000000000000000021
  293. VAULT_SCAN_RECEIPT_SCHEMA=0x... # register via packages/contracts
  294. VAULT_THREAT_RECORD_SCHEMA=0x...
  295. # Telemetry (default on when URL set)
  296. VAULT_TELEMETRY=1
  297. VAULT_TELEMETRY_URL=https://your-collector.example.com/ingest
  298. VAULT_TELEMETRY=0 # opt out
  299. ```
  300. ---
  301. ## Audit log
  302. Every verdict is written to an append-only JSONL file:
  303. ```bash
  304. VAULT_AUDIT_LOG=/var/log/vault-mcp.jsonl
  305. # View with the built-in CLI
  306. npx vault-audit /var/log/vault-mcp.jsonl
  307. npx vault-audit --type detection --verdict malicious
  308. npx vault-audit --tool read_file --since 1h
  309. npx vault-audit --raw | jq .
  310. ```
  311. ---
  312. ## Detection modes
  313. | Mode | Behavior |
  314. |---------|-------------------------------------------------------------|
  315. | `block` | Malicious content is replaced with an error response (default) |
  316. | `warn` | Original content is returned with a warning prepended |
  317. | `log` | Content passes through; detection recorded in audit log only |
  318. ---
  319. ## Privacy
  320. Vault never sends raw content anywhere. The telemetry pipeline transmits only SHA-256 hashes of content and arguments, verdict labels, latency measurements, and pattern names. Raw text never leaves the proxy process.
  321. See [PRIVACY.md](packages/proxy/PRIVACY.md) for the full data inventory.
  322. To disable telemetry entirely: `VAULT_TELEMETRY=0`.
  323. ---
  324. ## Project layout
  325. ```
  326. packages/
  327. proxy/ # @aimcpvault/mcp-proxy — the core proxy (this is what you install)
  328. corpus/ # @vaultmcp/corpus — curated attack/clean embedding corpus
  329. contracts/ # @vaultmcp/contracts — VaultReputation.sol + EAS schema registration
  330. collector/ # @vaultmcp/collector — telemetry ingest + aggregation server
  331. eval/ # @vaultmcp/eval — detection benchmarks vs. competitors
  332. demo-site/ # Next.js demo + live threat feed
  333. ```
  334. ---
  335. ## Eval methodology — how to reproduce
  336. The [Measured performance](#measured-performance) numbers above come from the harness and datasets committed in this repo. To reproduce:
  337. ```bash
  338. # Clone and install
  339. git clone https://github.com/vaultmcp/vault.git
  340. cd vault
  341. pnpm install
  342. # Run the eval (L1+L2 only, no API key required)
  343. pnpm --filter @vaultmcp/eval run eval -- --set both
  344. # To include L3, export an API key first
  345. export ANTHROPIC_API_KEY=sk-ant-...
  346. pnpm --filter @vaultmcp/eval run eval -- --set both
  347. ```
  348. Outputs land in `packages/eval/results/eval-<timestamp>.{md,json}`. Each run prints TPR, FPR, per-category breakdown, per-layer attribution, latency percentiles, and the worst false negatives and false positives by severity.
  349. The headline numbers come from the v2 holdout, constructed after detection code was frozen: `packages/eval/datasets/holdout-v2-novel/` (50 novel attacks) + `packages/eval/datasets/holdout-v2-paraphrase/` (30 paraphrased attacks) = 80 attacks, plus `packages/eval/datasets/benign-v2/` (100 benign entries). Each has a `MANIFEST.md` describing provenance. (The earlier v1 set, `packages/eval/datasets/holdout-attacks-burned-v1/`, is retired — it was used to motivate contaminated work; see the postmortem.)
  350. **Operators are encouraged to author their own attacks and submit pull requests.** Add a new file under `packages/eval/datasets/holdout-v2-novel/` following the existing JSON schema, update the `MANIFEST.md`, and open a PR. The harness picks new files up automatically.
  351. The self red-team (`packages/eval/red-team/`) is the other half of the honest-eval story: 38 hand-crafted bypass attempts, 9 of which still pass L1+L2 after our P2 fixes. They are categorized in [`packages/LIMITATIONS.md`](packages/LIMITATIONS.md).
  352. ---
  353. ## Development
  354. ```bash
  355. # Prerequisites: Node 20+, pnpm 9+, Foundry 1.7+ (for contracts only)
  356. # Install
  357. pnpm install
  358. # Run proxy in dev mode (wraps the MCP filesystem server)
  359. pnpm --filter @aimcpvault/mcp-proxy dev -- npx -y @modelcontextprotocol/server-filesystem /tmp
  360. # Run all tests
  361. pnpm -r test
  362. # Typecheck
  363. pnpm -r typecheck
  364. # Build all packages
  365. pnpm -r build
  366. # Run eval benchmarks
  367. pnpm --filter @vaultmcp/eval run eval
  368. # Contracts (requires Foundry)
  369. cd packages/contracts
  370. forge test
  371. forge build
  372. ```
  373. ---
  374. ## Security
  375. Vault is a defense-in-depth layer, not a complete solution. No regex or embedding model catches every attack — adversaries can craft payloads that evade any single detection strategy. Measured detection on our public v2 holdout is **100% TPR / 0.0% FPR** (80 attacks, L3 enabled, 95.5%+ lower bound at 95% CI). Without L3, TPR is near 0% on novel out-of-distribution attacks — L3 is the detection backbone. The layered approach raises the bar; operators should treat Vault as one layer in a broader security posture.
  376. For details:
  377. - [`packages/LIMITATIONS.md`](packages/LIMITATIONS.md) — measured gaps, red-team evidence, and which mitigations are planned vs. accepted.
  378. - [`packages/SECURITY_MODEL.md`](packages/SECURITY_MODEL.md) — threats Vault defends against, threats it does not, assumptions, and what an attacker still has to do.
  379. - [`SECURITY.md`](SECURITY.md) — vulnerability reporting policy.
  380. To report a vulnerability: open a [GitHub Security Advisory](../../security/advisories/new) (preferred) or email the maintainer directly. We aim to triage within 48 hours and ship a patch within 7 days of a confirmed critical.
  381. ---
  382. ## License
  383. MIT

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.