Skip to content
Back to search

mcp-observatory

CI-native security testing for MCP servers. Attack simulation, schema drift detection, and health scoring before agents depend on them.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/KryptosAI/mcp-observatory with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/KryptosAI/mcp-observatory'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/kryptosai--mcp-observatory/badge.svg)](/listings/kryptosai--mcp-observatory)

HTML

<a href="/listings/kryptosai--mcp-observatory"><img src="/api/listings/kryptosai--mcp-observatory/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Run a first check. No arguments starts `demo` (your servers, or the packaged local demo):

npx @kryptosai/mcp-observatory

Go deeper — also invoke safe tools to verify they actually run:

npx @kryptosai/mcp-observatory scan deep

Test a specific server:

npx @kryptosai/mcp-observatory test npx -y @modelcontextprotocol/server-everything

Add it to Claude Code as an MCP server:

claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve

Or add it manually to your config:

{
  "mcpServers": {
    "mcp-observatory": {
      "command": "npx",
      "args": ["-y", "@kryptosai/mcp-observatory", "serve"]
    }
  }
}

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 628 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <p align="center">
  2. <img src="docs/assets/mcp-observatory-logo.png" alt="MCP Observatory" width="482"/>
  3. </p>
  4. <h1 align="center">MCP Observatory</h1>
  5. [![CI](https://github.com/KryptosAI/mcp-observatory/actions/workflows/ci.yml/badge.svg)](https://github.com/KryptosAI/mcp-observatory/actions/workflows/ci.yml)
  6. [![CodeQL](https://github.com/KryptosAI/mcp-observatory/actions/workflows/codeql.yml/badge.svg)](https://github.com/KryptosAI/mcp-observatory/actions/workflows/codeql.yml)
  7. [![Coverage Workflow](https://github.com/KryptosAI/mcp-observatory/actions/workflows/coverage.yml/badge.svg)](https://github.com/KryptosAI/mcp-observatory/actions/workflows/coverage.yml)
  8. [![npm](https://img.shields.io/npm/v/@kryptosai/mcp-observatory)](https://www.npmjs.com/package/@kryptosai/mcp-observatory)
  9. [![GitHub stars](https://img.shields.io/github/stars/KryptosAI/mcp-observatory?style=flat)](https://github.com/KryptosAI/mcp-observatory/stargazers)
  10. [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE)
  11. <details>
  12. <summary>More badges</summary>
  13. [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/KryptosAI/mcp-observatory/badge)](https://securityscorecards.dev/viewer/?uri=github.com/KryptosAI/mcp-observatory)
  14. [![Dependabot](https://img.shields.io/badge/dependabot-enabled-025e8c?logo=dependabot)](./.github/dependabot.yml)
  15. [![npm provenance workflow](https://img.shields.io/badge/npm%20provenance-workflow-blue)](./.github/workflows/release.yml)
  16. [![npm weekly downloads](https://img.shields.io/npm/dw/@kryptosai/mcp-observatory)](https://www.npmjs.com/package/@kryptosai/mcp-observatory)
  17. [![Node >= 20](https://img.shields.io/badge/node-%3E%3D20-339933)](./package.json)
  18. [![Smithery](https://smithery.ai/badge/@kryptosai/mcp-observatory)](https://smithery.ai/server/@kryptosai/mcp-observatory)
  19. [![mcp-observatory MCP server](https://glama.ai/mcp/servers/KryptosAI/mcp-observatory/badges/score.svg)](https://glama.ai/mcp/servers/KryptosAI/mcp-observatory)
  20. [![All Contributors](https://img.shields.io/badge/all_contributors-8-orange.svg?style=flat-square)](./CONTRIBUTORS.md)
  21. [![Gitee Stars](https://gitee.com/williamweishuhn/mcp-observatory/badge/star.svg)](https://gitee.com/williamweishuhn/mcp-observatory)
  22. [![Gitee Forks](https://gitee.com/williamweishuhn/mcp-observatory/badge/fork.svg)](https://gitee.com/williamweishuhn/mcp-observatory)
  23. [![MCP Registry](https://img.shields.io/badge/MCP_Registry-listed-blue)](https://registry.modelcontextprotocol.io)
  24. [![MCP Market](https://img.shields.io/badge/MCP_Market-premium-gold)](https://mcpmarket.com)
  25. [![MCP Hub China](https://img.shields.io/badge/MCP_Hub_China-listed-red)](https://mcp-hub.cn)
  26. [![OpenTools](https://img.shields.io/badge/OpenTools-listed-green)](https://opentools.ai)
  27. [![Gitee](https://img.shields.io/badge/Gitee-mirror-orange)](https://gitee.com/williamweishuhn/mcp-observatory)
  28. </details>
  29. **Free, open-source MCP security testing from ClearFrame.** MCP Observatory is the CI-native security tool for teams shipping custom MCP servers. Test during development, catch schema drift, simulate attacks, and generate compliance evidence — before agents depend on your servers.
  30. Also available in [Simplified Chinese](README-zh-CN.md).
  31. > **Runtime enforcement:** Use [mcp-seatbelt](https://github.com/KryptosAI/mcp-seatbelt) to block dangerous MCP tool calls at runtime based on observatory scan results.
  32. ## Get Started
  33. **[Step-by-step setup guide](https://mcp-observatory.com/start/)** · macOS, Windows, Linux · Node.js 20+ and npm ([install Node.js LTS](https://nodejs.org/en/download)).
  34. ### 1. Run your first scan
  35. Open Terminal or PowerShell in your project folder and paste:
  36. ```bash
  37. npx -y @kryptosai/mcp-observatory@latest
  38. ```
  39. This downloads Observatory and checks your first configured MCP server. With no configured server, it uses an included example. To try only that example, run `npx -y @kryptosai/mcp-observatory@latest demo --example`.
  40. ### 2. Read the result
  41. The terminal shows the target, grade, findings, and saved receipt path. Failed checks are findings to review, not a request to pay. If connection failed, fix the startup command or credentials and rerun. A demo grade describes the example, not your own server.
  42. ### 3. Review locally, then continue in ClearFrame
  43. ```bash
  44. npx -y @kryptosai/mcp-observatory@latest dashboard
  45. npx -y @kryptosai/mcp-observatory@latest clearframe
  46. ```
  47. The dashboard shows your local scan history, HTML reports, and JSON downloads. No account, upload, subscription, or time-based retention limit. Stop it with Ctrl+C.
  48. The `clearframe` command writes a private instruction file and prints a read-only CLI command. Review the file, then run that command or paste it into a new ClearFrame Mac chat with your project open. It does not launch ClearFrame or upload evidence automatically.
  49. **[Get ClearFrame CLI or the Mac app](https://clearframecode.com/download/?utm_source=mcp-observatory&utm_medium=readme&utm_campaign=opensource)** · Currently Apple Silicon Mac. ClearFrame has its own account and usage terms; all Observatory features work without it.
  50. MCP Observatory is free and MIT licensed. Product telemetry is off by default and makes no telemetry network request before explicit opt-in. Use `telemetry preview` to inspect fields or `telemetry enable` to opt in. [Privacy details](./PRIVACY.md).
  51. ```bash
  52. npx -y @kryptosai/mcp-observatory@latest test npx -y @modelcontextprotocol/server-everything
  53. ```
  54. ```yaml
  55. - uses: KryptosAI/mcp-observatory/action@v1
  56. ```
  57. If this repo is an MCP package, that tests `npx -y <package.json name>`. Otherwise set `command:`.
  58. Or generate a full workflow: `npx -y @kryptosai/mcp-observatory@latest setup-ci --all --command "npx -y my-mcp-server" --sarif`
  59. Install: `npx -y @kryptosai/mcp-observatory@latest` · `brew tap kryptosai/mcp-observatory https://github.com/KryptosAI/mcp-observatory && brew install mcp-observatory` · `docker pull ghcr.io/kryptosai/mcp-observatory:latest` · `curl -fsSL https://raw.githubusercontent.com/KryptosAI/mcp-observatory/main/scripts/install.sh | sh`
  60. ```bash
  61. docker run --rm ghcr.io/kryptosai/mcp-observatory:latest demo
  62. docker run --rm ghcr.io/kryptosai/mcp-observatory:latest test npx -y @modelcontextprotocol/server-everything
  63. ```
  64. Scanning a server on the host needs network and, for local files, a volume mount (`-v "$PWD:$PWD" -w "$PWD"`).
  65. Agents: `npx -y @kryptosai/mcp-observatory@latest serve`
  66. Default-deny connect: `npx -y @kryptosai/mcp-observatory@latest protect` rewrites every discovered client config (Cursor, Claude, VS Code, `.mcp.json`, …). Undo: `protect --undo`. HTTP/SSE servers are left alone.
  67. Runtime deny-default: `npx -y @kryptosai/mcp-observatory@latest enforce --start-proxy` writes a Seatbelt policy from the scan and starts the proxy. Local scan stays free.
  68. Proof: [Safety Index](https://mcp-observatory.com/safety-index/). Local history, evidence export, CI, and remediation task preparation are free.
  69. ## Learn from production usage
  70. Connect real tool calls to user intent with the [usage-learning SDK and CLI](./docs/usage-learning.md): sanitized invocation evidence, opted-in “asked / expected / got” reports, selected conversation excerpts, one-click dissatisfaction flags, intent grouping across captured requests, and regression-case export. Run `mcp-observatory usage review --dir private-usage` to inspect linked evidence in a loopback-only browser screen. Server logs cannot reveal unshared conversations; observed execution and inferred breakdown signals stay separate. Collection is local and requires explicit integration.
  71. ## Why MCP Observatory
  72. MCP servers are becoming production dependencies. If agents rely on them, teams need a way to catch broken tools, unsafe schemas, schema drift, slow responses, and security footguns before those failures reach users.
  73. Observatory gives maintainers and teams:
  74. - **One-command CI setup** with `setup-ci --all`
  75. - **Profile-mapped audits** with `audit --profile nsa-mcp`
  76. - **MCP receipts** that package target, evidence, verdict, action, and reproduction commands
  77. - **MCP risk graphs** that group servers by capability boundary, receipt state, CI posture, and recommended action
  78. - **Action receipts** that say `allow`, `gate`, `rerun`, `quarantine`, or `escalate`
  79. - **GitHub PR comments** for compatibility, drift, and security findings
  80. - **GitHub Code Scanning SARIF** for normalized MCP findings
  81. - **Health score badges** for public trust signals
  82. - **Record/replay/verify** workflows for regression testing
  83. - **MCP server mode** so agents can inspect other MCP servers directly
  84. - **Review and remediation** with free local history, CI, and optional ClearFrame CLI or Mac tasks
  85. See [GitHub Code Scanning for MCP servers](./docs/github-code-scanning-for-mcp.md), [MCP Receipts](./docs/mcp-receipts.md), [Troubleshooting](./docs/troubleshooting.md), [Safety Methodology](./docs/methodology.md), [MCP Server Safety Index](./docs/mcp-server-safety-index.md), [MCP Observatory Contributors](./docs/contributor-recognition.md), [hosted client contract](./docs/api.md), [repository boundary](./docs/repository-boundary.md), [open-source policy](./docs/commercial-boundary.md), and [free project policy](./COMMERCIAL.md).
  86. ### Self-Assessment
  87. We scan ourselves with mcp-observatory on every release. [See results →](docs/self-assessment.md)
  88. ## Self-hosted evidence
  89. The [MIT-licensed hosting source](https://github.com/KryptosAI/mcp-observatory/tree/main/hosting) includes private receipt storage, GitHub sign-in, multi-target history, downloads, hosted HTTP checks and retention cleanup. Deploy it with your own configuration; no Observatory subscription is required. The account-free local dashboard is the fastest start.
  90. ## Free tools for maintainers and teams
  91. All scanner features, source audits, skill scans, package-name review, cross-server capability review, receipts, record/replay, local dashboards, and CI integrations are open source under MIT. No hosted account or private API is needed.
  92. ```bash
  93. # Inspect local code without executing it
  94. npx -y @kryptosai/mcp-observatory@latest source-audit ./src
  95. # Review a launch package name without running it
  96. npx -y @kryptosai/mcp-observatory@latest package-check -- npx -y your-mcp-package
  97. # Review capability combinations across saved receipts
  98. npx -y @kryptosai/mcp-observatory@latest toxic-flow first-receipt.json second-receipt.json
  99. ```
  100. These checks identify review candidates; they do not prove a server is safe. See [coverage and competitor feature evidence](./docs/feature-coverage.md), [ClearFrame workflow](./docs/clearframe.md), and [project policy](./COMMERCIAL.md).
  101. ## How It Compares
  102. | Feature | mcp-observatory | Snyk agent-scan | Cisco mcp-scanner | agent-shield |
  103. |---|---|---|---|---|
  104. | MCP-native | ✓ | ✓ | ✓ | ✓ |
  105. | Attack simulation | ✓ | ✗ | ✗ | ✗ |
  106. | Schema drift detection | ✓ | ✗ | ✗ | ✗ |
  107. | Record/replay/verify | ✓ | ✗ | ✗ | ✗ |
  108. | Health scoring (0-100) | ✓ | ✗ | ✗ | ✗ |
  109. | SARIF output | ✓ | ✓ | ✓ | ✓ |
  110. | CI/CD native (setup-ci) | ✓ | ✓ | ✓ | ✓ |
  111. | Public Safety Index | ✓ | ✗ | ✗ | ✗ |
  112. | Runtime enforcement via mcp-seatbelt | ✓ | ✗ | ✗ | ✗ |
  113. ## Quick Start
  114. Run a first check. No arguments starts `demo` (your servers, or the packaged local demo):
  115. ```bash
  116. npx @kryptosai/mcp-observatory
  117. ```
  118. Go deeper — also invoke safe tools to verify they actually run:
  119. ```bash
  120. npx @kryptosai/mcp-observatory scan deep
  121. ```
  122. Test a specific server:
  123. ```bash
  124. npx @kryptosai/mcp-observatory test npx -y @modelcontextprotocol/server-everything
  125. ```
  126. Add it to Claude Code as an MCP server:
  127. ```bash
  128. claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve
  129. ```
  130. Or add it manually to your config:
  131. ```json
  132. {
  133. "mcpServers": {
  134. "mcp-observatory": {
  135. "command": "npx",
  136. "args": ["-y", "@kryptosai/mcp-observatory", "serve"]
  137. }
  138. }
  139. }
  140. ```
  141. ## Commands
  142. | Command | What it does |
  143. |---------|-------------|
  144. | `scan` | Auto-discover servers, check them, and run safe attack-readiness simulation by default |
  145. | `scan deep` | Scan, run safe attack simulation, and also invoke safe tools to verify they execute |
  146. | `test <cmd>` / `test --target <file>` | Test one server and emit an action receipt by command or target config |
  147. | `record <cmd>` | Record a server session to a cassette file for offline replay |
  148. | `replay <cassette>` | Replay a cassette offline — no live server needed |
  149. | `verify <cassette> <cmd>` | Verify a live server still matches a recorded cassette |
  150. | `diff <base> <head>` | Compare two run artifacts for regressions and schema drift |
  151. | `watch <config>` | Watch a server for changes, alert on regressions |
  152. | `suggest` | Detect your stack and recommend MCP servers from the registry |
  153. | `serve` | Start as an MCP server for AI agents |
  154. | `lock` | Snapshot MCP server schemas into a lock file |
  155. | `lock verify` | Verify live servers match the lock file |
  156. | `history` | Show health score trends for your MCP servers |
  157. | `setup-ci` / `init-ci` | Create a GitHub Action and badge snippet for MCP compatibility/security checks |
  158. | `setup-ci --sarif` | Generate a workflow that uploads normalized findings to GitHub Code Scanning |
  159. | `setup-ci --doctor` | Inspect whether the repository has a complete CI adoption kit |
  160. | `telemetry status\|enable\|disable\|identify\|preview` | Inspect or control product telemetry and deliberately provide a contact email |
  161. | `risk-graph --input <path>` | Merge receipts and run artifacts into JSON, Markdown, and HTML MCP risk graphs |
  162. | `--no-attack-sim` | Opt out of the default safe attack simulation on `scan` or `test` |
  163. | `ci-report` | Generate CI report for GitHub issue creation |
  164. | `enterprise-report` | Generate a static production/security report from run artifacts |
  165. | `score <cmd>` | Score an MCP server's health (0-100) |
  166. | `badge <cmd>` | Generate an SVG health score badge for README |
  167. | `dashboard` | Browse local scan history, reports, and downloads without an account |
  168. | `clearframe` | Prepare a local review task for ClearFrame CLI or Mac |
  169. | `cloud` | Legacy hosted account compatibility and free local alternatives |
  170. Run with no arguments to start `demo`. Use `--menu` for the interactive catalog.
  171. ## What It Does
  172. **Check capabilities** — connects to a server and verifies tools, prompts, and resources respond correctly.
  173. **Invoke tools** — goes beyond listing. Actually calls safe tools (no required params / readOnlyHint) and reports which ones work and which ones crash.
  174. ```bash
  175. npx @kryptosai/mcp-observatory scan deep
  176. ```
  177. **Detect schema drift** — diffs two runs and surfaces added/removed fields, type changes, and breaking parameter changes.
  178. ```bash
  179. npx @kryptosai/mcp-observatory diff run-a.json run-b.json
  180. ```
  181. **Recommend servers** — scans your project for languages, frameworks, databases, and cloud providers, then cross-references the [MCP registry](https://registry.modelcontextprotocol.io) to suggest servers you're missing.
  182. ```bash
  183. npx @kryptosai/mcp-observatory suggest
  184. ```
  185. Or ask your agent "what MCP servers should I add?" when running in MCP server mode.
  186. **Security scanning** — analyzes tool schemas for dangerous patterns: shell injection surfaces, broad filesystem access, missing auth, and credential leakage in responses.
  187. ```bash
  188. npx @kryptosai/mcp-observatory test --security npx -y my-mcp-server
  189. ```
  190. **Record / replay / verify** — capture a live session, replay it offline in CI, and verify nothing changed. Like [VCR](https://github.com/vcr/vcr) for MCP.
  191. ```bash
  192. # Record a session
  193. npx @kryptosai/mcp-observatory record npx -y @modelcontextprotocol/server-everything
  194. # Replay offline (no server needed)
  195. npx @kryptosai/mcp-observatory replay .mcp-observatory/cassettes/latest.cassette.json
  196. # Verify the live server still matches
  197. npx @kryptosai/mcp-observatory verify cassette.json npx -y @modelcontextprotocol/server-everything
  198. ```
  199. **Watch for regressions** — re-runs checks on an interval and alerts when something changes.
  200. ```bash
  201. npx @kryptosai/mcp-observatory watch target.json
  202. ```
  203. ### Scan locations
  204. When you run `scan`, it looks for MCP configs in:
  205. - `~/.claude.json` (Claude Code)
  206. - `~/Library/Application Support/Claude/claude_desktop_config.json` (Claude Desktop, macOS)
  207. - `%APPDATA%/Claude/claude_desktop_config.json` (Claude Desktop, Windows)
  208. - `.claude.json` and `.mcp.json` (current directory)
  209. ## Architecture
  210. ```
  211. ┌─────────────────────────┐
  212. │ MCP Observatory CLI │
  213. │ npx @kryptosai/mcp- │
  214. │ observatory scan │
  215. └───────────┬─────────────┘
  216. │
  217. ┌───────────▼─────────────┐
  218. │ Config Discovery │
  219. │ (Claude, Cursor, etc.) │
  220. └───────────┬─────────────┘
  221. │
  222. ┌─────────────────┼─────────────────┐
  223. ▼ ▼ ▼
  224. ┌─────────────────┐ ┌──────────────┐ ┌──────────────────┐
  225. │ Security Scan │ │ Attack Sim │ │ Schema Drift │
  226. │ (shell, creds) │ │ (tool poison)│ │ (version diff) │
  227. └────────┬────────┘ └──────┬───────┘ └────────┬─────────┘
  228. │ │ │
  229. └─────────────────┼───────────────────┘
  230. ▼
  231. ┌─────────────────────┐
  232. │ Health Score │
  233. │ (0-100 + verdict) │
  234. └──────────┬──────────┘
  235. │
  236. ┌────────────────┼────────────────┐
  237. ▼ ▼ ▼
  238. ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
  239. │ SARIF │ │ Markdown │ │ CI Gateway │
  240. │ (Code Scan) │ │ Report │ │ (setup-ci) │
  241. └──────────────┘ └──────────────┘ └──────────────┘
  242. ```
  243. ## CI / GitHub Action
  244. Add Observatory to your MCP server's CI pipeline:
  245. ```bash
  246. npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server" --sarif --schedule weekly
  247. ```
  248. Check the adoption kit:
  249. ```bash
  250. npx @kryptosai/mcp-observatory setup-ci --doctor
  251. ```
  252. Successful `test`, `run`, and single-target `scan` checks also offer to convert the passing result into a CI adoption kit. That automatic conversion enables SARIF/Code Scanning and weekly scheduled checks by default; pass `--no-ci-sarif` when you only want a conservative workflow without Code Scanning upload.
  253. Or create the workflow manually:
  254. ```yaml
  255. # .github/workflows/observatory.yml
  256. name: MCP Server Check
  257. on: [pull_request]
  258. permissions:
  259. contents: read
  260. jobs:
  261. observatory:
  262. runs-on: ubuntu-latest
  263. steps:
  264. - uses: actions/checkout@v4
  265. - uses: KryptosAI/mcp-observatory/action@v1
  266. with:
  267. command: npx -y my-mcp-server
  268. deep: true
  269. security: true
  270. comment-on-pr: false
  271. set-status: false
  272. ```
  273. ### GitLab CI
  274. Runs the same `test` scan on merge requests and on `main`, publishing the SARIF as a GitLab SAST report so findings appear in the merge request security widget. This matches what `setup-ci --ci-provider gitlab-ci` generates.
  275. ```yaml
  276. # .gitlab-ci.yml
  277. mcp-observatory:
  278. image: node:22
  279. rules:
  280. - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
  281. - if: $CI_COMMIT_BRANCH == 'main'
  282. script:
  283. - npx @kryptosai/mcp-observatory test npx -y my-mcp-server --deep --security --sarif mcp-observatory.sarif
  284. artifacts:
  285. reports:
  286. sast: mcp-observatory.sarif
  287. ```
  288. Or let the CLI write it for you:
  289. ```bash
  290. npx @kryptosai/mcp-observatory setup-ci --ci-provider gitlab-ci --command "npx -y my-mcp-server"
  291. ```
  292. Action inputs:
  293. | Input | Description | Default |
  294. |-------|-------------|---------|
  295. | `command` | Server command to test | (optional; omit to scan discovered servers) |
  296. | `target` | Path to target config JSON | |
  297. | `targets` | Path to MCP config file for multi-server matrix scan | |
  298. | `deep` | Also invoke safe tools | `false` |
  299. | `security` | Run security analysis | `false` |
  300. | `fail-on-regression` | Fail the action on issues | `true` |
  301. | `fail-on-baseline-drift` | Fail the action when baseline verification detects drift | `true` |
  302. | `comment-on-pr` | Post report as PR comment. Requires `pull-requests: write`. | `true` |
  303. | `set-status` | Set a commit status check (green/red) on the HEAD SHA. Requires `statuses: write`. | `true` |
  304. | `github-token` | Token for PR comments and commit statuses | `${{ github.token }}` |
  305. The action can comment on PRs and set commit statuses when the workflow grants write permissions. `setup-ci` generates read-only third-party-friendly workflows by default and lets maintainers opt into comments/statuses later. `init-ci` remains available as a backward-compatible alias. See [`action/README.md`](./action/README.md) for all options.
  306. Observatory is free and open source. [Continue with ClearFrame CLI or Mac](./docs/clearframe.md) when you want an agent to help review and repair findings.
  307. ### Evidence badges for MCP Observatory
  308. MCP server maintainers can add a public compatibility/security signal to their README:
  309. ```md
  310. [![MCP Observatory](https://img.shields.io/badge/MCP%20Observatory-enabled-2563eb)](https://github.com/KryptosAI/mcp-observatory)
  311. ```
  312. Or generate a score badge from a live check:
  313. ```bash
  314. npx @kryptosai/mcp-observatory badge npx -y my-mcp-server --output docs/mcp-health.svg
  315. ```
  316. See the [evidence distribution loop](./docs/certification-distribution.md) for the GitHub Action template, maintainer PR body, and badge rollout playbook. A badge is a public evidence signal, not a certification or endorsement.
  317. Generate a production/security report from local run artifacts:
  318. ```bash
  319. npx @kryptosai/mcp-observatory enterprise-report \
  320. --account "Your Company" \
  321. --format html \
  322. --output observatory-enterprise-report.html
  323. ```
  324. For clearer internal account attribution in CI, set:
  325. ```bash
  326. MCP_OBSERVATORY_ORG=your-company.com
  327. ```
  328. To deliberately associate a contact email and optional contact-channel slug with private telemetry analytics, run `mcp-observatory telemetry identify --email <address> [--channel <slug>]`. Setting an organization or identifying a contact does not authorize outreach.
  329. Testing Feishu/Lark integrations? See the [Feishu/Lark MCP guide](./docs/feishu-lark-mcp.md).
  330. ### Lock Files
  331. ```bash
  332. $ npx @kryptosai/mcp-observatory lock # Snapshot all server schemas
  333. $ npx @kryptosai/mcp-observatory lock verify # Verify no drift since last lock
  334. ```
  335. Lock files are the package-lock for AI tools: commit the MCP contract, then make every tool, schema, prompt, or resource drift visible in CI. See [MCP lock files](./docs/mcp-lock-files.md).
  336. ### Trend Tracking
  337. ```bash
  338. $ npx @kryptosai/mcp-observatory history # Show health trends over time
  339. ```
  340. ### Nightly Scans
  341. ```bash
  342. $ npx @kryptosai/mcp-observatory ci-report # Generate regression report for CI
  343. ```
  344. ## MCP Server Mode
  345. **No other testing tool is itself an MCP server.** Add Observatory as a server and your AI agent can autonomously test, diagnose, and monitor your other MCP servers.
  346. ```bash
  347. claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serve
  348. ```
  349. Your agent gets 10 tools:
  350. | Tool | When to use it |
  351. |------|---------------|
  352. | `scan` | Check if all your configured MCP servers are healthy |
  353. | `check_server` | Test a specific server before installing or after updating |
  354. | `score_server` | Get a quick health score and grade for a server |
  355. | `record` | Capture a baseline of a working server for future comparison |
  356. | `replay` | Test against a recorded session — no live server needed |
  357. | `verify` | Confirm a server update didn't break anything |
  358. | `watch` | Check a server and see what changed since the last check |
  359. | `diff_runs` | Find regressions between two check results |
  360. | `get_last_run` | Retrieve previous check results for a server |
  361. | `suggest_servers` | Discover MCP servers that match your project stack |
  362. An AI tool that checks other AI tools. It is a tool testing tools that serve tools.
  363. ### Security
  364. The MCP server runs inside AI hosts where an LLM chooses which tools to call. To prevent prompt-injection attacks:
  365. - **Command allowlist:** Only `npx`, `node`, `python`, `python3`, `uvx`, `docker`, `deno`, `bun` are permitted as base executables. The CLI has no restrictions.
  366. - **Path validation:** File-reading tools are constrained to the runs/cassettes directories.
  367. - **No arbitrary execution:** Use the CLI for unrestricted commands.
  368. ### CLI vs MCP: Intentional Differences
  369. | Feature | CLI | MCP Server | Why |
  370. |---------|-----|------------|-----|
  371. | `watch` | Polling loop | Single check + diff | Request/response doesn't support long-polling |
  372. | Interactive menu | Arrow-key navigation | Not available | MCP has no interactive UI |
  373. | Color output | `--no-color` flag | Always plain text | MCP returns structured content |
  374. | `report` | Renders saved artifacts | Not available | Agents read artifacts directly |
  375. | `serve` | Starts MCP server | N/A | Is the MCP server |
  376. | `run` | Reads target config files | Inline params | MCP tools accept params directly |
  377. | `get_last_run` | Not available (use `ls` + `diff`) | Available | Convenience for agents |
  378. ## Compatibility
  379. Works with any MCP server that uses standard transports:
  380. | Transport | Examples | Adapter |
  381. |-----------|----------|---------|
  382. | **stdio** (most servers) | [filesystem](https://www.npmjs.com/package/@modelcontextprotocol/server-filesystem), [memory](https://www.npmjs.com/package/@modelcontextprotocol/server-memory), [context7](https://www.npmjs.com/package/@upstash/context7-mcp), [brave-search](https://www.npmjs.com/package/@modelcontextprotocol/server-brave-search), [sentry](https://www.npmjs.com/package/@sentry/mcp-server), [notion](https://www.npmjs.com/package/@notionhq/notion-mcp-server), [stripe](https://www.npmjs.com/package/@stripe/mcp) | `local-process` |
  383. | **HTTP/SSE** (remote) | [Cloudflare](https://developers.cloudflare.com/mcp/), [Exa](https://exa.ai), [Tavily](https://tavily.com) | `http` |
  384. | **Docker** | All `@modelcontextprotocol/server-*` images | `local-process` via `docker run -i` |
  385. Servers needing API keys work via `env` in the target config. Python servers work via `uvx`. See the [full compatibility matrix](./docs/compatibility.md) for tested servers and known issues.
  386. ### Target config files
  387. For more control (env vars, metadata, custom timeout):
  388. ```json
  389. {
  390. "targetId": "filesystem-server",
  391. "adapter": "local-process",
  392. "command": "npx",
  393. "args": ["-y", "@modelcontextprotocol/server-filesystem", "."],
  394. "timeoutMs": 15000,
  395. "skipInvoke": false
  396. }
  397. ```
  398. ```bash
  399. npx @kryptosai/mcp-observatory run --target ./target.json
  400. ```
  401. ### HTTP / SSE targets
  402. ```json
  403. {
  404. "targetId": "my-remote-server",
  405. "adapter": "http",
  406. "url": "https://mcp.example.com/mcp",
  407. "authToken": "${MCP_SERVER_TOKEN}",
  408. "headers": {
  409. "X-Api-Key": "$MCP_SERVER_API_KEY"
  410. },
  411. "timeoutMs": 15000
  412. }
  413. ```
  414. Target configs support `${VAR}`, `$VAR`, and `env:VAR` references in `authToken`, `headers`, and local-process `env` values.
  415. ## How It Compares
  416. | Feature | Observatory | [mcp-recorder](https://github.com/punkpeye/mcp-recorder) | [MCPBench](https://github.com/QuantGeekDev/mcpbench) | [mcp-jest](https://github.com/nicobailon/mcp-jest) |
  417. |---------|:-----------:|:----------:|:-------:|:-------:|
  418. | Auto-discover servers | ✅ | — | — | — |
  419. | Check capabilities | ✅ | — | ✅ | ✅ |
  420. | Invoke tools | ✅ | — | — | ✅ |
  421. | Schema drift detection | ✅ | — | — | — |
  422. | Record / replay | ✅ | ✅ | — | — |
  423. | Verify against cassette | ✅ | — | — | — |
  424. | Response snapshot diffs | ✅ | — | — | — |
  425. | Benchmarking / latency | — | — | ✅ | — |
  426. | Jest integration | — | — | — | ✅ |
  427. | **Works as MCP server** | **✅** | — | — | — |
  428. Each tool has strengths. Observatory focuses on regression detection and CI-friendly workflows. mcp-recorder is great as a transparent proxy. MCPBench is the go-to for performance benchmarking. mcp-jest is ideal if you're already in a Jest workflow.
  429. ## Prior Art
  430. The record/replay/verify pattern is inspired by:
  431. - [VCR](https://github.com/vcr/vcr) (Ruby) — pioneered cassette-based HTTP record/replay
  432. - [Polly.js](https://github.com/Netflix/pollyjs) (Netflix) — HTTP interaction recording for JavaScript
  433. - [mcp-recorder](https://github.com/punkpeye/mcp-recorder) — MCP-specific traffic recording proxy
  434. - [MCPBench](https://github.com/QuantGeekDev/mcpbench) — MCP server benchmarking
  435. - [mcp-jest](https://github.com/nicobailon/mcp-jest) — Jest-style testing for MCP servers
  436. ## Limitations
  437. - Servers requiring interactive OAuth (e.g., Google Drive) need pre-authentication before Observatory can connect
  438. - Custom WebSocket transports (e.g., BrowserTools MCP) are not supported
  439. - A few servers time out or close before init — see [known issues](./docs/known-issues.md) and [compatibility](./docs/compatibility.md)
  440. ## Works with mcp-seatbelt
  441. Scan before you trust. Then run `npx -y @kryptosai/mcp-observatory@latest enforce --start-proxy` — Observatory writes a deny-default [mcp-seatbelt](https://github.com/KryptosAI/mcp-seatbelt) policy and starts the proxy. Observatory validates; seatbelt enforces.
  442. ## Works with agent-obs
  443. Secure your servers with Observatory. Trace your agents with [agent-obs](https://github.com/KryptosAI/agent-observability) — an open-source agent execution tracer that records every tool call, computes A-F session grades, and shows you exactly where your agents spend time, burn tokens, and hit errors. Observatory tells you if a server is safe. agent-obs tells you what your agent did with it. Free, local-first, `npm install -g agent-obs`.
  444. ## Contributors ✨
  445. Thanks to these amazing people who have contributed:
  446. - [leemeo3](https://github.com/leemeo3) — 3 Safety Index targets (Git, Chrome DevTools, Filesystem MCP)
  447. - [albatrossflyon-coder](https://github.com/albatrossflyon-coder) — GitHub MCP Safety Index (#201)
  448. - [tanishxdev](https://github.com/tanishxdev) — Legacy CLI deprecation warnings (#187)
  449. - [sansynx](https://github.com/sansynx) — CLI format validation (#182)
  450. [See all contributors →](CONTRIBUTORS.md)
  451. ## Contributing
  452. We welcome contributors! This project follows a [Contributor Covenant Code of Conduct](./CODE_OF_CONDUCT.md). The fastest way to get involved:
  453. [![good first issue](https://img.shields.io/github/issues-search/KryptosAI/mcp-observatory?query=is%3Aopen%20label%3A%22good%20first%20issue%22&label=good%20first%20issue&color=green)](https://github.com/KryptosAI/mcp-observatory/issues?q=is%3Aopen+label%3A%22good+first+issue%22)
  454. ```bash
  455. git clone https://github.com/KryptosAI/mcp-observatory.git && cd mcp-observatory && npm install && npm test
  456. ```
  457. The most common first contribution is adding an MCP server to the Safety Index (10-15 minutes). See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines, code standards, and the contributor recognition ladder.
  458. ---
  459. If Observatory saved you a broken deploy, consider giving it a [star](https://github.com/KryptosAI/mcp-observatory). It helps others find the project.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.