Skip to content
Back to search

rhesis

The collaboration layer for AI teams: domain experts annotate and review agent behavior, engineers improve the agent from what they find.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/rhesis-ai/rhesis with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/rhesis-ai/rhesis'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/rhesis-ai--rhesis/badge.svg)](/listings/rhesis-ai--rhesis)

HTML

<a href="/listings/rhesis-ai--rhesis"><img src="/api/listings/rhesis-ai--rhesis/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 300 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <p align="center">
  2. <img src=".github/images/GH-collaborate.png"
  3. loading="lazy"
  4. width="1920"
  5. alt="Rhesis: Get the feedback you need to improve your agents">
  6. </p>
  7. <p align="center">
  8. <a href="https://github.com/rhesis-ai/rhesis/blob/main/LICENSE">
  9. <img src="https://img.shields.io/badge/license-MIT%20%2B%20Enterprise-blue" alt="License">
  10. </a>
  11. <a href="https://pypi.org/project/rhesis-sdk/">
  12. <img src="https://img.shields.io/pypi/v/rhesis-sdk" alt="PyPI Version">
  13. </a>
  14. <a href="https://pypi.org/project/rhesis-sdk/">
  15. <img src="https://img.shields.io/pypi/pyversions/rhesis-sdk" alt="Python Versions">
  16. </a>
  17. <a href="https://codecov.io/gh/rhesis-ai/rhesis">
  18. <img src="https://codecov.io/gh/rhesis-ai/rhesis/graph/badge.svg?token=1XQV983JEJ" alt="codecov">
  19. </a>
  20. <a href="https://discord.rhesis.ai">
  21. <img src="https://img.shields.io/discord/1340989671601209408?color=7289da&label=Discord&logo=discord&logoColor=white" alt="Discord">
  22. </a>
  23. <a href="https://www.linkedin.com/company/rhesis-ai">
  24. <img src="https://img.shields.io/badge/LinkedIn-Rhesis_AI-blue?logo=linkedin" alt="LinkedIn">
  25. </a>
  26. <a href="https://docs.rhesis.ai">
  27. <img src="https://img.shields.io/badge/docs-rhesis.ai-blue" alt="Documentation">
  28. </a>
  29. </p>
  30. <p align="center">
  31. <a href="#get-started"><strong>Quickstart</strong></a> ·
  32. <a href="https://youtu.be/sCfhabhk6ww"><strong>Connect your agent (video)</strong></a> ·
  33. <a href="https://rhesis.ai"><strong>Website</strong></a> ·
  34. <a href="https://docs.rhesis.ai"><strong>Docs</strong></a> ·
  35. <a href="https://discord.rhesis.ai"><strong>Discord</strong></a> ·
  36. <a href="https://github.com/rhesis-ai/rhesis/blob/main/CHANGELOG.md"><strong>Changelog</strong></a>
  37. </p>
  38. <h3 align="center">The open-source collaboration layer for teams building AI agents.<br>
  39. <strong>Open source · SaaS or self-hosted · UI, SDK, and MCP</strong></h3>
  40. <p align="center">
  41. Connect the agent you are building, share the link with your team, and get structured feedback from the people who know the right answers.
  42. </p>
  43. ---
  44. ## Why Rhesis?
  45. Most tools start with tests or traces. Rhesis starts one step earlier: the people who know what the
  46. agent should answer are not the people building it, and their feedback rarely arrives in a form you
  47. can act on.
  48. - **Feedback that stays attached.** Every review sits on the test case and the agent version that produced it, not in a Slack thread
  49. - **One pass/fail bar.** The whole team reviews against the same tests, and you measure against them
  50. - **UI for reviewers, SDK and MCP for builders.** Same data, three ways in
  51. - **From feedback to CI.** Recurring feedback becomes tests and metrics that run on every change
  52. <p align="center">
  53. <a href="https://rhesis.ai/?video=open" target="_blank">
  54. <img src=".github/images/GH-demo-video.jpg"
  55. loading="lazy"
  56. width="1920"
  57. alt="Watch: how engineers and domain experts test AI agents together">
  58. </a>
  59. </p>
  60. ---
  61. ## How it works
  62. <p align="center">
  63. <img src=".github/images/GH-how-it-works.png"
  64. loading="lazy"
  65. width="1920"
  66. alt="Rhesis workflow: requirements and conversations become tests, metrics and expert verdicts evaluate the agent, CI gates releases, traces feed back as new tests">
  67. </p>
  68. **1. Connect the agent you are building.** Via the SDK or a REST endpoint, no public URL needed
  69. ([how](#connect-your-agent), [video](https://youtu.be/sCfhabhk6ww)).
  70. **2. Share the Rhesis link with your stakeholders.** Domain experts, product managers, and
  71. reviewers open it in a browser. Nothing to install, no code.
  72. **3. They put the agent to work.** In the playground they chat with the live agent and turn
  73. interesting conversations into tests. They create and run test sets, simulate multi-turn
  74. conversations, and leave structured feedback on what came back: pass/fail verdicts and comments,
  75. down to the individual metric or conversation turn.
  76. **4. Pull that feedback back into development.** Read it from the SDK or the REST API, or work
  77. with it from Cursor, Claude Code, and other MCP clients through the Rhesis skill. Fix the agent,
  78. run the same tests again.
  79. **5. Agree on what the agent has to get right.** Each cycle, feedback that arrived as prose becomes
  80. tests and metrics that check the same thing automatically. Reviews on a handful of cases end up as
  81. evals that run on every change.
  82. ---
  83. ## Who it’s for
  84. | Role | How they use Rhesis |
  85. |------|---------------------|
  86. | **AI engineers** | Connect the agent, pull feedback and reviewed test sets into the SDK, CI, and MCP while you build |
  87. | **Domain experts** | Try the agent, review its answers, say what is wrong and what a correct answer looks like. UI, no code |
  88. | **Product managers** | Turn scattered feedback into tests, and see whether the agent improves against the PRD. UI or MCP |
  89. ---
  90. ## Capabilities
  91. Nobody can review every case by hand. Once the first feedback is in, you can grow coverage from it:
  92. - **[Architect](https://docs.rhesis.ai/docs/architect)**: a chat agent that turns a PRD into requirements, metrics, and test sets, runs them, and charts the results
  93. - **Test generation** from your requirements, a PRD, or an uploaded file
  94. - **Conversation simulation** with Penelope; **adversarial probing** with Polyphemus and [garak](https://github.com/leondz/garak)
  95. - **[Requirement coverage](https://docs.rhesis.ai/docs/requirements)**: see which requirements pass or fail, and in which conversations
  96. - **[Production tracing](https://docs.rhesis.ai/docs/tracing)**: OpenTelemetry traces with latency, tokens, and cost. Auto-instrumentation for LangChain, LangGraph, Microsoft Agent Framework, Pydantic AI, Google ADK, and Haystack
  97. - **[Experiments](https://docs.rhesis.ai/docs/experiments)**: compare model, prompt, or retrieval changes on the same test set, by quality and cost
  98. Generated tests are only as good as the requirements behind them. Instead of retyping a spec into a
  99. prompt, connect the tools your requirements already live in (Notion, GitHub, Jira, Confluence) and
  100. Rhesis writes tests from the real thing. See [Tools](https://docs.rhesis.ai/docs/tools).
  101. ### From what your experts know to what runs in CI
  102. | Who knows it | What they know | What Rhesis checks |
  103. |---|---|---|
  104. | **Legal** | "Bookings more than 24 h before departure are fully refundable." | The agent quotes the right refund terms for this booking's dates, across multi-turn conversations |
  105. | **Support lead** | "Business accounts always escalate to a named contact." | Escalation is offered on the business tier, never on the consumer tier, and the contact is named |
  106. | **Product** | "If the customer changes topic, the agent still finishes the cancellation." | Simulated users interrupt and rephrase; the agent keeps the booking reference and completes the task |
  107. | **Marketing** | "We never promise delivery dates." | Adversarial prompts try to get a promise out of it; the brand-voice metric flags any that work |
  108. None of these come from a metric library. Your team writes them once as requirements, and they run on every change.
  109. Details: [docs.rhesis.ai](https://docs.rhesis.ai)
  110. ---
  111. ## Get started
  112. ### Cloud
  113. [app.rhesis.ai](https://app.rhesis.ai): managed service, connect your agent and invite your team.
  114. ### Local (Docker)
  115. ```bash
  116. git clone https://github.com/rhesis-ai/rhesis.git && cd rhesis && ./rh start
  117. ```
  118. `./rh start` pulls prebuilt images from GHCR. To build from the repo instead, use `./rh start --build` (and `./rh restart --build` after local Dockerfile changes).
  119. **Access:** Frontend at `localhost:3000`, API at `localhost:8080/docs`
  120. **Commands:** `./rh logs` · `./rh stop` · `./rh restart` · `./rh delete`
  121. > Local setup auto-logs you in. Not for production; see below.
  122. Once the platform is running, connect your agent with the SDK:
  123. ```bash
  124. pip install rhesis-sdk
  125. ```
  126. See [sdk/README.md](sdk/README.md).
  127. ### Deployment options
  128. | Option | Best for |
  129. |--------|----------|
  130. | **[Rhesis Cloud](https://app.rhesis.ai)** | Managed by us, nothing to run |
  131. | **Local Docker (`./rh start`)** | Development and trying the platform |
  132. | **[Docker Compose](https://docs.rhesis.ai/self-hosting/docker-compose)** | A single server |
  133. | **[Kubernetes (Helm)](https://docs.rhesis.ai/self-hosting/kubernetes-helm)** | Production clusters, via ArgoCD and the [`charts/rhesis`](charts/rhesis) chart we run ourselves |
  134. | **[GCP (Terraform)](https://docs.rhesis.ai/self-hosting/gcp-terraform)** | Provisioning a GKE cluster from scratch |
  135. More: [self-hosting docs](https://docs.rhesis.ai/self-hosting).
  136. ---
  137. ## Connect your agent
  138. Rhesis needs a way to invoke the agent under test. Two ways to do it: we recommend the SDK
  139. connector, but if your agent already has a public REST endpoint, that is the fastest way to get
  140. started.
  141. ### SDK connector (recommended)
  142. Your process opens a persistent outbound WebSocket. Rhesis sends test inputs down that connection,
  143. so the agent needs no public URL and can stay on your laptop or inside your VPC. You write a
  144. function instead of describing a payload, and the same SDK carries tracing.
  145. ```python
  146. from rhesis.sdk.decorators import endpoint
  147. @endpoint(name="my-chatbot")
  148. def chat(message: str) -> str:
  149. # Call your agent here
  150. return my_agent(message)
  151. ```
  152. Run it, and the endpoint registers itself in Rhesis. See the [SDK README](sdk/README.md) for
  153. install, environments, and tracing.
  154. ### Your agent's REST endpoint (fastest start if it is public)
  155. Already serving HTTP on a reachable URL? Register it in the UI, no code and nothing to deploy. You
  156. supply auth headers plus request and response mapping, or let Rhesis derive the configuration from
  157. an OpenAPI spec or by exploring the endpoint. See
  158. [Creating endpoints](https://docs.rhesis.ai/docs/endpoints/creating-endpoints).
  159. Either way, the next step is the same: share the link, and let your team start using the agent
  160. through the [playground](https://docs.rhesis.ai/docs/playground) and test runs.
  161. ---
  162. ## Work from your own tools
  163. Feedback lands in Rhesis, but you do not have to leave your editor to act on it.
  164. ### MCP and skills
  165. Use Rhesis from MCP-capable clients (Cursor, Claude Code, and others): build test sets, pull
  166. results, trigger runs, and read the feedback in the tools you already work in. Install with the
  167. [skills](https://github.com/vercel-labs/skills) CLI:
  168. ```bash
  169. npx skills add rhesis-ai/rhesis
  170. ```
  171. See the [skills README](skills/rhesis/README.md).
  172. ### SDK and REST API
  173. Pull test runs, results, and the reviews attached to them from Python, or hit the API directly from
  174. CI in any language: [OpenAPI spec](https://api.rhesis.ai/docs).
  175. > The SDK also handles [tracing](https://docs.rhesis.ai/docs/tracing). LLMs for generation and judges: [Models](https://docs.rhesis.ai/sdk/models).
  176. ---
  177. ## In this repo
  178. | Path | What it covers |
  179. |------|----------------|
  180. | [`sdk/`](sdk/README.md) | Python SDK: connector, synthesizers, metrics, tracing |
  181. | [`skills/rhesis/`](skills/rhesis/README.md) | Agent skill + MCP workflows for Cursor, Claude Code, and others |
  182. | [`apps/backend/`](apps/backend/README.md) | API and workers |
  183. | [`apps/frontend/`](apps/frontend/README.md) | Web UI |
  184. | [`docs/`](docs/README.md) | Documentation site source |
  185. | [`charts/rhesis/`](charts/rhesis) | Helm chart for Kubernetes deployments |
  186. | [`terraform/`](terraform) | Infrastructure as code for GCP |
  187. ---
  188. ## Open source
  189. [MIT licensed](LICENSE). No plans to relicense core features. Enterprise features live in `ee/` and remain separate.
  190. We built Rhesis because the feedback that mattered most kept getting stuck outside the development loop. If you face the same problem, contributions are welcome.
  191. ---
  192. ## Contributing
  193. See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
  194. **Ways to contribute:** Fix bugs or add features · Contribute test sets for common failure modes · Improve documentation · Help others in Discord or GitHub discussions
  195. ---
  196. ## Support
  197. - **[Documentation](https://docs.rhesis.ai)**: Guides and API reference
  198. - **[Discord](https://discord.rhesis.ai)**: Community support
  199. - **[GitHub Issues](https://github.com/rhesis-ai/rhesis/issues)**: Bug reports and feature requests
  200. ---
  201. ## Security & privacy
  202. We take data security seriously. See our [Privacy Policy](https://rhesis.ai/privacy-policy) for details.
  203. **Telemetry:** Rhesis collects basic, anonymized usage statistics to improve the product. No sensitive data is collected or shared with third parties.
  204. - **Self-hosted:** Opt out by setting `OTEL_RHESIS_TELEMETRY_ENABLED=false`
  205. - **Cloud:** Telemetry enabled as part of Terms & Conditions
  206. ---
  207. <p align="center">
  208. <strong>Made with <img src="https://github.com/user-attachments/assets/598c2d81-572c-46bd-b718-dee32cdc749c" height="16" alt="Rhesis logo"> in Potsdam, Germany 🇩🇪</strong>
  209. </p>
  210. <p align="center">
  211. <a href="https://rhesis.ai">Learn more at rhesis.ai</a>
  212. </p>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.