Skip to content
Back to search

agent-confidence-plugins

RUPA uncertainty guidance for Claude Code

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/mazhengzhao--uncertainty-guard-claude-code/badge.svg)](/listings/mazhengzhao--uncertainty-guard-claude-code)

HTML

<a href="/listings/mazhengzhao--uncertainty-guard-claude-code"><img src="/api/listings/mazhengzhao--uncertainty-guard-claude-code/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Requirements: Claude Code with local `--plugin-dir` and command hooks (tested on **2.1.296**), Python **3.10+**, NumPy, and Linux/macOS (`fcntl` is used for audit locking). Native Windows is not validated.

Install from the GitHub-hosted plugin marketplace:

python3 -m venv "$HOME/.local/share/uncertainty-guard/venv"
export UNCERTAINTY_GUARD_PYTHON="$HOME/.local/share/uncertainty-guard/venv/bin/python"
"$UNCERTAINTY_GUARD_PYTHON" -m pip install 'numpy>=1.24,<3'
claude plugin marketplace add mazhengzhao/uncertainty-guard-claude-code
claude plugin install uncertainty-guard@agent-confidence-plugins

This is an independently hosted marketplace. Official Claude directory listing is a separate review process. To use a source checkout or extracted release:

# Run from the extracted plugin root; keep this interpreter after installing.
python3 -m venv "$HOME/.local/share/uncertainty-guard/venv"
export UNCERTAINTY_GUARD_PYTHON="$HOME/.local/share/uncertainty-guard/venv/bin/python"
"$UNCERTAINTY_GUARD_PYTHON" -m pip install -r requirements.txt
"$UNCERTAINTY_GUARD_PYTHON" scripts/doctor.py
claude plugin validate "$PWD"
claude --plugin-dir "$PWD"

Alternatively install from the independent local marketplace:

claude plugin marketplace add "$PWD"
claude plugin install uncertainty-guard@agent-confidence-plugins

Choose either marketplace installation or `--plugin-dir` for a session. The bundled marketplace points at this plugin's root; the parent research repository is not required. Persist `UNCERTAINTY_GUARD_PYTHON` in your shell or launcher. Hooks require this interpreter even when installed from a marketplace. See [PUBLISHING.md](PUBLISHING.md) for marketplace maintenance and directory submission.

The plugin manifest is `.claude-plugin/plugin.json`, and `hooks/hooks.json` is loaded automatically by Claude Code. Keep these directories at the plugin root. Do not run with `--bare`, which skips hooks. The plugin does not change your Claude Code authentication, provider or user-level settings.

Configuration defaults match the pi package. Copy `uncertainty-guard.example.json` to your project's `.claude/uncertainty-guard.json`, or set `UNCERTAINTY_GUARD_CONFIG` to an absolute JSON path. Choose Python with the environment variable above (not `pythonBin` in this adapter's JSON). Defaults are medium=0.6, high=1.2, auto-block disabled, strict mode off, last 40 steps, and 2000 characters per step.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 13 lines
  1. {
  2. "name": "agent-confidence-plugins",
  3. "description": "RUPA uncertainty guidance for Claude Code",
  4. "owner": {"name": "Zhengzhao Ma"},
  5. "plugins": [
  6. {
  7. "name": "uncertainty-guard",
  8. "source": "./",
  9. "description": "Local trajectory risk auditing and tool-call gating with RUPA",
  10. "version": "0.1.0"
  11. }
  12. ]
  13. }
README.md · README · 134 lines
  1. # uncertainty-guard for Claude Code
  2. A self-contained Claude Code plugin adapting the pi uncertainty-guard policy to
  3. `PreToolUse` hooks. It uses the same bundled RUPA `trajectory_tau` scoring source
  4. and thresholds. No Agent-Confidence checkout, model API key, Harbor package, or
  5. decision-model endpoint is required by the hook itself.
  6. **Version 0.1.0.** Author: [Zhengzhao Ma](https://github.com/mazhengzhao).
  7. MIT licensed. See
  8. [CHANGELOG](CHANGELOG.md) and [publishing instructions](PUBLISHING.md).
  9. ## Install and load
  10. Requirements: Claude Code with local `--plugin-dir` and command hooks (tested on
  11. **2.1.296**), Python **3.10+**, NumPy, and Linux/macOS (`fcntl` is used for audit
  12. locking). Native Windows is not validated.
  13. Install from the GitHub-hosted plugin marketplace:
  14. ```bash
  15. python3 -m venv "$HOME/.local/share/uncertainty-guard/venv"
  16. export UNCERTAINTY_GUARD_PYTHON="$HOME/.local/share/uncertainty-guard/venv/bin/python"
  17. "$UNCERTAINTY_GUARD_PYTHON" -m pip install 'numpy>=1.24,<3'
  18. claude plugin marketplace add mazhengzhao/uncertainty-guard-claude-code
  19. claude plugin install uncertainty-guard@agent-confidence-plugins
  20. ```
  21. This is an independently hosted marketplace. Official Claude directory listing
  22. is a separate review process. To use a source checkout or extracted release:
  23. ```bash
  24. # Run from the extracted plugin root; keep this interpreter after installing.
  25. python3 -m venv "$HOME/.local/share/uncertainty-guard/venv"
  26. export UNCERTAINTY_GUARD_PYTHON="$HOME/.local/share/uncertainty-guard/venv/bin/python"
  27. "$UNCERTAINTY_GUARD_PYTHON" -m pip install -r requirements.txt
  28. "$UNCERTAINTY_GUARD_PYTHON" scripts/doctor.py
  29. claude plugin validate "$PWD"
  30. claude --plugin-dir "$PWD"
  31. ```
  32. Alternatively install from the independent local marketplace:
  33. ```bash
  34. claude plugin marketplace add "$PWD"
  35. claude plugin install uncertainty-guard@agent-confidence-plugins
  36. ```
  37. Choose either marketplace installation or `--plugin-dir` for a session.
  38. The bundled marketplace points at this plugin's root; the parent research
  39. repository is not required. Persist `UNCERTAINTY_GUARD_PYTHON` in your shell or
  40. launcher. Hooks require this interpreter even when installed from a marketplace.
  41. See [PUBLISHING.md](PUBLISHING.md) for marketplace maintenance and directory submission.
  42. The plugin manifest is `.claude-plugin/plugin.json`, and `hooks/hooks.json` is
  43. loaded automatically by Claude Code. Keep these directories at the plugin root.
  44. Do not run with `--bare`, which skips hooks. The plugin does not change your
  45. Claude Code authentication, provider or user-level settings.
  46. Configuration defaults match the pi package. Copy
  47. `uncertainty-guard.example.json` to your project's
  48. `.claude/uncertainty-guard.json`, or set `UNCERTAINTY_GUARD_CONFIG` to an absolute
  49. JSON path. Choose Python with the environment variable above (not `pythonBin`
  50. in this adapter's JSON). Defaults are medium=0.6, high=1.2, auto-block disabled,
  51. strict mode off, last 40 steps, and 2000 characters per step.
  52. ## Policy and audit
  53. The hook reads the current transcript, maps text/tool calls/results into RUPA
  54. steps, and scores them locally before the next tool call. Private thinking blocks
  55. are omitted, matching the pi adapter. Missing pending tool calls are appended
  56. once, matched by tool-use ID. Transcript messages are deduplicated by UUID.
  57. If the first user message has not yet been flushed, the queued prompt supplies
  58. the goal until the regular user message is available, without duplicating it.
  59. | Situation | Hook output |
  60. | --- | --- |
  61. | Low/medium risk or nonsensitive high risk | Audit only; no permission override |
  62. | High-risk sensitive operation | `permissionDecision=ask` |
  63. | Same situation with `UNCERTAINTY_GUARD_NONINTERACTIVE=1` | `permissionDecision=deny` |
  64. | Auto-block threshold reached | `permissionDecision=deny` |
  65. | Scorer unavailable/error | Fail open, write error audit and stderr warning |
  66. `Bash`, `Read`, `Write`, `Edit`, `MultiEdit`, and `NotebookEdit` are mapped to the pi tool
  67. categories. Other tools are still scored and audited. The default sensitive
  68. patterns are case-insensitive substrings, not a shell parser or security policy.
  69. In headless evaluations, explicitly set `UNCERTAINTY_GUARD_NONINTERACTIVE=1` to
  70. avoid unresolved human confirmation. Audit-only calls do not force an `allow`,
  71. so they preserve Claude Code's normal permission checks.
  72. Audit JSONL defaults to `.claude/uncertainty-guard/audit.jsonl` under the working
  73. directory. Override with `UNCERTAINTY_GUARD_AUDIT`. Records include session ID,
  74. tool-use ID, score, level, decision kind, number of steps and hook latency; tool
  75. arguments and prompts are not copied into the audit. Concurrent writes use a
  76. file lock. Full trajectory text is only read locally from Claude Code's own
  77. transcript. RUPA scores are heuristic risk signals, not calibrated probabilities.
  78. Unlike pi's persistent bridge, Claude Code starts a Python hook process for each
  79. tool invocation. The runtime enforces a 10-second hook timeout; the default
  80. 5-second scoring budget is checked before applying any decision. Timeouts and
  81. runtime-level hook failures can bypass the adapter, so a benchmark must verify
  82. that tool calls produced successful audit records. Transcript layouts and
  83. permission prompts differ between harnesses; this is a policy adaptation,
  84. not a claim that CC and pi will receive identical trajectories or permissions.
  85. ## Verify, update and remove
  86. The offline `scripts/doctor.py` command must return `"ok": true`. It invokes the
  87. actual hook and scorer with a synthetic transcript in a temporary directory;
  88. it does not contact a model or change the project's audit file. After a real
  89. Claude Code tool call, check the project's audit JSONL for a new `kind: audit`
  90. record with a finite score. A successful model response alone does not verify
  91. that Claude Code has loaded the hook.
  92. Use `claude plugin update uncertainty-guard@agent-confidence-plugins` to update
  93. an installed marketplace plugin, or
  94. `claude plugin uninstall uncertainty-guard@agent-confidence-plugins` to remove
  95. it. For `--plugin-dir`, simply omit that option on the next launch. Uninstalling
  96. does not erase project audit records; manage those according to your needs.
  97. ## Test and distribute
  98. ```bash
  99. python3 -m unittest discover -s tests
  100. claude plugin validate "$PWD" --strict
  101. ```
  102. Copy this entire directory to distribute locally. Its only Python dependency is
  103. NumPy. Do not include `.venv`, `__pycache__`, sessions, user settings, or API keys.
  104. For a local marketplace, place this plugin directory beside a marketplace JSON
  105. entry with `source` pointing at it; the repository's `plugins/.claude-plugin/marketplace.json`
  106. provides a working example. Local `--plugin-dir` loading requires no marketplace.
  107. The bundled scoring source is unmodified from the research repository; see
  108. `THIRD_PARTY_NOTICES.md` and `vendor/LICENSE.tau2`. The hook adapter is MIT-licensed.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.