Skip to content
Back to search

sugar-plugins

🍰 Sugar - the coordination and evidence layer for AI agent fleets: one durable board, live Mission Control, audit exports and shared memory

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/roboticforce/sugar with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/roboticforce/sugar'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/roboticforce--sugar/badge.svg)](/listings/roboticforce--sugar)

HTML

<a href="/listings/roboticforce--sugar"><img src="/api/listings/roboticforce--sugar/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

# Install once, use in any project
pipx install 'sugarai[all]'

cd ~/dev/my-app
sugar init

# Give the fleet work
sugar add "Fix the auth timeout" --type bug_fix --urgent
sugar add "Add profile settings" --type feature

# Watch it, or let Sugar run it
sugar dashboard --open
sugar run

Your agent gets memory and the board over MCP:

claude mcp add sugar       -- sugar mcp memory
claude mcp add sugar-tasks -- sugar mcp tasks
# Other clients
sugar opencode setup     # OpenCode, one command
goose configure          # Goose: Add Extension -> Command-line -> sugar mcp memory

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 23 lines
  1. {
  2. "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
  3. "name": "sugar-plugins",
  4. "version": "1.0.0",
  5. "description": "🍰 Sugar - the coordination and evidence layer for AI agent fleets: one durable board, live Mission Control, audit exports and shared memory",
  6. "owner": {
  7. "name": "Steven Leggett",
  8. "email": "contact@roboticforce.io"
  9. },
  10. "plugins": [
  11. {
  12. "name": "sugar",
  13. "description": "Put Claude Code and your other harnesses on one durable board: atomic claims and leases, live Mission Control, cost attribution, tamper-evident audit and shared memory",
  14. "version": "3.10.2",
  15. "author": {
  16. "name": "Steven Leggett",
  17. "email": "contact@roboticforce.io"
  18. },
  19. "source": ".",
  20. "category": "development"
  21. }
  22. ]
  23. }
README.md · README · 308 lines
  1. <div align="center">
  2. # Sugar
  3. **Mission control for your agent fleet.**
  4. One durable board for Claude Code, Codex, Cursor, Goose, OpenCode, DSH — and any harness that can claim a task.
  5. Atomic claims, live agent activity, cost attribution and tamper-evident history. Local-first, no account.
  6. <!-- mcp-name: io.github.cdnsteve/sugar -->
  7. [![PyPI](https://img.shields.io/pypi/v/sugarai?style=flat-square&label=pypi)](https://pypi.org/project/sugarai/)
  8. [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-3776AB?style=flat-square)](pyproject.toml)
  9. [![CI](https://github.com/roboticforce/sugar/actions/workflows/ci.yml/badge.svg?branch=develop&style=flat-square)](https://github.com/roboticforce/sugar/actions/workflows/ci.yml)
  10. [![License](https://img.shields.io/badge/license-AGPL--3.0%20%2B%20commercial-blue?style=flat-square)](LICENSE)
  11. [![MCP](https://img.shields.io/badge/MCP-server-8A2BE2?style=flat-square)](docs/local-dashboard.md)
  12. [Quick start](#quick-start) · [See it work](#see-your-fleet-work) · [Why Sugar](#why-sugar) · [Works with](#works-with-your-harnesses) · [Docs](docs/README.md) · [Team edition](#team-edition-self-hosted)
  13. <img src="docs/images/app-tasks.png" alt="SugarDash: the task board with a task inspector open, live activity and history" width="880">
  14. </div>
  15. ## Your agents can't see each other
  16. You run four agents across three harnesses on one repo. Then:
  17. - Two agents pick up the same task and overwrite each other's work.
  18. - A crashed run keeps a task "claimed" forever; the queue quietly stalls.
  19. - "What happened yesterday?" lives in six terminal tabs and a scrollback buffer.
  20. - You can't answer *who did what, with which harness, for which task — and what did it cost?*
  21. - Your agent starts every session with amnesia: last week's decisions, conventions and dead ends are gone.
  22. A prettier kanban board doesn't fix this. The missing layer is underneath: a **durable queue**, **atomic claims and leases**, **provenance for every actor**, and a **history you can replay and export**. That's what Sugar is.
  23. ## One board. Any harness. Full history.
  24. Sugar is local-first infrastructure for a fleet of coding agents. Every harness gets the same durable work
  25. queue and the same memory. Every claim, transition and tool call lands in an append-only event spine. You watch
  26. the fleet in a web dashboard or a native macOS app — on your machine, in your files, with no account.
  27. Your agents keep their own subscriptions and workflows. Sugar doesn't replace Claude Code or Codex; it sits
  28. between them and your repo.
  29. ### Coordinate — a queue agents can't double-claim
  30. - Durable SQLite queue with priorities, dependencies, attempts, hold/release and retries.
  31. - **Atomic claim + lease + heartbeat.** If a worker dies, the lease expires and a reaper requeues the task.
  32. - The same verbs from the CLI (`sugar add`, `sugar list`, `sugar run`), MCP tools, REST API and dashboard.
  33. - Keep your existing board: serve Sugar's queue, or mirror a Hermes Kanban board with `--board kanban`
  34. (Sugar reads it; Hermes keeps its write path).
  35. ### See — Mission Control for the fleet
  36. - Web dashboard and native macOS app, both on the same REST + SSE contract.
  37. - **Live floor:** every working agent with its current action, elapsed/lease countdowns and lease health.
  38. - **Streaming console:** tool calls, run starts/ends and usage events as they happen.
  39. - **Trends:** throughput, failures, cycle time and reported spend over time, by agent, harness or project.
  40. - Activity feed, search, filters and direct task links across all of it.
  41. <img src="docs/images/trends-view.png" alt="SugarDash Trends view: tasks completed and cost per agent over a week" width="880">
  42. ### Prove — history with receipts
  43. - **Append-only event spine** (`~/.sugar/events.db`, retention configurable) records every task transition and
  44. every harness event: actor, harness, machine, session, task, project, run and trace id.
  45. - **Usage and cost attribution** (`sugar usage`, `/api/usage`): tokens and cost as your harness reports them —
  46. no invented pricing table.
  47. - **Tamper-evident audit exports** (`sugar audit export` / `sugar audit verify`): canonical JSONL plus a
  48. SHA-256 manifest and optional HMAC signature. Export before retention prunes, keep the evidence.
  49. - Claims are recorded in the same transaction as the work item, so "who is holding this task?" is a query,
  50. not a guess.
  51. ### Connect — one board, many harnesses
  52. - **MCP servers** for tasks and memory: Claude Code, Codex, Cursor, Goose, OpenCode and any MCP client can
  53. read context, claim work and complete it.
  54. - **REST + SSE** for custom scripts, bots and CI.
  55. - **CLI** for everything a human needs, including `sugar emit` for harnesses that stream their own telemetry.
  56. - **TaskStore port + conformance suite**: SQLite today; the contract is backend-agnostic.
  57. ### Remember — your context, not the vendor's
  58. - Project memory (`.sugar/memory.db`) and global memory (`~/.sugar/memory.db`), with seven memory types:
  59. decisions, preferences, file context, error patterns, research, outcomes and guidelines.
  60. - Semantic search via local embeddings (`sugarai[memory]`), with FTS5 keyword fallback. No external API calls.
  61. - Agents read and write memory directly through MCP: `search_memory`, `store_learning`, `recall`,
  62. `get_project_context`, `list_recent_memories`.
  63. ## Quick start
  64. ```bash
  65. # Install once, use in any project
  66. pipx install 'sugarai[all]'
  67. cd ~/dev/my-app
  68. sugar init
  69. # Give the fleet work
  70. sugar add "Fix the auth timeout" --type bug_fix --urgent
  71. sugar add "Add profile settings" --type feature
  72. # Watch it, or let Sugar run it
  73. sugar dashboard --open
  74. sugar run
  75. ```
  76. Your agent gets memory and the board over MCP:
  77. ```bash
  78. claude mcp add sugar -- sugar mcp memory
  79. claude mcp add sugar-tasks -- sugar mcp tasks
  80. ```
  81. ```bash
  82. # Other clients
  83. sugar opencode setup # OpenCode, one command
  84. goose configure # Goose: Add Extension -> Command-line -> sugar mcp memory
  85. ```
  86. ## See your fleet work
  87. The dashboard and app are only half the story — the Live view is the part that makes a fleet feel real.
  88. Harnesses stream their tool calls with one line each:
  89. ```bash
  90. sugar emit --kind tool.invoked --task-id "$TASK_ID" \
  91. --message "running pytest tests/test_api.py" --payload file=src/api.py
  92. ```
  93. <img src="docs/images/live-floor.png" alt="SugarDash Live floor: agents at work with real leases, current actions and countdowns" width="880">
  94. No harness wired up yet? The repo ships a simulator for demos:
  95. ```bash
  96. git clone https://github.com/roboticforce/sugar.git && cd sugar
  97. python scripts/demo_dashboard.py --serve --open # demo queue + dashboard
  98. python scripts/simulate_agent_activity.py # live agents in a second shell
  99. ```
  100. Trends need history; seed a throwaway spine and open the Trends view:
  101. ```bash
  102. python scripts/seed_demo_trends.py --db /tmp/sugar-dashboard-demo/.sugar/events.db --reset
  103. ```
  104. Full walkthrough: [Mission Control](docs/mission-control.md) · [Telemetry](docs/telemetry.md) · [Trends](docs/trends.md)
  105. ## Works with your harnesses
  106. Any harness that can run a CLI or speak MCP can use the board and the memory. The built-in runner
  107. (`sugar run`) drives Claude Code through the Agent SDK today; the other harnesses keep doing their own thing
  108. while Sugar coordinates them.
  109. | Harness | Board (MCP / CLI / REST) | Shared memory | Notes |
  110. |---|---|---|---|
  111. | [Claude Code](https://docs.anthropic.com/en/docs/claude-code) | Yes | Yes | Built-in runner; plugin + Skills; full support |
  112. | Codex | Yes | Yes | Via MCP/CLI; no runner coupling |
  113. | Cursor | Yes | Yes | Via MCP |
  114. | [Goose](https://block.github.io/goose) | Yes | Yes | Via MCP extension |
  115. | [OpenCode](https://github.com/opencode-ai/opencode) | Yes | Yes | `sugar opencode setup` |
  116. | [Aider](https://aider.chat) | Yes | Yes | Via CLI/MCP |
  117. | Hermes Kanban | Yes | — | Mirror a board with `--board kanban`; Hermes owns writes |
  118. | Custom scripts / bots | Yes | Yes | REST + SSE, or MCP |
  119. ## Team edition (self-hosted)
  120. The default dashboard binds loopback and needs no auth. For a team, run one board on a shared machine with
  121. scoped tokens — reads and writes, nothing in a vendor cloud:
  122. ```bash
  123. cat ~/.sugar/board-tokens # name:scope:secret (scope: read | write; file must be 0600)
  124. sugar dashboard --bind 0.0.0.0 --allow-remote --token-file ~/.sugar/board-tokens
  125. ```
  126. The browser exchanges a one-time token for a `SameSite=Strict` cookie; API/SSE accept `Authorization: Bearer`.
  127. Remote binds are refused without both `--allow-remote` and a token source, the foreign-Origin write guard
  128. stays on, and TLS is expected from a reverse proxy. See [Board Server](docs/board-server.md) for the threat
  129. model and Caddy/nginx setup.
  130. What teams get today: one live board, provenance, cost attribution, and audit exports. Approval policies,
  131. RBAC/SSO, quotas and a hosted relay are on the roadmap — [tell us what your team needs](mailto:contact@roboticforce.io?subject=Sugar%20team%20edition).
  132. ## Why Sugar
  133. - **Harness-neutral by design.** You don't have to pick a winner. Claude Code, Codex, Cursor, Goose, OpenCode,
  134. DSH and your own scripts share one queue and one history.
  135. - **Local-first, not cloud-first.** Task state, memory and the event spine are SQLite files on your disk. No
  136. account, no repo upload, no vendor in the loop. Remote mode is opt-in and self-hosted.
  137. - **Evidence beats dashboards.** Any tool can show you a board. Sugar records the claim, the lease, the actor,
  138. the harness, the transitions and the spend — then lets you export and verify it.
  139. - **Durable, not demo-grade.** Atomic claims, lease heartbeats, crash reaping, dependency gating and retries are
  140. tested against a backend-agnostic conformance suite.
  141. - **Open and inspectable.** Dual-licensed AGPL-3.0 + commercial; the REST/SSE contract, event schema and
  142. adapter playbook are documented and versioned.
  143. **What Sugar is not:** not a model vendor, not an agent runtime or cloud coding agent, not a code host, and not
  144. another kanban UI. It's the neutral layer underneath the harnesses you already chose.
  145. <details>
  146. <summary><strong>FAQ</strong></summary>
  147. **Is this just a kanban board for agents?**
  148. No. A board shows status; Sugar owns the queue, the claim/lease transaction, dependency gating, provenance and
  149. an append-only history. It can mirror a Hermes Kanban board, but it doesn't depend on one.
  150. **How is this different from Claude Code Agent Teams or a single-harness orchestrator?**
  151. Those make one harness better at running its own agents. Sugar coordinates *across* harnesses and keeps the
  152. durable record. You can run Claude Code Agent Teams and still put their work on the same board as Codex.
  153. **Does Sugar send my code or data anywhere?**
  154. No. State is local SQLite by default; the dashboard binds loopback. Telemetry is what your harness explicitly
  155. emits (`sugar emit` / `POST /api/events`), and audit exports are files you write yourself.
  156. **Do I need to change how my agents work?**
  157. No. Add the MCP servers (tasks + memory) or use the CLI/REST API. The built-in runner is optional.
  158. **Is it production-ready?**
  159. The 3.10 line is an early release: used daily, with a large test suite and CI across Linux/macOS/Windows, but
  160. expect rough edges. Start with a demo board or a side project, then move your real queue over.
  161. </details>
  162. ## Install
  163. **Recommended: pipx** — one install, available everywhere:
  164. ```bash
  165. pipx install 'sugarai[all]' # dashboard, memory search, MCP, GitHub, OpenCode
  166. pipx upgrade sugarai
  167. ```
  168. <details>
  169. <summary>Other options</summary>
  170. ```bash
  171. pip install 'sugarai[all]' # inside a venv
  172. uv pip install 'sugarai[all]' # uv
  173. pipx install sugarai # core only (keyword memory, no dashboard)
  174. pipx install 'sugarai[dashboard]' # web dashboard + macOS app backend
  175. pipx install 'sugarai[memory]' # semantic memory (local embeddings)
  176. ```
  177. **Native macOS app:** [latest release](https://github.com/roboticforce/sugar/releases/latest) — or build it:
  178. ```bash
  179. cd apps/macos && ./build-app.sh && open dist/SugarDash.app
  180. ```
  181. **Requirements:** Python 3.11+, macOS 14+ for the app, and a CLI/MCP-capable coding agent if you want agents to
  182. claim work.
  183. </details>
  184. ## Also in the box
  185. - **Discovery** — turn GitHub issues, error logs and code analysis into queue items.
  186. - **Autonomous runner** — `sugar run` claims, executes, tests, commits and moves on; `--ralph` iterates until
  187. tests pass.
  188. - **Orchestration** — decompose a large task into a research → plan → implement → review workflow.
  189. - **Quality gates and hooks** — pre/post-execution checks, file protection and command guardrails.
  190. - **Agent Skills** — `sugar-memory`, `sugar-task-planner`, `sugar-quality-guardian`, `sugar-orchestrator`,
  191. with evaluation data in `skills/`.
  192. - **Claude Code plugin** — slash commands and hooks via `.claude-plugin/`.
  193. - **Task types, learning and memory types** — the boring machinery that makes repeated runs better.
  194. ## Documentation
  195. - [Mission Control](docs/mission-control.md) — dashboard, macOS app, event spine
  196. - [Local Dashboard API](docs/local-dashboard.md) — frozen REST/SSE contract
  197. - [Trends](docs/trends.md) · [Telemetry](docs/telemetry.md) · [Usage & cost](docs/usage.md) · [Audit exports](docs/audit.md)
  198. - [Board Server](docs/board-server.md) — team/remote access and TLS
  199. - [Storage Port](docs/storage-port.md) · [Adapters](docs/adapters.md) — backends and harness playbook
  200. - [Memory System](docs/user/memory.md) · [CLI Reference](docs/user/cli-reference.md) · [Quick Start](docs/user/quick-start.md)
  201. - [Agent Teams: market & validation](docs/agent-teams-market.md) · [Business strategy](docs/business-strategy.md)
  202. ## Status and roadmap
  203. Sugar is an early 3.10 release. The stable parts: the task store contract, the event spine schema, the REST/SSE
  204. contract and the audit format. The moving parts: live telemetry wiring, remote/mobile access, approval policies
  205. and additional runners.
  206. Next up, roughly in order: zero-config harness hooks (so Live lights up without manual `sugar emit`), pairing and
  207. a mobile-friendly Mission Control, permission modes with ledger-recorded approvals, then a Postgres backend.
  208. Ideas and bug reports are welcome — [open an issue](https://github.com/roboticforce/sugar/issues).
  209. ## Contributing
  210. ```bash
  211. git clone https://github.com/roboticforce/sugar.git
  212. cd sugar && git checkout develop
  213. uv pip install -e ".[dev,test,github,mcp]"
  214. pytest tests/ -v
  215. ```
  216. See [CONTRIBUTING](docs/dev/contributing.md). PRs target `develop`; releases go to `main`.
  217. ## License
  218. **Dual license: AGPL-3.0 + commercial.**
  219. - **AGPL-3.0** — free for open source and personal use.
  220. - **Commercial license** — for proprietary use. [Contact RoboticForce](mailto:contact@roboticforce.io) or see
  221. [LICENSE](LICENSE).
  222. ---
  223. > Sugar is provided "AS IS" without warranty. Review all AI-generated code before use.
scripts/benchmark_v3.py · Bundled script · 383 lines · 3 findings marked
  1. #!/usr/bin/env python3
  2. """
  3. Sugar V3 Benchmark Script
  4. Measures the concrete overhead improvements of V3 (Agent SDK) vs V2 (subprocess).
  5. These benchmarks focus on what V3 actually improves - startup overhead,
  6. hook latency, and memory footprint - NOT Claude API call time which is identical.
  7. Usage:
  8. python scripts/benchmark_v3.py
  9. """
  10. import asyncio
  11. import statistics
  12. import sys
  13. import time
  14. import tracemalloc
  15. from dataclasses import dataclass
  16. from pathlib import Path
  17. # Add project root to path
  18. sys.path.insert(0, str(Path(__file__).parent.parent))
  19. @dataclass
  20. class BenchmarkResult:
  21. """Results from a benchmark run."""
  22. name: str
  23. samples: list[float]
  24. unit: str
  25. @property
  26. def mean(self) -> float:
  27. return statistics.mean(self.samples)
  28. @property
  29. def median(self) -> float:
  30. return statistics.median(self.samples)
  31. @property
  32. def stdev(self) -> float:
  33. return statistics.stdev(self.samples) if len(self.samples) > 1 else 0
  34. @property
  35. def min_val(self) -> float:
  36. return min(self.samples)
  37. @property
  38. def max_val(self) -> float:
  39. return max(self.samples)
  40. def __str__(self) -> str:
  41. return (
  42. f"{self.name}:\n"
  43. f" Mean: {self.mean:.2f} {self.unit}\n"
  44. f" Median: {self.median:.2f} {self.unit}\n"
  45. f" Std Dev: {self.stdev:.2f} {self.unit}\n"
  46. f" Range: {self.min_val:.2f} - {self.max_val:.2f} {self.unit}"
  47. )
  48. def benchmark_v3_import_time(iterations: int = 10) -> BenchmarkResult:
  49. """Measure time to import V3 agent modules."""
  50. import importlib
  51. times = []
  52. for _ in range(iterations):
  53. # Clear cached imports
  54. modules_to_clear = [k for k in sys.modules.keys() if k.startswith('sugar.agent')]
  55. for mod in modules_to_clear:
  56. del sys.modules[mod]
  57. start = time.perf_counter()
  58. importlib.import_module('sugar.agent.base')
  59. importlib.import_module('sugar.agent.hooks')
  60. elapsed = (time.perf_counter() - start) * 1000
  61. times.append(elapsed)
  62. return BenchmarkResult("V3 Module Import Time", times, "ms")
  63. def benchmark_v3_config_creation(iterations: int = 100) -> BenchmarkResult:
  64. """Measure time to create agent config."""
  65. from sugar.agent.base import SugarAgentConfig
  66. times = []
  67. for _ in range(iterations):
  68. start = time.perf_counter()
  69. config = SugarAgentConfig(
  70. model="claude-sonnet-4-20250514",
  71. max_tokens=8192,
  72. permission_mode="acceptEdits",
  73. quality_gates_enabled=True,
  74. )
  75. elapsed = (time.perf_counter() - start) * 1000
  76. times.append(elapsed)
  77. return BenchmarkResult("V3 Config Creation", times, "ms")
  78. def benchmark_v3_agent_creation(iterations: int = 50) -> BenchmarkResult:
  79. """Measure time to create SugarAgent instance."""
  80. from sugar.agent.base import SugarAgent, SugarAgentConfig
  81. config = SugarAgentConfig(
  82. model="claude-sonnet-4-20250514",
  83. max_tokens=8192,
  84. permission_mode="acceptEdits",
  85. )
  86. times = []
  87. for _ in range(iterations):
  88. start = time.perf_counter()
  89. agent = SugarAgent(config)
  90. elapsed = (time.perf_counter() - start) * 1000
  91. times.append(elapsed)
  92. return BenchmarkResult("V3 Agent Creation", times, "ms")
  93. def benchmark_hooks_creation(iterations: int = 100) -> BenchmarkResult:
  94. """Measure time to create quality gate hooks."""
  95. from sugar.agent.hooks import QualityGateHooks
  96. times = []
  97. for _ in range(iterations):
  98. start = time.perf_counter()
  99. hooks = QualityGateHooks({
  100. "protected_files": [".env", "*.pem", "credentials.json"],
  101. "blocked_commands": ["sudo", "rm -rf /"],
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 130 on GitHub
  102. })
  103. elapsed = (time.perf_counter() - start) * 1000
  104. times.append(elapsed)
  105. return BenchmarkResult("Quality Gate Hooks Creation", times, "ms")
  106. async def benchmark_hooks_check(iterations: int = 100) -> BenchmarkResult:
  107. """Measure time for a security check on a file path."""
  108. from sugar.agent.hooks import QualityGateHooks
  109. hooks = QualityGateHooks({
  110. "protected_files": [".env", "*.pem", "credentials.json", "secrets/*"],
  111. "blocked_commands": ["sudo", "rm -rf /", "chmod 777"],
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 144 on GitHub
  112. })
  113. # Simulate tool use input
  114. test_inputs = [
  115. {"file_path": "/project/src/main.py"},
  116. {"file_path": "/project/.env"},
  117. {"file_path": "/project/config/settings.yaml"},
  118. {"command": "ls -la"},
  119. {"command": "sudo rm -rf /"},
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 153 on GitHub
  120. ]
  121. times = []
  122. for i in range(iterations):
  123. input_data = test_inputs[i % len(test_inputs)]
  124. start = time.perf_counter()
  125. result = await hooks.pre_tool_security_check(
  126. input_data=input_data,
  127. tool_use_id=f"test_{i}",
  128. context={"tool_name": "Write" if "file_path" in input_data else "Bash"}
  129. )
  130. elapsed = (time.perf_counter() - start) * 1000
  131. times.append(elapsed)
  132. return BenchmarkResult("Security Check Latency", times, "ms")
  133. def benchmark_response_serialization(iterations: int = 100) -> BenchmarkResult:
  134. """Measure time to serialize agent response."""
  135. from sugar.agent.base import AgentResponse
  136. response = AgentResponse(
  137. success=True,
  138. content="Task completed successfully. Created new file.",
  139. tool_uses=[
  140. {"tool": "Write", "input": {"file_path": "/project/src/new.py"}},
  141. {"tool": "Bash", "input": {"command": "python -m pytest"}},
  142. ],
  143. files_modified=["/project/src/new.py", "/project/tests/test_new.py"],
  144. execution_time=45.2,
  145. error=None,
  146. quality_gate_results={
  147. "total_tool_executions": 5,
  148. "blocked_operations": 0,
  149. "security_violations": 0,
  150. },
  151. )
  152. times = []
  153. for _ in range(iterations):
  154. start = time.perf_counter()
  155. data = response.to_dict()
  156. elapsed = (time.perf_counter() - start) * 1000
  157. times.append(elapsed)
  158. return BenchmarkResult("Response Serialization", times, "ms")
  159. def benchmark_memory_footprint() -> dict:
  160. """Measure memory footprint of V3 components."""
  161. import gc
  162. results = {}
  163. # Baseline
  164. gc.collect()
  165. tracemalloc.start()
  166. baseline = tracemalloc.get_traced_memory()[0]
  167. # Import agent modules
  168. from sugar.agent.base import SugarAgent, SugarAgentConfig
  169. from sugar.agent.hooks import QualityGateHooks
  170. gc.collect()
  171. after_import = tracemalloc.get_traced_memory()[0]
  172. results["module_import_mb"] = (after_import - baseline) / (1024 * 1024)
  173. # Create config
  174. config = SugarAgentConfig(
  175. model="claude-sonnet-4-20250514",
  176. max_tokens=8192,
  177. permission_mode="acceptEdits",
  178. quality_gates_enabled=True,
  179. )
  180. gc.collect()
  181. after_config = tracemalloc.get_traced_memory()[0]
  182. results["config_mb"] = (after_config - after_import) / (1024 * 1024)
  183. # Create agent
  184. agent = SugarAgent(config)
  185. gc.collect()
  186. after_agent = tracemalloc.get_traced_memory()[0]
  187. results["agent_mb"] = (after_agent - after_config) / (1024 * 1024)
  188. # Create hooks
  189. hooks = QualityGateHooks({
  190. "protected_files": [".env", "*.pem"],
  191. "blocked_commands": ["sudo"],
  192. })
  193. gc.collect()
  194. after_hooks = tracemalloc.get_traced_memory()[0]
  195. results["hooks_mb"] = (after_hooks - after_agent) / (1024 * 1024)
  196. # Total
  197. results["total_mb"] = (after_hooks - baseline) / (1024 * 1024)
  198. tracemalloc.stop()
  199. return results
  200. def benchmark_transient_error_detection(iterations: int = 100) -> BenchmarkResult:
  201. """Measure time to detect transient errors."""
  202. from sugar.agent.base import is_transient_error
  203. test_errors = [
  204. Exception("rate_limit exceeded"),
  205. Exception("Connection timeout"),
  206. Exception("503 Service Unavailable"),
  207. Exception("429 Too Many Requests"),
  208. Exception("API overloaded"),
  209. Exception("Invalid request"), # Not transient
  210. ValueError("Bad input"), # Not transient
  211. ]
  212. times = []
  213. for i in range(iterations):
  214. error = test_errors[i % len(test_errors)]
  215. start = time.perf_counter()
  216. is_transient_error(error)
  217. elapsed = (time.perf_counter() - start) * 1000
  218. times.append(elapsed)
  219. return BenchmarkResult("Transient Error Detection", times, "ms")
  220. async def run_benchmarks():
  221. """Run all benchmarks and display results."""
  222. print("=" * 60)
  223. print("Sugar V3 Benchmark Suite")
  224. print("=" * 60)
  225. print()
  226. print("These benchmarks measure V3 overhead improvements.")
  227. print("Note: Claude API call time is NOT measured (identical in V2/V3).")
  228. print()
  229. print("-" * 60)
  230. results = []
  231. # Timing benchmarks
  232. print("\nRunning timing benchmarks...")
  233. results.append(benchmark_v3_import_time())
  234. print(f" - Module import: {results[-1].mean:.2f}ms")
  235. results.append(benchmark_v3_config_creation())
  236. print(f" - Config creation: {results[-1].mean:.3f}ms")
  237. results.append(benchmark_v3_agent_creation())
  238. print(f" - Agent creation: {results[-1].mean:.3f}ms")
  239. results.append(benchmark_hooks_creation())
  240. print(f" - Hooks creation: {results[-1].mean:.3f}ms")
  241. results.append(await benchmark_hooks_check())
  242. print(f" - Security check: {results[-1].mean:.3f}ms")
  243. results.append(benchmark_response_serialization())
  244. print(f" - Response serialization: {results[-1].mean:.3f}ms")
  245. results.append(benchmark_transient_error_detection())
  246. print(f" - Error detection: {results[-1].mean:.4f}ms")
  247. # Memory benchmark
  248. print("\nRunning memory benchmarks...")
  249. memory = benchmark_memory_footprint()
  250. # Calculate total startup overhead
  251. total_startup = sum(r.mean for r in results[:4])
  252. print()
  253. print("=" * 60)
  254. print("RESULTS SUMMARY")
  255. print("=" * 60)
  256. print("\n## Startup Overhead (V3)")
  257. print(f" Total initialization: {total_startup:.2f}ms")
  258. print(f" - Module import: {results[0].mean:.2f}ms")
  259. print(f" - Config creation: {results[1].mean:.3f}ms")
  260. print(f" - Agent creation: {results[2].mean:.3f}ms")
  261. print(f" - Hooks creation: {results[3].mean:.3f}ms")
  262. print("\n## Per-Operation Latency")
  263. print(f" Security check: {results[4].mean:.3f}ms")
  264. print(f" Response serialization: {results[5].mean:.3f}ms")
  265. print(f" Error detection: {results[6].mean:.4f}ms")
  266. print("\n## Memory Footprint")
  267. print(f" Total V3 components: {memory['total_mb']:.2f}MB")
  268. print(f" - Module import: {memory['module_import_mb']:.2f}MB")
  269. print(f" - Agent instance: {memory['agent_mb']:.3f}MB")
  270. print(f" - Hooks instance: {memory['hooks_mb']:.3f}MB")
  271. print("\n## Comparison with V2 (Subprocess)")
  272. print(" V2 subprocess spawn: ~300-500ms")
  273. print(f" V3 native startup: ~{total_startup:.0f}ms")
  274. print(f" Improvement: ~{(400 - total_startup):.0f}ms faster")
  275. print()
  276. print(" V2 memory (separate process): ~50-100MB additional")
  277. print(f" V3 memory (shared process): ~{memory['total_mb']:.1f}MB additional")
  278. print(f" Improvement: ~{50 - memory['total_mb']:.0f}MB less")
  279. print()
  280. print("=" * 60)
  281. print("CONTEXT")
  282. print("=" * 60)
  283. print("""
  284. These improvements apply to OVERHEAD only. The actual Claude API
  285. call takes 30-300 seconds depending on task complexity.
  286. Example total task time breakdown:
  287. V2: 400ms overhead + 60s Claude = 60.4s total
  288. V3: 50ms overhead + 60s Claude = 60.05s total
  289. The ~350ms improvement is real but represents < 1% of total time.
  290. V3's main benefits are:
  291. 1. Security gates (can BLOCK dangerous operations)
  292. 2. Observability (structured tool tracking)
  293. 3. Reliability (automatic retry on transient errors)
  294. """)
  295. return results, memory
  296. if __name__ == "__main__":
  297. asyncio.run(run_benchmarks())
sugar/agent/hooks.py · Bundled script · 455 lines · 5 findings marked
  1. """
  2. Quality Gate Hooks for Claude Agent SDK
  3. These hooks integrate Sugar's quality gates system with the Agent SDK's
  4. PreToolUse and PostToolUse hook points.
  5. """
  6. import fnmatch
  7. import logging
  8. from datetime import datetime, timezone
  9. from typing import Any, Callable, Dict, List, Optional
  10. logger = logging.getLogger(__name__)
  11. class HookContext:
  12. """Context passed to hook callbacks (placeholder for SDK's HookContext)"""
  13. pass
  14. class QualityGateHooks:
  15. """
  16. Quality gate hooks for the Claude Agent SDK.
  17. Provides PreToolUse and PostToolUse hooks that enforce Sugar's
  18. quality gates during agent execution.
  19. """
  20. def __init__(
  21. self,
  22. config: Optional[Dict[str, Any]] = None,
  23. bash_permissions: Optional[List[str]] = None,
  24. ):
  25. """
  26. Initialize quality gate hooks.
  27. Args:
  28. config: Quality gates configuration
  29. bash_permissions: List of allowed bash command patterns (e.g., ["pytest *", "git status*"])
  30. Empty list means all commands allowed, None means use default restrictions
  31. """
  32. self.config = config or {}
  33. self.enabled = self.config.get("enabled", True)
  34. # Tracking state
  35. self._tool_executions: List[Dict[str, Any]] = []
  36. self._blocked_operations: List[Dict[str, Any]] = []
  37. self._files_modified: List[str] = []
  38. self._security_violations: List[Dict[str, Any]] = []
  39. # Configuration
  40. self._protected_paths = self.config.get(
  41. "protected_paths",
  42. [
  43. ".env",
  44. ".env.local",
  45. "credentials.json",
  46. "secrets.yaml",
  47. ".git/config",
  48. ],
  49. )
  50. self._dangerous_commands = self.config.get(
  51. "dangerous_commands",
  52. [
  53. "rm -rf /",
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 66 on GitHub
  54. "rm -rf ~",
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 67 on GitHub
  55. "> /dev/sda",
  56. "mkfs.",
  57. ":(){:|:&};:",
    • Note: Quoted as data in code: Matches a classic shell fork-bomb pattern
    • Line 70 on GitHub
  58. "chmod -R 777 /",
    • Note: Quoted as data in code: Sets world-writable permissions on a root path (chmod 777 /)
    • Line 71 on GitHub
  59. ],
  60. )
  61. # Bash command permissions (wildcard patterns)
  62. # Empty list = all allowed, None = use dangerous commands check only
  63. self._bash_permissions = bash_permissions
  64. async def pre_tool_security_check(
  65. self,
  66. input_data: Dict[str, Any],
  67. tool_use_id: Optional[str],
  68. context: HookContext,
  69. ) -> Dict[str, Any]:
  70. """
  71. PreToolUse hook: Security check before tool execution.
  72. Checks for:
  73. - Protected file access (.env, credentials, etc.)
  74. - Dangerous bash commands
  75. - Unauthorized modifications
  76. Args:
  77. input_data: Tool input data with 'tool_name' and 'tool_input'
  78. tool_use_id: Unique ID for correlating with PostToolUse
  79. context: Hook context (reserved for future use)
  80. Returns:
  81. Empty dict to allow, or dict with hookSpecificOutput to block
  82. """
  83. if not self.enabled:
  84. return {}
  85. tool_name = input_data.get("tool_name", "")
  86. tool_input = input_data.get("tool_input", {})
  87. # Check for protected file access
  88. if tool_name in ("Write", "Edit", "Read"):
  89. file_path = tool_input.get("file_path", "")
  90. if self._is_protected_file(file_path):
  91. violation = {
  92. "tool": tool_name,
  93. "file": file_path,
  94. "reason": "Protected file access blocked",
  95. "timestamp": datetime.now(timezone.utc).isoformat(),
  96. }
  97. self._security_violations.append(violation)
  98. self._blocked_operations.append(violation)
  99. logger.warning(f"Blocked access to protected file: {file_path}")
  100. return {
  101. "hookSpecificOutput": {
  102. "hookEventName": input_data.get(
  103. "hook_event_name", "PreToolUse"
  104. ),
  105. "permissionDecision": "deny",
  106. "permissionDecisionReason": (
  107. f"Access to protected file '{file_path}' is not allowed. "
  108. "This file may contain sensitive information."
  109. ),
  110. }
  111. }
  112. # Check for bash command permissions (dangerous commands + whitelist)
  113. if tool_name == "Bash":
  114. command = tool_input.get("command", "")
  115. # First check dangerous commands (always blocked)
  116. if self._is_dangerous_command(command):
  117. violation = {
  118. "tool": tool_name,
  119. "command": command,
  120. "reason": "Dangerous command blocked",
  121. "timestamp": datetime.now(timezone.utc).isoformat(),
  122. }
  123. self._security_violations.append(violation)
  124. self._blocked_operations.append(violation)
  125. logger.warning(f"Blocked dangerous command: {command}")
  126. return {
  127. "hookSpecificOutput": {
  128. "hookEventName": input_data.get(
  129. "hook_event_name", "PreToolUse"
  130. ),
  131. "permissionDecision": "deny",
  132. "permissionDecisionReason": (
  133. "This command has been blocked for safety reasons. "
  134. "It could potentially cause harm to the system."
  135. ),
  136. }
  137. }
  138. # Then check bash permissions whitelist if configured
  139. if self._bash_permissions is not None and len(self._bash_permissions) > 0:
  140. if not self._is_bash_command_allowed(command):
  141. violation = {
  142. "tool": tool_name,
  143. "command": command,
  144. "reason": "Command not in allowed bash permissions",
  145. "timestamp": datetime.now(timezone.utc).isoformat(),
  146. }
  147. self._security_violations.append(violation)
  148. self._blocked_operations.append(violation)
  149. logger.warning(
  150. f"Blocked bash command not in allowed permissions: {command}"
  151. )
  152. return {
  153. "hookSpecificOutput": {
  154. "hookEventName": input_data.get(
  155. "hook_event_name", "PreToolUse"
  156. ),
  157. "permissionDecision": "deny",
  158. "permissionDecisionReason": (
  159. f"This command is not in the allowed bash permissions for this task type. "
  160. f"Allowed patterns: {', '.join(self._bash_permissions)}"
  161. ),
  162. }
  163. }
  164. # Track the tool execution (will be completed in post hook)
  165. self._tool_executions.append(
  166. {
  167. "tool_use_id": tool_use_id,
  168. "tool_name": tool_name,
  169. "tool_input": tool_input,
  170. "started_at": datetime.now(timezone.utc).isoformat(),
  171. "completed": False,
  172. }
  173. )
  174. return {}
  175. async def post_tool_audit(
  176. self,
  177. input_data: Dict[str, Any],
  178. tool_use_id: Optional[str],
  179. context: HookContext,
  180. ) -> Dict[str, Any]:
  181. """
  182. PostToolUse hook: Audit after tool execution.
  183. Logs tool usage and tracks file modifications for quality gate reporting.
  184. Args:
  185. input_data: Tool result data
  186. tool_use_id: Unique ID for correlating with PreToolUse
  187. context: Hook context (reserved for future use)
  188. Returns:
  189. Empty dict (audit-only, no blocking)
  190. """
  191. if not self.enabled:
  192. return {}
  193. tool_name = input_data.get("tool_name", "")
  194. tool_response = input_data.get("tool_response", {})
  195. # Find and update the matching execution record
  196. for execution in self._tool_executions:
  197. if execution.get("tool_use_id") == tool_use_id and not execution.get(
  198. "completed"
  199. ):
  200. execution["completed"] = True
  201. execution["completed_at"] = datetime.now(timezone.utc).isoformat()
  202. execution["response"] = tool_response
  203. break
  204. # Track file modifications
  205. if tool_name in ("Write", "Edit"):
  206. tool_input = input_data.get("tool_input", {})
  207. file_path = tool_input.get("file_path", "")
  208. if file_path and file_path not in self._files_modified:
  209. self._files_modified.append(file_path)
  210. logger.debug(f"File modified: {file_path}")
  211. logger.debug(f"Tool executed: {tool_name} (id: {tool_use_id})")
  212. return {}
  213. def _is_protected_file(self, file_path: str) -> bool:
  214. """Check if a file path is protected"""
  215. if not file_path:
  216. return False
  217. # Normalize path
  218. file_name = file_path.split("/")[-1]
  219. normalized_path = file_path.lower()
  220. for protected in self._protected_paths:
  221. protected_lower = protected.lower()
  222. if file_name.lower() == protected_lower:
  223. return True
  224. if normalized_path.endswith(protected_lower):
  225. return True
  226. if protected_lower in normalized_path:
  227. return True
  228. return False
  229. def _is_dangerous_command(self, command: str) -> bool:
  230. """Check if a bash command is dangerous"""
  231. if not command:
  232. return False
  233. command_lower = command.lower().strip()
  234. for dangerous in self._dangerous_commands:
  235. if dangerous.lower() in command_lower:
  236. return True
  237. return False
  238. def _is_bash_command_allowed(self, command: str) -> bool:
  239. """
  240. Check if a bash command matches any of the allowed permission patterns.
  241. Uses fnmatch for wildcard matching. Patterns like "pytest *" will match
  242. "pytest tests/" and "pytest -v".
  243. Args:
  244. command: The bash command to check
  245. Returns:
  246. True if command matches any allowed pattern, False otherwise
  247. """
  248. if not command:
  249. return False
  250. # None or empty list means no whitelist restrictions
  251. if self._bash_permissions is None or len(self._bash_permissions) == 0:
  252. return True
  253. command = command.strip()
  254. # Check against each allowed pattern
  255. for pattern in self._bash_permissions:
  256. if fnmatch.fnmatch(command, pattern):
  257. logger.debug(f"Bash command '{command}' matched pattern '{pattern}'")
  258. return True
  259. return False
  260. def get_execution_summary(self) -> Dict[str, Any]:
  261. """Get summary of all executions during this session"""
  262. return {
  263. "total_tool_executions": len(self._tool_executions),
  264. "completed_executions": sum(
  265. 1 for e in self._tool_executions if e.get("completed")
  266. ),
  267. "blocked_operations": len(self._blocked_operations),
  268. "security_violations": len(self._security_violations),
  269. "files_modified": self._files_modified.copy(),
  270. "violations": self._security_violations.copy(),
  271. }
  272. def reset(self) -> None:
  273. """Reset tracking state for a new session"""
  274. self._tool_executions = []
  275. self._blocked_operations = []
  276. self._files_modified = []
  277. self._security_violations = []
  278. def create_preflight_hook(
  279. checks: List[Callable[[Dict[str, Any]], bool]],
  280. ) -> Callable:
  281. """
  282. Factory to create a preflight check hook.
  283. Args:
  284. checks: List of check functions that return True if the operation is allowed
  285. Returns:
  286. Hook callback function
  287. """
  288. async def preflight_hook(
  289. input_data: Dict[str, Any],
  290. tool_use_id: Optional[str],
  291. context: HookContext,
  292. ) -> Dict[str, Any]:
  293. for check in checks:
  294. if not check(input_data):
  295. return {
  296. "hookSpecificOutput": {
  297. "hookEventName": input_data.get(
  298. "hook_event_name", "PreToolUse"
  299. ),
  300. "permissionDecision": "deny",
  301. "permissionDecisionReason": "Preflight check failed",
  302. }
  303. }
  304. return {}
  305. return preflight_hook
  306. def create_audit_hook(
  307. log_func: Optional[Callable[[str], None]] = None,
  308. ) -> Callable:
  309. """
  310. Factory to create an audit logging hook.
  311. Args:
  312. log_func: Optional custom logging function
  313. Returns:
  314. Hook callback function
  315. """
  316. log = log_func or logger.info
  317. async def audit_hook(
  318. input_data: Dict[str, Any],
  319. tool_use_id: Optional[str],
  320. context: HookContext,
  321. ) -> Dict[str, Any]:
  322. tool_name = input_data.get("tool_name", "unknown")
  323. log(f"[AUDIT] Tool executed: {tool_name}")
  324. return {}
  325. return audit_hook
  326. def create_security_hook(
  327. protected_paths: Optional[List[str]] = None,
  328. dangerous_commands: Optional[List[str]] = None,
  329. ) -> Callable:
  330. """
  331. Factory to create a security check hook.
  332. Args:
  333. protected_paths: List of file paths/patterns to protect
  334. dangerous_commands: List of dangerous command patterns to block
  335. Returns:
  336. Hook callback function
  337. """
  338. paths = protected_paths or [".env", "credentials.json", "secrets.yaml"]
  339. commands = dangerous_commands or ["rm -rf /", "rm -rf ~"]
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 413 on GitHub
  340. async def security_hook(
  341. input_data: Dict[str, Any],
  342. tool_use_id: Optional[str],
  343. context: HookContext,
  344. ) -> Dict[str, Any]:
  345. tool_name = input_data.get("tool_name", "")
  346. tool_input = input_data.get("tool_input", {})
  347. # Check file access
  348. if tool_name in ("Write", "Edit", "Read"):
  349. file_path = tool_input.get("file_path", "")
  350. for protected in paths:
  351. if protected.lower() in file_path.lower():
  352. return {
  353. "hookSpecificOutput": {
  354. "hookEventName": input_data.get(
  355. "hook_event_name", "PreToolUse"
  356. ),
  357. "permissionDecision": "deny",
  358. "permissionDecisionReason": f"Access to {protected} is blocked",
  359. }
  360. }
  361. # Check bash commands
  362. if tool_name == "Bash":
  363. command = tool_input.get("command", "")
  364. for dangerous in commands:
  365. if dangerous.lower() in command.lower():
  366. return {
  367. "hookSpecificOutput": {
  368. "hookEventName": input_data.get(
  369. "hook_event_name", "PreToolUse"
  370. ),
  371. "permissionDecision": "deny",
  372. "permissionDecisionReason": "Dangerous command blocked",
  373. }
  374. }
  375. return {}
  376. return security_hook

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.