sugar-plugins
🍰 Sugar - the coordination and evidence layer for AI agent fleets: one durable board, live Mission Control, audit exports and shared memory
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
Its scripts were only partly scanned.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/roboticforce/sugar with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/roboticforce/sugar'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/roboticforce--sugar)HTML
<a href="/listings/roboticforce--sugar"><img src="/api/listings/roboticforce--sugar/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
# Install once, use in any project pipx install 'sugarai[all]' cd ~/dev/my-app sugar init # Give the fleet work sugar add "Fix the auth timeout" --type bug_fix --urgent sugar add "Add profile settings" --type feature # Watch it, or let Sugar run it sugar dashboard --open sugar run
Your agent gets memory and the board over MCP:
claude mcp add sugar -- sugar mcp memory claude mcp add sugar-tasks -- sugar mcp tasks
# Other clients sugar opencode setup # OpenCode, one command goose configure # Goose: Add Extension -> Command-line -> sugar mcp memory
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
.claude-plugin/marketplace.json · Manifest · 23 lines
- {
- "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
- "name": "sugar-plugins",
- "version": "1.0.0",
- "description": "🍰 Sugar - the coordination and evidence layer for AI agent fleets: one durable board, live Mission Control, audit exports and shared memory",
- "owner": {
- "name": "Steven Leggett",
- "email": "contact@roboticforce.io"
- },
- "plugins": [
- {
- "name": "sugar",
- "description": "Put Claude Code and your other harnesses on one durable board: atomic claims and leases, live Mission Control, cost attribution, tamper-evident audit and shared memory",
- "version": "3.10.2",
- "author": {
- "name": "Steven Leggett",
- "email": "contact@roboticforce.io"
- },
- "source": ".",
- "category": "development"
- }
- ]
- }
README.md · README · 308 lines
- <div align="center">
- # Sugar
- **Mission control for your agent fleet.**
- One durable board for Claude Code, Codex, Cursor, Goose, OpenCode, DSH — and any harness that can claim a task.
- Atomic claims, live agent activity, cost attribution and tamper-evident history. Local-first, no account.
- <!-- mcp-name: io.github.cdnsteve/sugar -->
- [](https://pypi.org/project/sugarai/)
- [](pyproject.toml)
- [](https://github.com/roboticforce/sugar/actions/workflows/ci.yml)
- [](LICENSE)
- [](docs/local-dashboard.md)
- [Quick start](#quick-start) · [See it work](#see-your-fleet-work) · [Why Sugar](#why-sugar) · [Works with](#works-with-your-harnesses) · [Docs](docs/README.md) · [Team edition](#team-edition-self-hosted)
- <img src="docs/images/app-tasks.png" alt="SugarDash: the task board with a task inspector open, live activity and history" width="880">
- </div>
- ## Your agents can't see each other
- You run four agents across three harnesses on one repo. Then:
- - Two agents pick up the same task and overwrite each other's work.
- - A crashed run keeps a task "claimed" forever; the queue quietly stalls.
- - "What happened yesterday?" lives in six terminal tabs and a scrollback buffer.
- - You can't answer *who did what, with which harness, for which task — and what did it cost?*
- - Your agent starts every session with amnesia: last week's decisions, conventions and dead ends are gone.
- A prettier kanban board doesn't fix this. The missing layer is underneath: a **durable queue**, **atomic claims and leases**, **provenance for every actor**, and a **history you can replay and export**. That's what Sugar is.
- ## One board. Any harness. Full history.
- Sugar is local-first infrastructure for a fleet of coding agents. Every harness gets the same durable work
- queue and the same memory. Every claim, transition and tool call lands in an append-only event spine. You watch
- the fleet in a web dashboard or a native macOS app — on your machine, in your files, with no account.
- Your agents keep their own subscriptions and workflows. Sugar doesn't replace Claude Code or Codex; it sits
- between them and your repo.
- ### Coordinate — a queue agents can't double-claim
- - Durable SQLite queue with priorities, dependencies, attempts, hold/release and retries.
- - **Atomic claim + lease + heartbeat.** If a worker dies, the lease expires and a reaper requeues the task.
- - The same verbs from the CLI (`sugar add`, `sugar list`, `sugar run`), MCP tools, REST API and dashboard.
- - Keep your existing board: serve Sugar's queue, or mirror a Hermes Kanban board with `--board kanban`
- (Sugar reads it; Hermes keeps its write path).
- ### See — Mission Control for the fleet
- - Web dashboard and native macOS app, both on the same REST + SSE contract.
- - **Live floor:** every working agent with its current action, elapsed/lease countdowns and lease health.
- - **Streaming console:** tool calls, run starts/ends and usage events as they happen.
- - **Trends:** throughput, failures, cycle time and reported spend over time, by agent, harness or project.
- - Activity feed, search, filters and direct task links across all of it.
- <img src="docs/images/trends-view.png" alt="SugarDash Trends view: tasks completed and cost per agent over a week" width="880">
- ### Prove — history with receipts
- - **Append-only event spine** (`~/.sugar/events.db`, retention configurable) records every task transition and
- every harness event: actor, harness, machine, session, task, project, run and trace id.
- - **Usage and cost attribution** (`sugar usage`, `/api/usage`): tokens and cost as your harness reports them —
- no invented pricing table.
- - **Tamper-evident audit exports** (`sugar audit export` / `sugar audit verify`): canonical JSONL plus a
- SHA-256 manifest and optional HMAC signature. Export before retention prunes, keep the evidence.
- - Claims are recorded in the same transaction as the work item, so "who is holding this task?" is a query,
- not a guess.
- ### Connect — one board, many harnesses
- - **MCP servers** for tasks and memory: Claude Code, Codex, Cursor, Goose, OpenCode and any MCP client can
- read context, claim work and complete it.
- - **REST + SSE** for custom scripts, bots and CI.
- - **CLI** for everything a human needs, including `sugar emit` for harnesses that stream their own telemetry.
- - **TaskStore port + conformance suite**: SQLite today; the contract is backend-agnostic.
- ### Remember — your context, not the vendor's
- - Project memory (`.sugar/memory.db`) and global memory (`~/.sugar/memory.db`), with seven memory types:
- decisions, preferences, file context, error patterns, research, outcomes and guidelines.
- - Semantic search via local embeddings (`sugarai[memory]`), with FTS5 keyword fallback. No external API calls.
- - Agents read and write memory directly through MCP: `search_memory`, `store_learning`, `recall`,
- `get_project_context`, `list_recent_memories`.
- ## Quick start
- ```bash
- # Install once, use in any project
- pipx install 'sugarai[all]'
- cd ~/dev/my-app
- sugar init
- # Give the fleet work
- sugar add "Fix the auth timeout" --type bug_fix --urgent
- sugar add "Add profile settings" --type feature
- # Watch it, or let Sugar run it
- sugar dashboard --open
- sugar run
- ```
- Your agent gets memory and the board over MCP:
- ```bash
- claude mcp add sugar -- sugar mcp memory
- claude mcp add sugar-tasks -- sugar mcp tasks
- ```
- ```bash
- # Other clients
- sugar opencode setup # OpenCode, one command
- goose configure # Goose: Add Extension -> Command-line -> sugar mcp memory
- ```
- ## See your fleet work
- The dashboard and app are only half the story — the Live view is the part that makes a fleet feel real.
- Harnesses stream their tool calls with one line each:
- ```bash
- sugar emit --kind tool.invoked --task-id "$TASK_ID" \
- --message "running pytest tests/test_api.py" --payload file=src/api.py
- ```
- <img src="docs/images/live-floor.png" alt="SugarDash Live floor: agents at work with real leases, current actions and countdowns" width="880">
- No harness wired up yet? The repo ships a simulator for demos:
- ```bash
- git clone https://github.com/roboticforce/sugar.git && cd sugar
- python scripts/demo_dashboard.py --serve --open # demo queue + dashboard
- python scripts/simulate_agent_activity.py # live agents in a second shell
- ```
- Trends need history; seed a throwaway spine and open the Trends view:
- ```bash
- python scripts/seed_demo_trends.py --db /tmp/sugar-dashboard-demo/.sugar/events.db --reset
- ```
- Full walkthrough: [Mission Control](docs/mission-control.md) · [Telemetry](docs/telemetry.md) · [Trends](docs/trends.md)
- ## Works with your harnesses
- Any harness that can run a CLI or speak MCP can use the board and the memory. The built-in runner
- (`sugar run`) drives Claude Code through the Agent SDK today; the other harnesses keep doing their own thing
- while Sugar coordinates them.
- | Harness | Board (MCP / CLI / REST) | Shared memory | Notes |
- |---|---|---|---|
- | [Claude Code](https://docs.anthropic.com/en/docs/claude-code) | Yes | Yes | Built-in runner; plugin + Skills; full support |
- | Codex | Yes | Yes | Via MCP/CLI; no runner coupling |
- | Cursor | Yes | Yes | Via MCP |
- | [Goose](https://block.github.io/goose) | Yes | Yes | Via MCP extension |
- | [OpenCode](https://github.com/opencode-ai/opencode) | Yes | Yes | `sugar opencode setup` |
- | [Aider](https://aider.chat) | Yes | Yes | Via CLI/MCP |
- | Hermes Kanban | Yes | — | Mirror a board with `--board kanban`; Hermes owns writes |
- | Custom scripts / bots | Yes | Yes | REST + SSE, or MCP |
- ## Team edition (self-hosted)
- The default dashboard binds loopback and needs no auth. For a team, run one board on a shared machine with
- scoped tokens — reads and writes, nothing in a vendor cloud:
- ```bash
- cat ~/.sugar/board-tokens # name:scope:secret (scope: read | write; file must be 0600)
- sugar dashboard --bind 0.0.0.0 --allow-remote --token-file ~/.sugar/board-tokens
- ```
- The browser exchanges a one-time token for a `SameSite=Strict` cookie; API/SSE accept `Authorization: Bearer`.
- Remote binds are refused without both `--allow-remote` and a token source, the foreign-Origin write guard
- stays on, and TLS is expected from a reverse proxy. See [Board Server](docs/board-server.md) for the threat
- model and Caddy/nginx setup.
- What teams get today: one live board, provenance, cost attribution, and audit exports. Approval policies,
- RBAC/SSO, quotas and a hosted relay are on the roadmap — [tell us what your team needs](mailto:contact@roboticforce.io?subject=Sugar%20team%20edition).
- ## Why Sugar
- - **Harness-neutral by design.** You don't have to pick a winner. Claude Code, Codex, Cursor, Goose, OpenCode,
- DSH and your own scripts share one queue and one history.
- - **Local-first, not cloud-first.** Task state, memory and the event spine are SQLite files on your disk. No
- account, no repo upload, no vendor in the loop. Remote mode is opt-in and self-hosted.
- - **Evidence beats dashboards.** Any tool can show you a board. Sugar records the claim, the lease, the actor,
- the harness, the transitions and the spend — then lets you export and verify it.
- - **Durable, not demo-grade.** Atomic claims, lease heartbeats, crash reaping, dependency gating and retries are
- tested against a backend-agnostic conformance suite.
- - **Open and inspectable.** Dual-licensed AGPL-3.0 + commercial; the REST/SSE contract, event schema and
- adapter playbook are documented and versioned.
- **What Sugar is not:** not a model vendor, not an agent runtime or cloud coding agent, not a code host, and not
- another kanban UI. It's the neutral layer underneath the harnesses you already chose.
- <details>
- <summary><strong>FAQ</strong></summary>
- **Is this just a kanban board for agents?**
- No. A board shows status; Sugar owns the queue, the claim/lease transaction, dependency gating, provenance and
- an append-only history. It can mirror a Hermes Kanban board, but it doesn't depend on one.
- **How is this different from Claude Code Agent Teams or a single-harness orchestrator?**
- Those make one harness better at running its own agents. Sugar coordinates *across* harnesses and keeps the
- durable record. You can run Claude Code Agent Teams and still put their work on the same board as Codex.
- **Does Sugar send my code or data anywhere?**
- No. State is local SQLite by default; the dashboard binds loopback. Telemetry is what your harness explicitly
- emits (`sugar emit` / `POST /api/events`), and audit exports are files you write yourself.
- **Do I need to change how my agents work?**
- No. Add the MCP servers (tasks + memory) or use the CLI/REST API. The built-in runner is optional.
- **Is it production-ready?**
- The 3.10 line is an early release: used daily, with a large test suite and CI across Linux/macOS/Windows, but
- expect rough edges. Start with a demo board or a side project, then move your real queue over.
- </details>
- ## Install
- **Recommended: pipx** — one install, available everywhere:
- ```bash
- pipx install 'sugarai[all]' # dashboard, memory search, MCP, GitHub, OpenCode
- pipx upgrade sugarai
- ```
- <details>
- <summary>Other options</summary>
- ```bash
- pip install 'sugarai[all]' # inside a venv
- uv pip install 'sugarai[all]' # uv
- pipx install sugarai # core only (keyword memory, no dashboard)
- pipx install 'sugarai[dashboard]' # web dashboard + macOS app backend
- pipx install 'sugarai[memory]' # semantic memory (local embeddings)
- ```
- **Native macOS app:** [latest release](https://github.com/roboticforce/sugar/releases/latest) — or build it:
- ```bash
- cd apps/macos && ./build-app.sh && open dist/SugarDash.app
- ```
- **Requirements:** Python 3.11+, macOS 14+ for the app, and a CLI/MCP-capable coding agent if you want agents to
- claim work.
- </details>
- ## Also in the box
- - **Discovery** — turn GitHub issues, error logs and code analysis into queue items.
- - **Autonomous runner** — `sugar run` claims, executes, tests, commits and moves on; `--ralph` iterates until
- tests pass.
- - **Orchestration** — decompose a large task into a research → plan → implement → review workflow.
- - **Quality gates and hooks** — pre/post-execution checks, file protection and command guardrails.
- - **Agent Skills** — `sugar-memory`, `sugar-task-planner`, `sugar-quality-guardian`, `sugar-orchestrator`,
- with evaluation data in `skills/`.
- - **Claude Code plugin** — slash commands and hooks via `.claude-plugin/`.
- - **Task types, learning and memory types** — the boring machinery that makes repeated runs better.
- ## Documentation
- - [Mission Control](docs/mission-control.md) — dashboard, macOS app, event spine
- - [Local Dashboard API](docs/local-dashboard.md) — frozen REST/SSE contract
- - [Trends](docs/trends.md) · [Telemetry](docs/telemetry.md) · [Usage & cost](docs/usage.md) · [Audit exports](docs/audit.md)
- - [Board Server](docs/board-server.md) — team/remote access and TLS
- - [Storage Port](docs/storage-port.md) · [Adapters](docs/adapters.md) — backends and harness playbook
- - [Memory System](docs/user/memory.md) · [CLI Reference](docs/user/cli-reference.md) · [Quick Start](docs/user/quick-start.md)
- - [Agent Teams: market & validation](docs/agent-teams-market.md) · [Business strategy](docs/business-strategy.md)
- ## Status and roadmap
- Sugar is an early 3.10 release. The stable parts: the task store contract, the event spine schema, the REST/SSE
- contract and the audit format. The moving parts: live telemetry wiring, remote/mobile access, approval policies
- and additional runners.
- Next up, roughly in order: zero-config harness hooks (so Live lights up without manual `sugar emit`), pairing and
- a mobile-friendly Mission Control, permission modes with ledger-recorded approvals, then a Postgres backend.
- Ideas and bug reports are welcome — [open an issue](https://github.com/roboticforce/sugar/issues).
- ## Contributing
- ```bash
- git clone https://github.com/roboticforce/sugar.git
- cd sugar && git checkout develop
- uv pip install -e ".[dev,test,github,mcp]"
- pytest tests/ -v
- ```
- See [CONTRIBUTING](docs/dev/contributing.md). PRs target `develop`; releases go to `main`.
- ## License
- **Dual license: AGPL-3.0 + commercial.**
- - **AGPL-3.0** — free for open source and personal use.
- - **Commercial license** — for proprietary use. [Contact RoboticForce](mailto:contact@roboticforce.io) or see
- [LICENSE](LICENSE).
- ---
- > Sugar is provided "AS IS" without warranty. Review all AI-generated code before use.
scripts/benchmark_v3.py · Bundled script · 383 lines · 3 findings marked
- #!/usr/bin/env python3
- """
- Sugar V3 Benchmark Script
- Measures the concrete overhead improvements of V3 (Agent SDK) vs V2 (subprocess).
- These benchmarks focus on what V3 actually improves - startup overhead,
- hook latency, and memory footprint - NOT Claude API call time which is identical.
- Usage:
- python scripts/benchmark_v3.py
- """
- import asyncio
- import statistics
- import sys
- import time
- import tracemalloc
- from dataclasses import dataclass
- from pathlib import Path
- # Add project root to path
- sys.path.insert(0, str(Path(__file__).parent.parent))
- @dataclass
- class BenchmarkResult:
- """Results from a benchmark run."""
- name: str
- samples: list[float]
- unit: str
- @property
- def mean(self) -> float:
- return statistics.mean(self.samples)
- @property
- def median(self) -> float:
- return statistics.median(self.samples)
- @property
- def stdev(self) -> float:
- return statistics.stdev(self.samples) if len(self.samples) > 1 else 0
- @property
- def min_val(self) -> float:
- return min(self.samples)
- @property
- def max_val(self) -> float:
- return max(self.samples)
- def __str__(self) -> str:
- return (
- f"{self.name}:\n"
- f" Mean: {self.mean:.2f} {self.unit}\n"
- f" Median: {self.median:.2f} {self.unit}\n"
- f" Std Dev: {self.stdev:.2f} {self.unit}\n"
- f" Range: {self.min_val:.2f} - {self.max_val:.2f} {self.unit}"
- )
- def benchmark_v3_import_time(iterations: int = 10) -> BenchmarkResult:
- """Measure time to import V3 agent modules."""
- import importlib
- times = []
- for _ in range(iterations):
- # Clear cached imports
- modules_to_clear = [k for k in sys.modules.keys() if k.startswith('sugar.agent')]
- for mod in modules_to_clear:
- del sys.modules[mod]
- start = time.perf_counter()
- importlib.import_module('sugar.agent.base')
- importlib.import_module('sugar.agent.hooks')
- elapsed = (time.perf_counter() - start) * 1000
- times.append(elapsed)
- return BenchmarkResult("V3 Module Import Time", times, "ms")
- def benchmark_v3_config_creation(iterations: int = 100) -> BenchmarkResult:
- """Measure time to create agent config."""
- from sugar.agent.base import SugarAgentConfig
- times = []
- for _ in range(iterations):
- start = time.perf_counter()
- config = SugarAgentConfig(
- model="claude-sonnet-4-20250514",
- max_tokens=8192,
- permission_mode="acceptEdits",
- quality_gates_enabled=True,
- )
- elapsed = (time.perf_counter() - start) * 1000
- times.append(elapsed)
- return BenchmarkResult("V3 Config Creation", times, "ms")
- def benchmark_v3_agent_creation(iterations: int = 50) -> BenchmarkResult:
- """Measure time to create SugarAgent instance."""
- from sugar.agent.base import SugarAgent, SugarAgentConfig
- config = SugarAgentConfig(
- model="claude-sonnet-4-20250514",
- max_tokens=8192,
- permission_mode="acceptEdits",
- )
- times = []
- for _ in range(iterations):
- start = time.perf_counter()
- agent = SugarAgent(config)
- elapsed = (time.perf_counter() - start) * 1000
- times.append(elapsed)
- return BenchmarkResult("V3 Agent Creation", times, "ms")
- def benchmark_hooks_creation(iterations: int = 100) -> BenchmarkResult:
- """Measure time to create quality gate hooks."""
- from sugar.agent.hooks import QualityGateHooks
- times = []
- for _ in range(iterations):
- start = time.perf_counter()
- hooks = QualityGateHooks({
- "protected_files": [".env", "*.pem", "credentials.json"],
- "blocked_commands": ["sudo", "rm -rf /"],
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 130 on GitHub
- })
- elapsed = (time.perf_counter() - start) * 1000
- times.append(elapsed)
- return BenchmarkResult("Quality Gate Hooks Creation", times, "ms")
- async def benchmark_hooks_check(iterations: int = 100) -> BenchmarkResult:
- """Measure time for a security check on a file path."""
- from sugar.agent.hooks import QualityGateHooks
- hooks = QualityGateHooks({
- "protected_files": [".env", "*.pem", "credentials.json", "secrets/*"],
- "blocked_commands": ["sudo", "rm -rf /", "chmod 777"],
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 144 on GitHub
- })
- # Simulate tool use input
- test_inputs = [
- {"file_path": "/project/src/main.py"},
- {"file_path": "/project/.env"},
- {"file_path": "/project/config/settings.yaml"},
- {"command": "ls -la"},
- {"command": "sudo rm -rf /"},
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 153 on GitHub
- ]
- times = []
- for i in range(iterations):
- input_data = test_inputs[i % len(test_inputs)]
- start = time.perf_counter()
- result = await hooks.pre_tool_security_check(
- input_data=input_data,
- tool_use_id=f"test_{i}",
- context={"tool_name": "Write" if "file_path" in input_data else "Bash"}
- )
- elapsed = (time.perf_counter() - start) * 1000
- times.append(elapsed)
- return BenchmarkResult("Security Check Latency", times, "ms")
- def benchmark_response_serialization(iterations: int = 100) -> BenchmarkResult:
- """Measure time to serialize agent response."""
- from sugar.agent.base import AgentResponse
- response = AgentResponse(
- success=True,
- content="Task completed successfully. Created new file.",
- tool_uses=[
- {"tool": "Write", "input": {"file_path": "/project/src/new.py"}},
- {"tool": "Bash", "input": {"command": "python -m pytest"}},
- ],
- files_modified=["/project/src/new.py", "/project/tests/test_new.py"],
- execution_time=45.2,
- error=None,
- quality_gate_results={
- "total_tool_executions": 5,
- "blocked_operations": 0,
- "security_violations": 0,
- },
- )
- times = []
- for _ in range(iterations):
- start = time.perf_counter()
- data = response.to_dict()
- elapsed = (time.perf_counter() - start) * 1000
- times.append(elapsed)
- return BenchmarkResult("Response Serialization", times, "ms")
- def benchmark_memory_footprint() -> dict:
- """Measure memory footprint of V3 components."""
- import gc
- results = {}
- # Baseline
- gc.collect()
- tracemalloc.start()
- baseline = tracemalloc.get_traced_memory()[0]
- # Import agent modules
- from sugar.agent.base import SugarAgent, SugarAgentConfig
- from sugar.agent.hooks import QualityGateHooks
- gc.collect()
- after_import = tracemalloc.get_traced_memory()[0]
- results["module_import_mb"] = (after_import - baseline) / (1024 * 1024)
- # Create config
- config = SugarAgentConfig(
- model="claude-sonnet-4-20250514",
- max_tokens=8192,
- permission_mode="acceptEdits",
- quality_gates_enabled=True,
- )
- gc.collect()
- after_config = tracemalloc.get_traced_memory()[0]
- results["config_mb"] = (after_config - after_import) / (1024 * 1024)
- # Create agent
- agent = SugarAgent(config)
- gc.collect()
- after_agent = tracemalloc.get_traced_memory()[0]
- results["agent_mb"] = (after_agent - after_config) / (1024 * 1024)
- # Create hooks
- hooks = QualityGateHooks({
- "protected_files": [".env", "*.pem"],
- "blocked_commands": ["sudo"],
- })
- gc.collect()
- after_hooks = tracemalloc.get_traced_memory()[0]
- results["hooks_mb"] = (after_hooks - after_agent) / (1024 * 1024)
- # Total
- results["total_mb"] = (after_hooks - baseline) / (1024 * 1024)
- tracemalloc.stop()
- return results
- def benchmark_transient_error_detection(iterations: int = 100) -> BenchmarkResult:
- """Measure time to detect transient errors."""
- from sugar.agent.base import is_transient_error
- test_errors = [
- Exception("rate_limit exceeded"),
- Exception("Connection timeout"),
- Exception("503 Service Unavailable"),
- Exception("429 Too Many Requests"),
- Exception("API overloaded"),
- Exception("Invalid request"), # Not transient
- ValueError("Bad input"), # Not transient
- ]
- times = []
- for i in range(iterations):
- error = test_errors[i % len(test_errors)]
- start = time.perf_counter()
- is_transient_error(error)
- elapsed = (time.perf_counter() - start) * 1000
- times.append(elapsed)
- return BenchmarkResult("Transient Error Detection", times, "ms")
- async def run_benchmarks():
- """Run all benchmarks and display results."""
- print("=" * 60)
- print("Sugar V3 Benchmark Suite")
- print("=" * 60)
- print()
- print("These benchmarks measure V3 overhead improvements.")
- print("Note: Claude API call time is NOT measured (identical in V2/V3).")
- print()
- print("-" * 60)
- results = []
- # Timing benchmarks
- print("\nRunning timing benchmarks...")
- results.append(benchmark_v3_import_time())
- print(f" - Module import: {results[-1].mean:.2f}ms")
- results.append(benchmark_v3_config_creation())
- print(f" - Config creation: {results[-1].mean:.3f}ms")
- results.append(benchmark_v3_agent_creation())
- print(f" - Agent creation: {results[-1].mean:.3f}ms")
- results.append(benchmark_hooks_creation())
- print(f" - Hooks creation: {results[-1].mean:.3f}ms")
- results.append(await benchmark_hooks_check())
- print(f" - Security check: {results[-1].mean:.3f}ms")
- results.append(benchmark_response_serialization())
- print(f" - Response serialization: {results[-1].mean:.3f}ms")
- results.append(benchmark_transient_error_detection())
- print(f" - Error detection: {results[-1].mean:.4f}ms")
- # Memory benchmark
- print("\nRunning memory benchmarks...")
- memory = benchmark_memory_footprint()
- # Calculate total startup overhead
- total_startup = sum(r.mean for r in results[:4])
- print()
- print("=" * 60)
- print("RESULTS SUMMARY")
- print("=" * 60)
- print("\n## Startup Overhead (V3)")
- print(f" Total initialization: {total_startup:.2f}ms")
- print(f" - Module import: {results[0].mean:.2f}ms")
- print(f" - Config creation: {results[1].mean:.3f}ms")
- print(f" - Agent creation: {results[2].mean:.3f}ms")
- print(f" - Hooks creation: {results[3].mean:.3f}ms")
- print("\n## Per-Operation Latency")
- print(f" Security check: {results[4].mean:.3f}ms")
- print(f" Response serialization: {results[5].mean:.3f}ms")
- print(f" Error detection: {results[6].mean:.4f}ms")
- print("\n## Memory Footprint")
- print(f" Total V3 components: {memory['total_mb']:.2f}MB")
- print(f" - Module import: {memory['module_import_mb']:.2f}MB")
- print(f" - Agent instance: {memory['agent_mb']:.3f}MB")
- print(f" - Hooks instance: {memory['hooks_mb']:.3f}MB")
- print("\n## Comparison with V2 (Subprocess)")
- print(" V2 subprocess spawn: ~300-500ms")
- print(f" V3 native startup: ~{total_startup:.0f}ms")
- print(f" Improvement: ~{(400 - total_startup):.0f}ms faster")
- print()
- print(" V2 memory (separate process): ~50-100MB additional")
- print(f" V3 memory (shared process): ~{memory['total_mb']:.1f}MB additional")
- print(f" Improvement: ~{50 - memory['total_mb']:.0f}MB less")
- print()
- print("=" * 60)
- print("CONTEXT")
- print("=" * 60)
- print("""
- These improvements apply to OVERHEAD only. The actual Claude API
- call takes 30-300 seconds depending on task complexity.
- Example total task time breakdown:
- V2: 400ms overhead + 60s Claude = 60.4s total
- V3: 50ms overhead + 60s Claude = 60.05s total
- The ~350ms improvement is real but represents < 1% of total time.
- V3's main benefits are:
- 1. Security gates (can BLOCK dangerous operations)
- 2. Observability (structured tool tracking)
- 3. Reliability (automatic retry on transient errors)
- """)
- return results, memory
- if __name__ == "__main__":
- asyncio.run(run_benchmarks())
sugar/agent/hooks.py · Bundled script · 455 lines · 5 findings marked
- """
- Quality Gate Hooks for Claude Agent SDK
- These hooks integrate Sugar's quality gates system with the Agent SDK's
- PreToolUse and PostToolUse hook points.
- """
- import fnmatch
- import logging
- from datetime import datetime, timezone
- from typing import Any, Callable, Dict, List, Optional
- logger = logging.getLogger(__name__)
- class HookContext:
- """Context passed to hook callbacks (placeholder for SDK's HookContext)"""
- pass
- class QualityGateHooks:
- """
- Quality gate hooks for the Claude Agent SDK.
- Provides PreToolUse and PostToolUse hooks that enforce Sugar's
- quality gates during agent execution.
- """
- def __init__(
- self,
- config: Optional[Dict[str, Any]] = None,
- bash_permissions: Optional[List[str]] = None,
- ):
- """
- Initialize quality gate hooks.
- Args:
- config: Quality gates configuration
- bash_permissions: List of allowed bash command patterns (e.g., ["pytest *", "git status*"])
- Empty list means all commands allowed, None means use default restrictions
- """
- self.config = config or {}
- self.enabled = self.config.get("enabled", True)
- # Tracking state
- self._tool_executions: List[Dict[str, Any]] = []
- self._blocked_operations: List[Dict[str, Any]] = []
- self._files_modified: List[str] = []
- self._security_violations: List[Dict[str, Any]] = []
- # Configuration
- self._protected_paths = self.config.get(
- "protected_paths",
- [
- ".env",
- ".env.local",
- "credentials.json",
- "secrets.yaml",
- ".git/config",
- ],
- )
- self._dangerous_commands = self.config.get(
- "dangerous_commands",
- [
- "rm -rf /",
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 66 on GitHub
- "rm -rf ~",
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 67 on GitHub
- "> /dev/sda",
- "mkfs.",
- ":(){:|:&};:",
- Note: Quoted as data in code: Matches a classic shell fork-bomb pattern
- Line 70 on GitHub
- "chmod -R 777 /",
- Note: Quoted as data in code: Sets world-writable permissions on a root path (chmod 777 /)
- Line 71 on GitHub
- ],
- )
- # Bash command permissions (wildcard patterns)
- # Empty list = all allowed, None = use dangerous commands check only
- self._bash_permissions = bash_permissions
- async def pre_tool_security_check(
- self,
- input_data: Dict[str, Any],
- tool_use_id: Optional[str],
- context: HookContext,
- ) -> Dict[str, Any]:
- """
- PreToolUse hook: Security check before tool execution.
- Checks for:
- - Protected file access (.env, credentials, etc.)
- - Dangerous bash commands
- - Unauthorized modifications
- Args:
- input_data: Tool input data with 'tool_name' and 'tool_input'
- tool_use_id: Unique ID for correlating with PostToolUse
- context: Hook context (reserved for future use)
- Returns:
- Empty dict to allow, or dict with hookSpecificOutput to block
- """
- if not self.enabled:
- return {}
- tool_name = input_data.get("tool_name", "")
- tool_input = input_data.get("tool_input", {})
- # Check for protected file access
- if tool_name in ("Write", "Edit", "Read"):
- file_path = tool_input.get("file_path", "")
- if self._is_protected_file(file_path):
- violation = {
- "tool": tool_name,
- "file": file_path,
- "reason": "Protected file access blocked",
- "timestamp": datetime.now(timezone.utc).isoformat(),
- }
- self._security_violations.append(violation)
- self._blocked_operations.append(violation)
- logger.warning(f"Blocked access to protected file: {file_path}")
- return {
- "hookSpecificOutput": {
- "hookEventName": input_data.get(
- "hook_event_name", "PreToolUse"
- ),
- "permissionDecision": "deny",
- "permissionDecisionReason": (
- f"Access to protected file '{file_path}' is not allowed. "
- "This file may contain sensitive information."
- ),
- }
- }
- # Check for bash command permissions (dangerous commands + whitelist)
- if tool_name == "Bash":
- command = tool_input.get("command", "")
- # First check dangerous commands (always blocked)
- if self._is_dangerous_command(command):
- violation = {
- "tool": tool_name,
- "command": command,
- "reason": "Dangerous command blocked",
- "timestamp": datetime.now(timezone.utc).isoformat(),
- }
- self._security_violations.append(violation)
- self._blocked_operations.append(violation)
- logger.warning(f"Blocked dangerous command: {command}")
- return {
- "hookSpecificOutput": {
- "hookEventName": input_data.get(
- "hook_event_name", "PreToolUse"
- ),
- "permissionDecision": "deny",
- "permissionDecisionReason": (
- "This command has been blocked for safety reasons. "
- "It could potentially cause harm to the system."
- ),
- }
- }
- # Then check bash permissions whitelist if configured
- if self._bash_permissions is not None and len(self._bash_permissions) > 0:
- if not self._is_bash_command_allowed(command):
- violation = {
- "tool": tool_name,
- "command": command,
- "reason": "Command not in allowed bash permissions",
- "timestamp": datetime.now(timezone.utc).isoformat(),
- }
- self._security_violations.append(violation)
- self._blocked_operations.append(violation)
- logger.warning(
- f"Blocked bash command not in allowed permissions: {command}"
- )
- return {
- "hookSpecificOutput": {
- "hookEventName": input_data.get(
- "hook_event_name", "PreToolUse"
- ),
- "permissionDecision": "deny",
- "permissionDecisionReason": (
- f"This command is not in the allowed bash permissions for this task type. "
- f"Allowed patterns: {', '.join(self._bash_permissions)}"
- ),
- }
- }
- # Track the tool execution (will be completed in post hook)
- self._tool_executions.append(
- {
- "tool_use_id": tool_use_id,
- "tool_name": tool_name,
- "tool_input": tool_input,
- "started_at": datetime.now(timezone.utc).isoformat(),
- "completed": False,
- }
- )
- return {}
- async def post_tool_audit(
- self,
- input_data: Dict[str, Any],
- tool_use_id: Optional[str],
- context: HookContext,
- ) -> Dict[str, Any]:
- """
- PostToolUse hook: Audit after tool execution.
- Logs tool usage and tracks file modifications for quality gate reporting.
- Args:
- input_data: Tool result data
- tool_use_id: Unique ID for correlating with PreToolUse
- context: Hook context (reserved for future use)
- Returns:
- Empty dict (audit-only, no blocking)
- """
- if not self.enabled:
- return {}
- tool_name = input_data.get("tool_name", "")
- tool_response = input_data.get("tool_response", {})
- # Find and update the matching execution record
- for execution in self._tool_executions:
- if execution.get("tool_use_id") == tool_use_id and not execution.get(
- "completed"
- ):
- execution["completed"] = True
- execution["completed_at"] = datetime.now(timezone.utc).isoformat()
- execution["response"] = tool_response
- break
- # Track file modifications
- if tool_name in ("Write", "Edit"):
- tool_input = input_data.get("tool_input", {})
- file_path = tool_input.get("file_path", "")
- if file_path and file_path not in self._files_modified:
- self._files_modified.append(file_path)
- logger.debug(f"File modified: {file_path}")
- logger.debug(f"Tool executed: {tool_name} (id: {tool_use_id})")
- return {}
- def _is_protected_file(self, file_path: str) -> bool:
- """Check if a file path is protected"""
- if not file_path:
- return False
- # Normalize path
- file_name = file_path.split("/")[-1]
- normalized_path = file_path.lower()
- for protected in self._protected_paths:
- protected_lower = protected.lower()
- if file_name.lower() == protected_lower:
- return True
- if normalized_path.endswith(protected_lower):
- return True
- if protected_lower in normalized_path:
- return True
- return False
- def _is_dangerous_command(self, command: str) -> bool:
- """Check if a bash command is dangerous"""
- if not command:
- return False
- command_lower = command.lower().strip()
- for dangerous in self._dangerous_commands:
- if dangerous.lower() in command_lower:
- return True
- return False
- def _is_bash_command_allowed(self, command: str) -> bool:
- """
- Check if a bash command matches any of the allowed permission patterns.
- Uses fnmatch for wildcard matching. Patterns like "pytest *" will match
- "pytest tests/" and "pytest -v".
- Args:
- command: The bash command to check
- Returns:
- True if command matches any allowed pattern, False otherwise
- """
- if not command:
- return False
- # None or empty list means no whitelist restrictions
- if self._bash_permissions is None or len(self._bash_permissions) == 0:
- return True
- command = command.strip()
- # Check against each allowed pattern
- for pattern in self._bash_permissions:
- if fnmatch.fnmatch(command, pattern):
- logger.debug(f"Bash command '{command}' matched pattern '{pattern}'")
- return True
- return False
- def get_execution_summary(self) -> Dict[str, Any]:
- """Get summary of all executions during this session"""
- return {
- "total_tool_executions": len(self._tool_executions),
- "completed_executions": sum(
- 1 for e in self._tool_executions if e.get("completed")
- ),
- "blocked_operations": len(self._blocked_operations),
- "security_violations": len(self._security_violations),
- "files_modified": self._files_modified.copy(),
- "violations": self._security_violations.copy(),
- }
- def reset(self) -> None:
- """Reset tracking state for a new session"""
- self._tool_executions = []
- self._blocked_operations = []
- self._files_modified = []
- self._security_violations = []
- def create_preflight_hook(
- checks: List[Callable[[Dict[str, Any]], bool]],
- ) -> Callable:
- """
- Factory to create a preflight check hook.
- Args:
- checks: List of check functions that return True if the operation is allowed
- Returns:
- Hook callback function
- """
- async def preflight_hook(
- input_data: Dict[str, Any],
- tool_use_id: Optional[str],
- context: HookContext,
- ) -> Dict[str, Any]:
- for check in checks:
- if not check(input_data):
- return {
- "hookSpecificOutput": {
- "hookEventName": input_data.get(
- "hook_event_name", "PreToolUse"
- ),
- "permissionDecision": "deny",
- "permissionDecisionReason": "Preflight check failed",
- }
- }
- return {}
- return preflight_hook
- def create_audit_hook(
- log_func: Optional[Callable[[str], None]] = None,
- ) -> Callable:
- """
- Factory to create an audit logging hook.
- Args:
- log_func: Optional custom logging function
- Returns:
- Hook callback function
- """
- log = log_func or logger.info
- async def audit_hook(
- input_data: Dict[str, Any],
- tool_use_id: Optional[str],
- context: HookContext,
- ) -> Dict[str, Any]:
- tool_name = input_data.get("tool_name", "unknown")
- log(f"[AUDIT] Tool executed: {tool_name}")
- return {}
- return audit_hook
- def create_security_hook(
- protected_paths: Optional[List[str]] = None,
- dangerous_commands: Optional[List[str]] = None,
- ) -> Callable:
- """
- Factory to create a security check hook.
- Args:
- protected_paths: List of file paths/patterns to protect
- dangerous_commands: List of dangerous command patterns to block
- Returns:
- Hook callback function
- """
- paths = protected_paths or [".env", "credentials.json", "secrets.yaml"]
- commands = dangerous_commands or ["rm -rf /", "rm -rf ~"]
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 413 on GitHub
- async def security_hook(
- input_data: Dict[str, Any],
- tool_use_id: Optional[str],
- context: HookContext,
- ) -> Dict[str, Any]:
- tool_name = input_data.get("tool_name", "")
- tool_input = input_data.get("tool_input", {})
- # Check file access
- if tool_name in ("Write", "Edit", "Read"):
- file_path = tool_input.get("file_path", "")
- for protected in paths:
- if protected.lower() in file_path.lower():
- return {
- "hookSpecificOutput": {
- "hookEventName": input_data.get(
- "hook_event_name", "PreToolUse"
- ),
- "permissionDecision": "deny",
- "permissionDecisionReason": f"Access to {protected} is blocked",
- }
- }
- # Check bash commands
- if tool_name == "Bash":
- command = tool_input.get("command", "")
- for dangerous in commands:
- if dangerous.lower() in command.lower():
- return {
- "hookSpecificOutput": {
- "hookEventName": input_data.get(
- "hook_event_name", "PreToolUse"
- ),
- "permissionDecision": "deny",
- "permissionDecisionReason": "Dangerous command blocked",
- }
- }
- return {}
- return security_hook
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.