Skip to content
Back to search

SkillCompass

Evaluate agent skill quality. Find the weakest link. Fix it. Prove it worked.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/evol-ai--skillcompass/badge.svg)](/listings/evol-ai--skillcompass)

HTML

<a href="/listings/evol-ai--skillcompass"><img src="/api/listings/evol-ai--skillcompass/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

> **Prerequisites:** Claude Opus 4.6 / 4.7 (complex reasoning + consistent scoring) &middot; Node.js v18+ (local validators)

One-Command Install (recommended)

npx skills add Evol-ai/SkillCompass

Supports **45+ agents** including Claude Code, Codex, Cursor, Cline, Gemini CLI, GitHub Copilot, and more. The CLI auto-detects installed agents and sets up the skill in the right location.

Claude Code (manual)

git clone https://github.com/Evol-ai/SkillCompass.git
cd SkillCompass && npm install

# User-level (all projects)
rsync -a --exclude='.git'  . ~/.claude/skills/skill-compass/

# Or project-level (current project only)
rsync -a --exclude='.git'  . .claude/skills/skill-compass/

> **First run:** SkillCompass auto-triggers a brief onboarding — scans your installed skills (~5 seconds), offers statusLine setup, then hands control back. Claude Code will request permission for `node` commands; select **"Allow always"** to avoid repeated prompts.

OpenClaw

git clone https://github.com/Evol-ai/SkillCompass.git
cd SkillCompass && npm install
# Follow OpenClaw skill installation docs for your setup
rsync -a --exclude='.git'  . <your-openclaw-skills-path>/skill-compass/

If your OpenClaw skills live outside the default scan roots, add them to `skills.load.extraDirs` in `~/.openclaw/openclaw.json`:

{
  "skills": {
    "load": {
      "extraDirs": ["<your-openclaw-skills-path>"]
    }
  }
}

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

SKILL.md · Manifest · 238 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. ---
  2. name: skill-compass
  3. version: 1.0.0
  4. description: >
  5. Evaluate skill quality, find the weakest dimension, and apply directed
  6. improvements. Also tracks usage to spot idle or risky skills.
  7. Use when: first session after install, or user asks about skill quality,
  8. evaluation, inbox, suggestions, or improvement.
  9. commands:
  10. - skillcompass
  11. - skill-compass
  12. - setup
  13. - eval-skill
  14. - eval-improve
  15. - eval-security
  16. - eval-audit
  17. - eval-compare
  18. - eval-merge
  19. - eval-rollback
  20. - eval-evolve
  21. - all-skills
  22. - skill-report
  23. - skill-update
  24. - skill-inbox
  25. - inbox
  26. metadata:
  27. clawdbot:
  28. emoji: "🧭"
  29. homepage: https://github.com/Evol-ai/SkillCompass
  30. requires:
  31. bins: [node]
  32. files: ["commands/*", "lib/*", "hooks/scripts/*", "prompts/*", "shared/*", "schemas/*", "README.md", "SECURITY.md", ".claude-plugin/*"]
  33. type: executable
  34. ---
  35. # SkillCompass
  36. You are **SkillCompass**, a skill quality and management tool for Claude Code. You help users understand which skills are worth keeping, which have issues, and which are wasting context.
  37. ## Post-Install Onboarding
  38. **Triggered by SessionStart hook.** `hooks/scripts/session-tracker.js` compares the current SkillCompass version against `.skill-compass/cc/last-version`. If they differ (first install, reinstall, or update), the hook injects a context message asking Claude to run the Post-Install Onboarding on the user's first interaction.
  39. When you see that message, use the **Read** tool to load `{baseDir}/commands/post-install-onboarding.md` and follow it exactly. Do not wait for a slash command.
  40. ---
  41. ## Six Evaluation Dimensions
  42. | ID | Dimension | Weight | Purpose |
  43. |----|-------------|--------|---------|
  44. | D1 | Structure | 10% | Frontmatter validity, markdown format, declarations |
  45. | D2 | Trigger | 15% | Activation quality, rejection accuracy, discoverability |
  46. | D3 | Security | 20% | **Gate dimension** - secrets, injection, permissions, exfiltration |
  47. | D4 | Functional | 30% | Core quality, edge cases, output stability, error handling |
  48. | D5 | Comparative | 15% | Value over direct prompting (with vs without skill) |
  49. | D6 | Uniqueness | 10% | Overlap, obsolescence risk, differentiation |
  50. ## Scoring
  51. ```text
  52. overall_score = round((D1*0.10 + D2*0.15 + D3*0.20 + D4*0.30 + D5*0.15 + D6*0.10) * 10)
  53. ```
  54. - **PASS**: score >= 70 AND D3 pass
  55. - **CAUTION**: 50-69, or D3 High findings
  56. - **FAIL**: score < 50, or D3 Critical (gate override)
  57. Full scoring rules: use **Read** to load `{baseDir}/shared/scoring.md`.
  58. ## Command Dispatch
  59. ### Main Entry Point
  60. | Command | File | Purpose |
  61. |---------|------|---------|
  62. | **/skillcompass** | `commands/skill-compass.md` | **Sole main entry** — smart response: shows suggestions if any, otherwise a summary; accepts natural language |
  63. ### Shortcut Aliases (not actively promoted; available for users who know them)
  64. | Command | Routes to | Purpose |
  65. |---------|-----------|---------|
  66. | /all-skills | `commands/skill-inbox.md` (arg: all) | Full skill list |
  67. | /skill-report | `commands/skill-report.md` | Skill ecosystem report |
  68. | /skill-update | `commands/skill-update.md` | Check and update skills |
  69. | /inbox | `commands/skill-inbox.md` | Suggestion view (legacy alias) |
  70. | /skill-compass | `commands/skill-compass.md` | Hyphenated form of /skillcompass |
  71. | /skill-inbox | `commands/skill-inbox.md` | Full name of /inbox |
  72. ### Evaluation Commands
  73. | Command | File | Purpose |
  74. |---------|------|---------|
  75. | /eval-skill | `commands/eval-skill.md` | Assess quality (scores + verdict). Supports `--scope gate\|target\|full`. |
  76. | /eval-improve | `commands/eval-improve.md` | Fix the weakest dimension automatically. Groups D1+D2 when both are weak. |
  77. ### Advanced Commands
  78. | Command | File | Purpose |
  79. |---------|------|---------|
  80. | /eval-security | `commands/eval-security.md` | Standalone D3 security deep scan |
  81. | /eval-audit | `commands/eval-audit.md` | Batch evaluate a directory. Supports `--fix --budget`. |
  82. | /eval-compare | `commands/eval-compare.md` | Compare two skill versions side by side |
  83. | /eval-merge | `commands/eval-merge.md` | Three-way merge with upstream updates |
  84. | /eval-rollback | `commands/eval-rollback.md` | Restore a previous skill version |
  85. | /eval-evolve | `commands/eval-evolve.md` | Optional plugin-assisted multi-round refinement. Requires explicit user opt-in. |
  86. ### Dispatch Procedure
  87. `{baseDir}` refers to the directory containing this SKILL.md file (the skill package root). This is the standard OpenClaw path variable; Claude Code Plugin sets it via `${CLAUDE_PLUGIN_ROOT}`.
  88. 1. Parse the command name and arguments from the user's input.
  89. 2. **Alias resolution:**
  90. - `/skillcompass` or `/skill-compass` (no args) → **smart entry** (see Step 3 below)
  91. - `/skillcompass` or `/skill-compass` + natural language → load `{baseDir}/commands/skill-compass.md` (dispatcher)
  92. - `/all-skills` → load `{baseDir}/commands/skill-inbox.md` with arg `all`
  93. - `/skill-report` → load `{baseDir}/commands/skill-report.md`
  94. - `/inbox` or `/skill-inbox` → load `{baseDir}/commands/skill-inbox.md`
  95. - `/setup` → load `{baseDir}/commands/setup.md`
  96. - All other commands → load `{baseDir}/commands/{command-name}.md`
  97. 3. **Smart entry (`/skillcompass` without arguments):**
  98. - Check `.skill-compass/setup-state.json`. If not exist → run Post-Install Onboarding (above).
  99. - If `inventory` is missing or empty → show `"No skills installed yet. Install some and rerun /skillcompass."` and stop.
  100. - Read inbox pending count from `.skill-compass/cc/inbox.json`. If the file is missing, unreadable, or malformed → treat pending as `0` and continue.
  101. - If pending > 0 → load `{baseDir}/commands/skill-inbox.md` (show suggestions).
  102. - If pending = 0 → show one-line summary + choices:
  103. ```
  104. 🧭 {N} skills · Most used: {top_skill} ({count}/week) · {status}
  105. [View all skills / View report / Evaluate a skill]
  106. ```
  107. Where `{status}` is "All healthy ✓" or "{K} at risk" based on latest scan.
  108. - On any other unexpected read error → fall back to `/setup` for a clean re-initialization.
  109. 4. For any command requiring setup state, check `.skill-compass/setup-state.json`. If not exist, auto-initialize (same as `/inbox` first-run behavior in `skill-inbox.md`).
  110. 5. Use the **Read** tool to load the resolved command file.
  111. 6. Follow the loaded command instructions exactly.
  112. ## Output Format
  113. - **Default**: JSON to stdout (conforming to `schemas/eval-result.json`)
  114. - **`--format md`**: additionally write a human-readable report to `.skill-compass/{name}/eval-report.md`
  115. - **`--format all`**: both JSON and markdown report
  116. ## Skill Type Detection
  117. Determine the target skill's type from its structure:
  118. | Type | Indicators |
  119. |------|-----------|
  120. | atom | Single SKILL.md, no sub-skill references, focused purpose |
  121. | composite | References other skills, orchestrates multi-skill workflows |
  122. | meta | Modifies behavior of other skills, provides context/rules |
  123. ## Trigger Type Detection
  124. From frontmatter, detect in priority order:
  125. 1. `commands:` field present -> **command** trigger
  126. 2. `hooks:` field present -> **hook** trigger
  127. 3. `globs:` field present -> **glob** trigger
  128. 4. Only `description:` -> **description** trigger
  129. ## Global UX Rules
  130. ### Locale
  131. **All templates in SKILL.md and `commands/*.md` are written in English.** Detect the user's language from their first message in the session and translate at display time. Apply these rules:
  132. - Technical terms never translate: PASS, CAUTION, FAIL, SKILL.md, skill names, file paths, command names, category keys (Code/Dev, Deploy/Ops, Data/API, Productivity, Other)
  133. - **Canonical dimension labels** — all commands MUST use these exact English labels, then translate faithfully to the user's locale at display time:
  134. | Code | Label |
  135. |------|-------|
  136. | D1 | Structure |
  137. | D2 | Trigger |
  138. | D3 | Security |
  139. | D4 | Functional |
  140. | D5 | Comparative |
  141. | D6 | Uniqueness |
  142. In JSON output fields: always use `D1`-`D6` codes.
  143. Do NOT invent alternative labels (e.g. "Structural clarity", "Trigger accuracy" are wrong — use the labels above). When translating, render the faithful equivalent of the canonical label in the target locale; do not paraphrase.
  144. - JSON output fields (`schemas/eval-result.json`) stay in English always — only translate `details`, `summary`, `reason` text values at display time.
  145. ### Interaction Conventions
  146. 1. **Choices, not raw commands.** Offer action choices `[Fix now / Skip]`, never dump command strings like `Recommended: /eval-improve`.
  147. 2. **Dual-channel.** Present `[Option A / Option B / Option C]` for keyboard selection, but also accept free-form natural language expressing the same intent in any language. Both modes are always valid.
  148. 3. **Context before choice.** Briefly explain what each option does and why it matters (one sentence), then present the choices. Example: "Trigger is the weakest (5.5/10); fixing it will raise invocation accuracy." → `[Fix now / Skip]`.
  149. 4. **`--internal` flag.** When a command invokes another command internally, pass `--internal`. The callee skips all interactive prompts and returns results only. Prevents nested prompt loops.
  150. 5. **`--ci` guard.** `--ci` suppresses all interactive output. Stdout is pure JSON.
  151. 6. **Flow continuity.** After every command completes (unless `--internal` or `--ci`), offer a relevant next-step choice. Never leave the user at a blank prompt.
  152. 7. **Max 3 choices.** Show at most 3 options at once; pick the top 3 by relevance.
  153. 8. **Hooks are lightweight.** Hook scripts collect data and write files. stderr output is minimal — at most one short status line. Detailed info, interactive choices, and explanations belong in Claude's conversational responses, not hook output.
  154. ### First-Run Guidance
  155. When setup completes for the first time (no previous `setup-state.json` existed), replace the old command list with a **smart guidance** based on what was discovered:
  156. ```
  157. Discovery flow:
  158. 1. Show one-line summary: "{N} skills (Code/Dev: {n}, Productivity: {n}, ...)"
  159. 2. Run Quick Scan D1+D2+D3 on all skills
  160. 3. Show context budget one-liner: "Context usage: {X} KB / 80 KB ({pct}%)"
  161. 4. Smart guidance — show ONLY the first matching condition:
  162. Condition Guidance
  163. ───────────────────────────────── ─────────────────────────────────────────────
  164. Has high-risk skill (any D ≤ 4) Surface risky skills + offer [Evaluate & fix / Later]
  165. Context > 60% "Context usage is high" + offer [See what can be cleaned → /skill-inbox all]
  166. Skill count > 8 "Many skills installed" + offer [Browse → /skill-inbox all]
  167. Skill count 3-8, all healthy "All set ✓ You'll be notified via /skill-inbox when suggestions arrive"
  168. Skill count 1-2 "Ready to use" + offer [Check quality → /eval-skill {name}]
  169. ```
  170. Do NOT show a list of all commands. Do NOT show the full skill inventory (that's `/skill-inbox all`'s job).
  171. ## Behavioral Constraints
  172. 1. **Never modify target SKILL.md frontmatter** for version tracking. All version metadata lives in the sidecar `.skill-compass/` directory.
  173. 2. **D3 security gate is absolute.** A single Critical finding forces FAIL verdict, no override.
  174. 3. **Always snapshot before modification.** Before eval-improve writes changes, snapshot the current version.
  175. 4. **Auto-rollback on regression.** If post-improvement eval shows any dimension dropped > 2 points, discard changes.
  176. 5. **Correction tracking is non-intrusive.** Record corrections in `.skill-compass/{name}/corrections.json`, never in the skill file.
  177. 6. **Tiered verification** based on change scope:
  178. - L0: syntax check (always)
  179. - L1: re-evaluate target dimension
  180. - L2: full six-dimension re-evaluation
  181. - L3: cross-skill impact check (for composite/meta)
  182. ## Security Notice
  183. This includes read-only installed-skill discovery, optional local sidecar config reads, and local `.skill-compass/` state writes.
  184. This is a **local evaluation and hardening tool**. Read-only evaluation commands are the default starting point. Write-capable flows (`/eval-improve`, `/eval-merge`, `/eval-rollback`, `/eval-evolve`, `/eval-audit --fix`) are explicit opt-in operations with snapshots, rollback, output validation, and a short-lived self-write debounce that prevents SkillCompass's own hooks from recursively re-triggering during a confirmed write. No network calls are made. See **[SECURITY.md](SECURITY.md)** for the full trust model and safeguards.
README.md · README · 232 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <h1 align="center">SkillCompass</h1>
  2. <p align="center">
  3. <strong>Evaluate quality. Find the weakest link. Fix it. Prove it worked. Repeat.</strong>
  4. </p>
  5. <p align="center">
  6. <a href="https://github.com/Evol-ai/SkillCompass">GitHub</a> &middot;
  7. <a href="SKILL.md">SKILL.md</a> &middot;
  8. <a href="schemas/">Schemas</a> &middot;
  9. <a href="CHANGELOG.md">Changelog</a>
  10. </p>
  11. <p align="center">
  12. <a href="https://linux.do/"><img src="https://img.shields.io/badge/Community-LINUX%20DO-blue.svg" alt="LINUX DO Community" /></a>
  13. <img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT License" />
  14. <img src="https://img.shields.io/badge/node-%3E%3D18-brightgreen.svg" alt="Node >= 18" />
  15. <img src="https://img.shields.io/badge/model-Claude%20Opus%204.6%20%2F%204.7-purple.svg" alt="Claude Opus 4.6 / 4.7" />
  16. </p>
  17. ---
  18. | | |
  19. |--|--|
  20. | **What it is** | A local-first skill quality evaluator and management tool for Claude Code / OpenClaw. Six-dimension scoring, usage-driven suggestions, guided improvement, version tracking. |
  21. | **Pain it solves** | Turns "tweak and hope" into diagnose → targeted fix → verified improvement. Turns "install and forget" into ongoing visibility over what's working, what's stale, and what's risky. |
  22. | **Use in 30 seconds** | `/skillcompass` — see your skill health at a glance. `/eval-skill {path}` — instant quality report showing exactly what's weakest and what to improve next. |
  23. > **Evaluate → find weakest link → fix it → prove it worked → next weakness → repeat.**
  24. > **Meanwhile, Skill Inbox watches your usage and tells you what needs attention.**
  25. ---
  26. ## Who This Is For
  27. <table>
  28. <tr><td width="50%">
  29. **For**
  30. - Anyone maintaining agent skills and wanting measurable quality
  31. - Developers who want directed improvement — not guesswork, but knowing exactly which dimension to fix next
  32. - Teams needing a quality gate — any tool that edits a skill gets auto-evaluated
  33. - Users who install many skills and need visibility over what's actually used, what's stale, and what's risky
  34. </td><td>
  35. **Not For**
  36. - General code review or runtime debugging
  37. - Creating new skills from scratch (use skill-creator)
  38. - Evaluating non-skill files
  39. </td></tr>
  40. </table>
  41. ---
  42. ## Quick Start
  43. > **Prerequisites:** Claude Opus 4.6 / 4.7 (complex reasoning + consistent scoring) &middot; Node.js v18+ (local validators)
  44. ### One-Command Install (recommended)
  45. ```bash
  46. npx skills add Evol-ai/SkillCompass
  47. ```
  48. Supports **45+ agents** including Claude Code, Codex, Cursor, Cline, Gemini CLI, GitHub Copilot, and more. The CLI auto-detects installed agents and sets up the skill in the right location.
  49. ### Claude Code (manual)
  50. ```bash
  51. git clone https://github.com/Evol-ai/SkillCompass.git
  52. cd SkillCompass && npm install
  53. # User-level (all projects)
  54. rsync -a --exclude='.git' . ~/.claude/skills/skill-compass/
  55. # Or project-level (current project only)
  56. rsync -a --exclude='.git' . .claude/skills/skill-compass/
  57. ```
  58. > **First run:** SkillCompass auto-triggers a brief onboarding — scans your installed skills (~5 seconds), offers statusLine setup, then hands control back. Claude Code will request permission for `node` commands; select **"Allow always"** to avoid repeated prompts.
  59. ### OpenClaw
  60. ```bash
  61. git clone https://github.com/Evol-ai/SkillCompass.git
  62. cd SkillCompass && npm install
  63. # Follow OpenClaw skill installation docs for your setup
  64. rsync -a --exclude='.git' . <your-openclaw-skills-path>/skill-compass/
  65. ```
  66. If your OpenClaw skills live outside the default scan roots, add them to `skills.load.extraDirs` in `~/.openclaw/openclaw.json`:
  67. ```json
  68. {
  69. "skills": {
  70. "load": {
  71. "extraDirs": ["<your-openclaw-skills-path>"]
  72. }
  73. }
  74. }
  75. ```
  76. ---
  77. ## Usage
  78. `/skillcompass` is the single entry point. Use it with a slash command or just talk naturally — both work:
  79. ```
  80. /skillcompass → see what needs attention
  81. /skillcompass evaluate my-skill → six-dimension quality report
  82. "improve the nano-banana skill" → fix weakest dimension, verify, next
  83. "what skills haven't I used recently?" → usage-based insights
  84. "security scan this skill" → D3 security deep-dive
  85. ```
  86. ---
  87. ## What It Does
  88. <p align="center">
  89. <img src="assets/skill-quality-report.png" alt="SkillCompass — Skill Quality Report" width="380" />
  90. </p>
  91. The score isn't the point — **the direction is.** You instantly see which dimension is the bottleneck and what to do about it.
  92. Each `/eval-improve` round follows a closed loop: **fix the weakest → re-evaluate → verify improvement → next weakest**. No fix is saved unless the re-evaluation confirms it actually helped.
  93. ---
  94. ## Six-Dimension Evaluation Model
  95. | ID | Dimension | Weight | What it evaluates |
  96. |:--:|-----------|:------:|-------------------|
  97. | **D1** | Structure | 10% | Frontmatter validity, markdown format, declarations |
  98. | **D2** | Trigger | 15% | Activation quality, rejection accuracy, discoverability |
  99. | **D3** | Security | 20% | Secrets, injection, permissions, exfiltration, embedded shell |
  100. | **D4** | Functional | 30% | Core quality, edge cases, output stability, error handling |
  101. | **D5** | Comparative | 15% | Value over direct prompting (with vs without skill) |
  102. | **D6** | Uniqueness | 10% | Overlap with similar skills, model supersession risk |
  103. ```
  104. overall_score = round((D1×0.10 + D2×0.15 + D3×0.20 + D4×0.30 + D5×0.15 + D6×0.10) × 10)
  105. ```
  106. | Verdict | Condition |
  107. |---------|-----------|
  108. | **PASS** | score >= 70 AND D3 pass |
  109. | **CAUTION** | 50–69, or D3 High findings |
  110. | **FAIL** | score < 50, or D3 Critical (gate override) |
  111. ---
  112. ## Skill Inbox — Usage-Driven Suggestions
  113. SkillCompass passively tracks which skills you actually use and surfaces suggestions when something needs attention — unused skills, stale evaluations, declining usage, available updates, and more. 9 built-in rules, all based on real invocation data.
  114. - Suggestions have a lifecycle: **pending → acted / snoozed / dismissed**, with auto-reactivation when conditions change
  115. - All data stays local — no network calls unless you explicitly request updates
  116. - Tracking is automatic via hooks (~one line per skill invocation), zero configuration
  117. ---
  118. ## Features
  119. ### Evaluate → Improve → Verify
  120. `/eval-skill` scores six dimensions and pinpoints the weakest. `/eval-improve` targets that dimension, applies a fix, and re-evaluates — only saves when the target dimension improved and security/functionality didn't regress. Then move to the next weakness.
  121. ### Skill Lifecycle
  122. SkillCompass covers the full lifecycle of your skills — not just one-time evaluation.
  123. **Install** — auto-scans your inventory, quick-checks security patterns across packages and sub-skills.
  124. **Ongoing** — usage hooks passively track every invocation. Skill Inbox turns this into actionable insights: which skills are never used, which are declining, which are heavily used but never evaluated, which have updates available.
  125. **On edit** — hooks auto-check structure + security on every SKILL.md write through Claude. Catches injection, exfiltration, embedded shell. Warns, never blocks.
  126. **On change** — SHA-256 snapshots ensure any version is recoverable. D3 or D4 regresses after improvement? Snapshot restored automatically.
  127. **On update** — update checker reads local git state passively; network only when you ask. Three-way merge preserves your local improvements region-by-region.
  128. ### Scale
  129. One skill or fifty — same workflow. `/eval-audit` scans a whole directory and ranks results worst-first so you fix what matters most. `/eval-evolve` chains multiple improve rounds automatically (default 6, stops at PASS or plateau). `--ci` flag outputs machine-readable JSON with exit codes for pipeline integration.
  130. ---
  131. ## Works With Everything
  132. No point-to-point integration needed. The Pre-Accept Gate intercepts all SKILL.md edits regardless of source.
  133. | Tool | How it works together | Guide |
  134. |------|----------------------|-------|
  135. | **Claudeception** | Extracts skill → auto-evaluation catches security holes + redundancy → directed fix | [guide](examples/guide-claudeception.md) |
  136. | **Self-Improving Agent** | Logs errors → feed as signals → SkillCompass maps to dimensions and fixes | [guide](examples/guide-self-improving-agent.md) |
  137. ---
  138. ## Design Principles
  139. - **Local-first**: All data stays on your machine. No network calls except when you explicitly request updates.
  140. - **Read-only by default**: Evaluation and reporting are read-only. Write operations (improve, merge, rollback) require explicit opt-in.
  141. - **Passive tracking, active decisions**: Hooks collect usage data silently. Suggestions are surfaced, never auto-acted on.
  142. - **Dual-channel UX**: Keyboard-selectable choices for actions, natural language for queries. Both always available.
  143. ---
  144. ## Feedback Signal Standard
  145. SkillCompass defines an open `feedback-signal.json` schema for any tool to report skill usage data:
  146. ```bash
  147. /eval-skill ./my-skill/SKILL.md --feedback ./feedback-signals.json
  148. ```
  149. Signals: `trigger_accuracy`, `correction_count`, `correction_patterns`, `adoption_rate`, `ignore_rate`, `usage_frequency`. The schema is extensible (`additionalProperties: true`) — any pipeline can produce or consume this format.
  150. ---
  151. ## Community
  152. This open-source project is affiliated with and endorsed by the **[LINUX DO](https://linux.do/)** community.
  153. ---
  154. ## License
  155. **MIT** — Use, modify, distribute freely. See [LICENSE](LICENSE) for details.
scripts/eval-v2/codespace-setup.sh · Bundled script · 87 lines · 1 finding marked
  1. #!/usr/bin/env bash
  2. # ============================================================
  3. # SkillCompass v2 Evaluation — Codespace Setup
  4. # Run this ONCE after opening the Codespace.
  5. # ============================================================
  6. set -euo pipefail
  7. echo "═══════════════════════════════════════════════════════"
  8. echo " SkillCompass v2 — Codespace Environment Setup"
  9. echo "═══════════════════════════════════════════════════════"
  10. # 1. Check Node.js
  11. echo ""
  12. echo "[1/5] Checking Node.js..."
  13. if command -v node &>/dev/null; then
  14. echo " ✓ Node.js $(node --version)"
  15. else
  16. echo " ✗ Node.js not found. Installing..."
  17. curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 19 on GitHub
  18. sudo apt-get install -y nodejs
  19. fi
  20. # 2. Install clawhub CLI
  21. echo ""
  22. echo "[2/5] Installing clawhub CLI..."
  23. if command -v clawhub &>/dev/null; then
  24. echo " ✓ clawhub $(clawhub --cli-version 2>/dev/null || echo 'installed')"
  25. else
  26. npm install -g clawhub
  27. echo " ✓ clawhub installed"
  28. fi
  29. # 3. Install Claude Code
  30. echo ""
  31. echo "[3/5] Checking Claude Code..."
  32. if command -v claude &>/dev/null; then
  33. echo " ✓ Claude Code installed"
  34. else
  35. echo " Installing Claude Code..."
  36. npm install -g @anthropic-ai/claude-code
  37. echo " ✓ Claude Code installed"
  38. echo ""
  39. echo " ⚠ You need to authenticate. Run:"
  40. echo " claude login"
  41. echo " Or set ANTHROPIC_API_KEY environment variable."
  42. fi
  43. # 4. Clone SkillCompass (if not already present)
  44. echo ""
  45. echo "[4/5] Setting up SkillCompass..."
  46. EVAL_DIR="$HOME/eval-workspace"
  47. mkdir -p "$EVAL_DIR"
  48. if [ ! -d "$EVAL_DIR/skill-compass" ]; then
  49. echo " Cloning SkillCompass..."
  50. git clone https://github.com/Evol-ai/SkillCompass.git "$EVAL_DIR/skill-compass"
  51. else
  52. echo " ✓ SkillCompass already present"
  53. cd "$EVAL_DIR/skill-compass" && git pull --ff-only 2>/dev/null || true
  54. fi
  55. # 5. Create evaluation workspace
  56. echo ""
  57. echo "[5/5] Creating evaluation workspace..."
  58. SKILLS_DIR="$EVAL_DIR/clawhub-skills"
  59. RESULTS_DIR="$EVAL_DIR/results"
  60. mkdir -p "$SKILLS_DIR" "$RESULTS_DIR"
  61. # Copy sample list and scripts
  62. cp -f "$EVAL_DIR/skill-compass/scripts/download-skills.sh" "$EVAL_DIR/" 2>/dev/null || true
  63. cp -f "$EVAL_DIR/skill-compass/scripts/batch-eval.sh" "$EVAL_DIR/" 2>/dev/null || true
  64. cp -f "$EVAL_DIR/skill-compass/scripts/analyze-results.js" "$EVAL_DIR/" 2>/dev/null || true
  65. echo ""
  66. echo "═══════════════════════════════════════════════════════"
  67. echo " Setup complete!"
  68. echo ""
  69. echo " Workspace: $EVAL_DIR"
  70. echo " Skills dir: $SKILLS_DIR"
  71. echo " Results dir: $RESULTS_DIR"
  72. echo ""
  73. echo " Next steps:"
  74. echo " 1. cd $EVAL_DIR"
  75. echo " 2. bash download-skills.sh # Download 45 skills"
  76. echo " 3. bash batch-eval.sh # Run evaluations"
  77. echo " 4. node analyze-results.js # Statistical analysis"
  78. echo "═══════════════════════════════════════════════════════"

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.