Skip to content
Back to search

compose-skill

Compose Skill (Compose Kit): agent skills for Jetpack Compose & Compose Multiplatform. Makes Claude Code, Codex, Cursor, Copilot, Gemini CLI and OpenCode write better Compose: MVI, Koin, Navigation 3, Ktor, Room. Tested blind on 5 models in 120 agentic runs.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/meet-miyani--compose-skill/badge.svg)](/listings/meet-miyani--compose-skill)

HTML

<a href="/listings/meet-miyani--compose-skill"><img src="/api/listings/meet-miyani--compose-skill/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Install all seven skills. `compose` is the entry point, and it hands each task to the other six. Step-by-step pages for each agent are on the website: [Claude Code](https://compose.avinya.dev/install/claude-code/), [Codex](https://compose.avinya.dev/install/codex/), [Cursor](https://compose.avinya.dev/install/cursor/), [GitHub Copilot](https://compose.avinya.dev/install/github-copilot/), [Gemini CLI and Antigravity](https://compose.avinya.dev/install/gemini-cli-and-antigravity/), [OpenCode](https://compose.avinya.dev/install/opencode/).

Every option below can install for **one project** (lives in the repo, so your team gets it too) or **globally** (for you, in every project on your machine).

npx skills

Needs Node 22.20+. Works with Claude Code, Codex, Cursor, OpenCode, Copilot, Gemini CLI, Antigravity and 70+ other agents.

# This project only (run it in the repo root)
npx skills add Meet-Miyani/compose-skill --skill '*' -a claude-code

# Globally, for every project
npx skills add Meet-Miyani/compose-skill --skill '*' -a claude-code -g

Swap `claude-code` for your agent: `codex`, `cursor`, `opencode`, `github-copilot`, `gemini-cli`, `antigravity`. Repeat `-a` to install for several at once. Leave it out and you get an interactive picker.

| Agent | This project | Global | |---|---|---| | Claude Code | `.claude/skills/` | `~/.claude/skills/` | | Codex, Cursor, OpenCode, Copilot, Gemini CLI, Antigravity | `.agents/skills/` | the agent's own folder, e.g. `~/.codex/skills/` |

gh skill

Needs GitHub CLI 2.90+. `--pin v6.1.0` fixes the version; without it, `gh skill` takes the latest stable release, which is currently v6.1.0. Pass `--agent` too: without it, a non-interactive run installs for GitHub Copilot.

for s in compose compose-architecture compose-feature compose-ui compose-data compose-project compose-platform; do
  gh skill install Meet-Miyani/compose-skill "$s" --pin v6.1.0 --agent claude-code --scope project
done

Use `--scope user` to install globally. Agent IDs: `claude-code`, `codex`, `cursor`, `opencode`, `gemini-cli`, `antigravity`, `github-copilot`.

Claude Code plugin

/plugin marketplace add Meet-Miyani/compose-skill
/plugin install compose-kit@compose-kit

From a terminal, the same with a scope: `user` (the default, every project), `project` (shared through the repo) or `local` (this project, just you).

claude plugin marketplace add Meet-Miyani/compose-skill
claude plugin install compose-kit@compose-kit --scope project

Codex plugin

codex plugin marketplace add Meet-Miyani/compose-skill
codex plugin add compose-kit@compose-kit

Codex plugins install for your user, so they're available in every project.

Then point your agent at it

Add this line to the project's `AGENTS.md` or `CLAUDE.md`:

Compose/CMP work: load the compose skill first; it picks the path and the files to read.

The skills also trigger on their own; the line just makes sure the entry tree loads first.

**Coming from the old single `compose` skill (v5.x or the `composekit` CLI)?** Delete the old `compose` folder from your agent's skills directory, then install with any option above. The old CLI's `update` can't fetch v6.

**Optional:** install the guard scripts into a project to catch drift in CI:

bash <skills dir>/compose-architecture/scripts/install-guards.sh <project>

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 14 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "compose-kit",
  3. "description": "Compose skill for AI coding agents: Jetpack Compose and Compose Multiplatform agent skills",
  4. "owner": {
  5. "name": "Meet Miyani"
  6. },
  7. "plugins": [
  8. {
  9. "name": "compose-kit",
  10. "source": "./",
  11. "description": "Compose skill for AI coding agents: seven skills for Jetpack Compose and Compose Multiplatform (MVI, Koin, Navigation 3, Ktor, Room)"
  12. }
  13. ]
  14. }
README.md · README · 404 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. # Compose Skill (Compose Kit)
  3. **A Compose skill for AI coding agents: seven agent skills that make Claude Code, Codex, Cursor, Copilot and Gemini
  4. write better Jetpack Compose and Compose Multiplatform code.**<br>
  5. One consistent house style, measured on seven models in real agent CLIs.
  6. [![Release](https://img.shields.io/github/v/release/Meet-Miyani/compose-skill?label=release&color=2a78d6)](https://github.com/Meet-Miyani/compose-skill/releases)
  7. [![CI](https://github.com/Meet-Miyani/compose-skill/actions/workflows/ci.yml/badge.svg)](https://github.com/Meet-Miyani/compose-skill/actions/workflows/ci.yml)
  8. [![License](https://img.shields.io/github/license/Meet-Miyani/compose-skill?color=52514e)](LICENSE)
  9. [![Website](https://img.shields.io/badge/website-compose.avinya.dev-ee3a20)](https://compose.avinya.dev)
  10. [![Skills](https://img.shields.io/badge/skills-7-2a78d6)](#whats-inside)
  11. [![v6 test](https://img.shields.io/badge/v6%20test-90%20%2B%2036%20agentic%20runs-2a78d6)](#results-v61)
  12. [![Guard tests](https://img.shields.io/badge/guard%20tests-90%20passing-0ca30c)](tests/skills)
  13. [![v6.1 A/B](https://img.shields.io/badge/v6.1%20A%2FB-stable-0ca30c)](#results-v61)
  14. **Works with** Claude Code · Codex · Cursor · OpenCode · Copilot · Gemini CLI · Antigravity · Android Studio
  15. **Website:** [compose.avinya.dev](https://compose.avinya.dev) · [Install guides per agent](https://compose.avinya.dev/install/) · [Results](https://compose.avinya.dev/results/)
  16. </div>
  17. <table>
  18. <tr>
  19. <td align="center" width="25%"><h2>+30.8</h2>points for<br><b>Gemini 3.8 Flash</b><br>with the v6.1 kit (v6 test)</td>
  20. <td align="center" width="25%"><h2>+23.1</h2>points for<br><b>GPT-6-Luna</b><br>with the v6.1 kit (v6 test)</td>
  21. <td align="center" width="25%"><h2>97.4%</h2>of rubric items for<br><b>DeepSeek V4.1 Flash</b><br>with v6.1 (cheap add-on model)</td>
  22. <td align="center" width="25%"><h2>90 + 36</h2>agentic runs (panel + add-ons);<br>2 blind graders must agree on every item</td>
  23. </tr>
  24. </table>
  25. <p align="center">
  26. <picture>
  27. <source media="(prefers-color-scheme: dark)" srcset="docs/assets/v6-models-dark.svg">
  28. <img alt="v6 test: rubric score without the kit, with the v6.0 kit and with the v6.1 kit, as percent of 39 items, for seven models, sorted by lift. Gemini 3.8 Flash 56.4%, 84.6%, 87.2% (+30.8). GPT-6-Luna 71.8%, 89.7%, 94.9% (+23.1). DeepSeek V4.1 Flash 82.1%, 89.7%, 97.4% (+15.4). DeepSeek V4 Pro 74.4%, 76.9%, 87.2% (+12.8). GPT-6-Sol 76.9%, 87.2%, 87.2% (+10.3). Sonnet 5.5 79.5%, 89.7%, 87.2% (+7.7). Muse Spark 1.3 69.2%, 69.2%, 74.4% (+5.1). The two DeepSeek models are add-ons graded by Claude and GPT, so compare across groups with care." src="docs/assets/v6-models-light.svg" width="760">
  29. </picture>
  30. </p>
  31. ## What we achieved
  32. - **Held-out v5 (120 agentic runs):** the kit lifted Gemini 3.8 Flash by +25.6 points and Sonnet 5.5 by +18.6, and
  33. the test found a real flaw: on a "match our conventions" task, models restructured too much with the kit
  34. ([results](#results-held-out-v5)).
  35. - **v6.0.0-preview.1 (2026-10-01):** seven skills, one entry tree, four install channels.
  36. - **Website live (2026-10-02):** [compose.avinya.dev](https://compose.avinya.dev), with install guides per agent and a
  37. build check that every number on it appears in the evidence files.
  38. - **v6 A/B (90 agentic runs):** v6.0 vs the v6.1 fixes vs no kit. The "no new UI controls, screens or features" item
  39. on conform tasks went from 2 of 10 cells (v6.0) to 8 of 10 (v6.1), and every pre-registered rule held
  40. ([results](#results-v61)).
  41. - **v6.1.0 stable (2026-10-05):** no model scored below its own no-kit result, so it is released as stable. v6.1.1
  42. (2026-10-11) fixes a guard-script security issue. v6.2.0 (2026-10-23) adds
  43. adaptive-layout guidance (screens follow the window size class, never the device) and more guard fixes. v6.3.0
  44. (2026-11-08), the GitHub "Latest" release, asks agents to run a visible change and look at it before calling it done.
  45. - **Cheap models (36 add-on runs):** DeepSeek V4.1 Flash went from 82.1% to 97.4% of items with v6.1, and DeepSeek
  46. V4 Pro from 74.4% to 87.2% ([results](#add-ons-cheap-deepseek-models)).
  47. ## Why
  48. AI agents write Compose that compiles and then drifts: state ends up in the wrong place, every screen gets its own
  49. pattern, error handling changes from file to file, and reviews flag perfectly fine code as blocking. Compose Kit gives
  50. the agent one house style plus the reasoning behind it, and only loads the parts the current task needs.
  51. - **One consistent style:** MVI on a shared `BaseViewModel` contract, Koin annotations, Navigation 3,
  52. feature-owned data/domain/presentation layers, and typed error handling.
  53. - **Progressive loading:** a ~1.3k-token entry tree picks the task path and the exact reference files. The full kit
  54. is about 127k tokens; in the v5 test, 35 of 40 kit runs loaded under 20k.
  55. - **Respects your project:** in a coherent existing codebase (Hilt, MVVM, Navigation 2, your own base class), the
  56. kit follows your pattern and proposes migration separately.
  57. - **Checked, not trusted:** 11 shell guard scripts catch drift in CI, backed by 90 fixture tests.
  58. ## What's inside
  59. | Skill | What it owns |
  60. |---|---|
  61. | **`compose`** | The entry decision tree (~1.3k tokens): task kind → affected area → exact reference files |
  62. | `compose-architecture` | Module graph, MVI/`BaseViewModel` contract, error tiers, state ownership, naming, Koin DI, Navigation 3, coroutines |
  63. | `compose-feature` | A screen or feature slice end to end: contract, ViewModel, UI, navigation, DI and tests; run and see; review mode |
  64. | `compose-ui` | Composables: Route/Screen split, stability, loading/empty/error states, lists, animation, accessibility, resources, adaptive layouts |
  65. | `compose-data` | Repositories, Ktor, Room, DataStore, Paging 3, offline-first, data-layer tests |
  66. | `compose-project` | New projects, adopting the kit, modules, convention plugins, version catalog, CI and agent hooks |
  67. | `compose-platform` | `commonMain` vs `expect`/`actual`, iOS/Swift interop, desktop and web targets, platform lifecycle |
  68. Also included: project and feature templates, and the guard scripts (`compose-architecture/scripts/`).
  69. ## How it works
  70. ```mermaid
  71. flowchart TD
  72. P["Your prompt"] --> E["compose entry tree · ~1.3k tokens"]
  73. E --> K{"What kind of task?<br/>new feature · change · bug fix · review<br/>conform · setup · question"}
  74. K --> A{"Which areas does it touch?<br/>architecture · feature · ui<br/>data · project · platform"}
  75. A --> F["Load one reference file per affected area"]
  76. F --> C["Code in the house style"]
  77. ```
  78. ## Install
  79. Install all seven skills. `compose` is the entry point, and it hands each task to the other six. Step-by-step pages
  80. for each agent are on the website: [Claude Code](https://compose.avinya.dev/install/claude-code/),
  81. [Codex](https://compose.avinya.dev/install/codex/), [Cursor](https://compose.avinya.dev/install/cursor/),
  82. [GitHub Copilot](https://compose.avinya.dev/install/github-copilot/),
  83. [Gemini CLI and Antigravity](https://compose.avinya.dev/install/gemini-cli-and-antigravity/),
  84. [OpenCode](https://compose.avinya.dev/install/opencode/).
  85. Every option below can install for **one project** (lives in the repo, so your team gets it too) or **globally** (for
  86. you, in every project on your machine).
  87. ### npx skills
  88. Needs Node 22.20+. Works with Claude Code, Codex, Cursor, OpenCode, Copilot, Gemini CLI, Antigravity and 70+ other agents.
  89. ```sh
  90. # This project only (run it in the repo root)
  91. npx skills add Meet-Miyani/compose-skill --skill '*' -a claude-code
  92. # Globally, for every project
  93. npx skills add Meet-Miyani/compose-skill --skill '*' -a claude-code -g
  94. ```
  95. Swap `claude-code` for your agent: `codex`, `cursor`, `opencode`, `github-copilot`, `gemini-cli`, `antigravity`. Repeat
  96. `-a` to install for several at once. Leave it out and you get an interactive picker.
  97. | Agent | This project | Global |
  98. |---|---|---|
  99. | Claude Code | `.claude/skills/` | `~/.claude/skills/` |
  100. | Codex, Cursor, OpenCode, Copilot, Gemini CLI, Antigravity | `.agents/skills/` | the agent's own folder, e.g. `~/.codex/skills/` |
  101. Android Studio (Quail 1 or newer) reads [`.agents/skills/`](https://developer.android.com/studio/gemini/skills) at the
  102. project root or in your home directory, so the project install for any agent in the second row covers it.
  103. Update later with `npx skills update`.
  104. ### gh skill
  105. Needs GitHub CLI 2.90+. It installs the latest stable release. Pass `--agent` too: without it, a non-interactive run
  106. installs for GitHub Copilot.
  107. ```sh
  108. for s in compose compose-architecture compose-feature compose-ui compose-data compose-project compose-platform; do
  109. gh skill install Meet-Miyani/compose-skill "$s" --agent claude-code --scope project
  110. done
  111. ```
  112. Update later with `gh skill update --all`. Installed with `--pin` before? Run `gh skill update --unpin --all` once:
  113. `gh skill update` skips pinned skills.
  114. Use `--scope user` to install globally. Agent IDs: `claude-code`, `codex`, `cursor`, `opencode`, `gemini-cli`,
  115. `antigravity`, `github-copilot`.
  116. ### Claude Code plugin
  117. ```text
  118. /plugin marketplace add Meet-Miyani/compose-skill
  119. /plugin install compose-kit@compose-kit
  120. ```
  121. From a terminal, the same with a scope: `user` (the default, every project), `project` (shared through the repo) or
  122. `local` (this project, just you).
  123. ```sh
  124. claude plugin marketplace add Meet-Miyani/compose-skill
  125. claude plugin install compose-kit@compose-kit --scope project
  126. ```
  127. Update later from a terminal: `claude plugin marketplace update compose-kit`, then
  128. `claude plugin update compose-kit@compose-kit`, then restart Claude Code.
  129. ### Codex plugin
  130. ```sh
  131. codex plugin marketplace add Meet-Miyani/compose-skill
  132. codex plugin add compose-kit@compose-kit
  133. ```
  134. Codex plugins install for your user, so they're available in every project.
  135. Update later with `codex plugin marketplace upgrade`: it refreshes the marketplace and the installed plugin.
  136. ### Then point your agent at it
  137. Add this line to the project's `AGENTS.md` or `CLAUDE.md`:
  138. ```text
  139. Compose/CMP work: load the compose skill first; it picks the path and the files to read.
  140. ```
  141. The skills also trigger on their own; the line just makes sure the entry tree loads first.
  142. **Coming from the old single `compose` skill (v5.x or the `composekit` CLI)?** Delete the old `compose` folder from
  143. your agent's skills directory, then install with any option above. The old CLI's `update` can't fetch v6.
  144. **Optional:** install the guard scripts into a project to catch drift in CI:
  145. ```sh
  146. bash <skills dir>/compose-architecture/scripts/install-guards.sh <project>
  147. ```
  148. ## Results: v6.1
  149. The v6.1 test: 5 models × 6 new tasks × 3 arms (no kit, v6.0 kit, v6.1 kit) = 90 agentic runs in the real CLIs. Each
  150. answer was graded blind by the two vendors other than its own, and an item passes only when both graders pass it.
  151. Items passed of 39 (27 `[eng]` items written by the independent task author, 12 `[kit]` items written by the
  152. moderator):
  153. | Model | No kit | v6.0 kit | v6.1 kit | v6.1 − no kit |
  154. |---|---:|---:|---:|---:|
  155. | Sonnet 5.5 | 31 (79.5%) | 35 (89.7%) | 34 (87.2%) | +3 (+7.7 pts) |
  156. | GPT-6-Luna | 28 (71.8%) | 35 (89.7%) | **37 (94.9%)** | **+9 (+23.1 pts)** |
  157. | GPT-6-Sol | 30 (76.9%) | 34 (87.2%) | 34 (87.2%) | +4 (+10.3 pts) |
  158. | Gemini 3.8 Flash | 22 (56.4%) | 33 (84.6%) | 34 (87.2%) | **+12 (+30.8 pts)** |
  159. | Muse Spark 1.3 | 27 (69.2%) | 27 (69.2%) | 29 (74.4%) | +2 (+5.1 pts) |
  160. By task type, all five models together (both graders agree):
  161. | Tasks | No kit | v6.0 kit | v6.1 kit |
  162. |---|---:|---:|---:|
  163. | Conform (2 tasks, 60 items) | 73.3% | 75.0% | **80.0%** |
  164. | Review (2 tasks, 75 items) | 78.7% | 94.7% | 94.7% |
  165. | New feature (2 tasks, 60 items) | 58.3% | 80.0% | 81.7% |
  166. <p align="center">
  167. <picture>
  168. <source media="(prefers-color-scheme: dark)" srcset="docs/assets/v6-task-types-dark.svg">
  169. <img alt="v6 test by task type, five panel models together, percent of rubric items passed without the kit, with v6.0 and with v6.1. Conform 73.3%, 75.0%, 80.0%. Review 78.7%, 94.7%, 94.7%. New feature 58.3%, 80.0%, 81.7%." src="docs/assets/v6-task-types-light.svg" width="760">
  170. </picture>
  171. </p>
  172. All pre-registered rules held, and no model scored below its own no-kit result, so v6.1.0 is the stable release.
  173. - **Conform over-scoping is mostly fixed.** The "no new UI controls, screens or features" item passed in 2 of 10
  174. conform cells with v6.0 and 8 of 10 with v6.1 (no kit: 6). v6.0 added retry buttons and error screens while
  175. conforming; v6.1 mostly stopped.
  176. - **Reviews: no difference between the kits.** Both lift reviews well above no kit, and neither called a "fine" item
  177. blocking. The v5 over-flagging did not show up on these tasks, so that is no difference, not a fix.
  178. - **Muse Spark 1.3 barely benefits** (+2 over no kit) and still over-scoped T1: v6.1 added a Retry button, and it
  179. scored 6/6 on T1 without the kit and 3/6 with v6.1.
  180. - **Context:** 5 of 10 v6.1 new-feature runs went over the 20k design budget (v6.0: 3 of 10), up to about 30k for
  181. GPT-6-Sol. Sonnet and Gemini 3.8 Flash stayed under 20k on every new-feature run with v6.1.
  182. - **Grading flaw, disclosed.** The hidden behaviour test passed 3/3 in 29 of 30 conform cells, but the grader packets
  183. did not name it or show its result, so both graders passed that item in only 11 of 30 cells, in every arm alike. A
  184. sensitivity check that sets the item to the mechanical result leaves every rule and the stable criterion unchanged.
  185. - **Limits:** one generation per cell and 6 tasks, so per-model differences of about 3 items or fewer are noise. Read
  186. v6.1 − v6.0 as "no regression" (Sonnet is −1), not as a ranking.
  187. Details: [VERDICT.md](evals-v2/results-v6/VERDICT.md) · [score output](evals-v2/results-v6/score-output.txt) ·
  188. [kit tokens](evals-v2/results-v6/kit-tokens.txt)
  189. ### Add-ons: cheap DeepSeek models
  190. The same 6 tasks and 3 arms on two cheap models through OpenCode (36 runs, graded blind by Claude and GPT, both must
  191. pass). Reported separately; they did not decide the release.
  192. | Model | No kit | v6.0 kit | v6.1 kit | v6.1 − no kit | Plan cost, 18 runs |
  193. |---|---:|---:|---:|---:|---:|
  194. | DeepSeek V4.1 Flash | 32 (82.1%) | 35 (89.7%) | **38 (97.4%)** | +6 (+15.4 pts) | $0.65 |
  195. | DeepSeek V4 Pro | 29 (74.4%) | 30 (76.9%) | **34 (87.2%)** | +5 (+12.8 pts) | $2.12 |
  196. - Both score highest with v6.1, and neither is below its no-kit result with either kit.
  197. - The biggest single gain: DeepSeek V4 Pro on the "books by author" feature, 1/6 without the kit, 6/6 with v6.1.
  198. - No over-scoping on conform tasks and no over-flagged reviews in any arm. v6.1 did not help Pro's yearly-goal
  199. feature (5/6 without the kit, 4/6 with it), which is within noise.
  200. - The add-ons had Claude and GPT as graders, while the panel's Claude and GPT answers had Gemini as one grader, so
  201. compare across the two tables with care.
  202. Details: [ADDONS.md](evals-v2/results-v6/ADDONS.md) (run record: two provider content-filter blocks, one disk-full
  203. discard, five hidden-test adaptations)
  204. ## Results: held-out v5
  205. This is the earlier test, the one v6.0.0-preview.1 shipped on. Five models got 8 tasks they had never seen, in a fresh app, inside
  206. their own agent CLIs. Each task ran three times: with no kit, with a short generic "senior engineer" prompt, and with
  207. the kit. That's 120 runs. Two models from the other vendors graded every answer blind, and an item only counts if
  208. both of them pass it. The rules were written down before the tasks existed, and the grades were frozen before
  209. anything was scored.
  210. | Model | No kit | Generic prompt | **With the kit** | Kit − no kit | Verdict |
  211. |---|---:|---:|---:|---:|---|
  212. | Gemini 3.8 Flash | 51.2% | 46.5% | **76.7%** | **+25.6** | ✅ Recommended |
  213. | Sonnet 5.5 | 55.8% | 60.5% | **74.4%** | **+18.6** | ✅ Recommended |
  214. | Opus 5.5 | 65.1% | 62.8% | **74.4%** | **+9.3** | ✅ Recommended (see erratum) |
  215. | GPT-6-Sol | 72.1% | 74.4% | **76.7%** | +4.6 | ➖ Neutral: within noise |
  216. | GPT-6-Luna | 65.1% | 60.5% | 67.4% | +2.3 | ❌ Not yet: engineering items fell (−11.1) |
  217. | *Muse Spark 1.3 (add-on)* | *58.1%* | *62.8%* | *60.5%* | *+2.4* | *No lift* |
  218. - **House-style items** rose for every model, from 37-50% without the kit to 56-69% with it.
  219. - **On engineering-only items,** Sonnet (+11.1) and Flash (+40.8) beat the generic prompt by 8+ points; for Opus,
  220. Sol and Luna a short generic prompt does about as well.
  221. <p align="center">
  222. <picture>
  223. <source media="(prefers-color-scheme: dark)" srcset="docs/assets/v5-kit-lift-dark.svg">
  224. <img alt="Held-out v5: rubric score with and without the kit. Gemini 3.8 Flash 51.2% to 76.7% (+25.6), Sonnet 5.5 55.8% to 74.4% (+18.6), Opus 5.5 65.1% to 74.4% (+9.3), GPT-6-Sol 72.1% to 76.7% (+4.6), GPT-6-Luna 65.1% to 67.4% (+2.3)." src="docs/assets/v5-kit-lift-light.svg" width="760">
  225. </picture>
  226. </p>
  227. By task type, all five panel models together (both graders agree):
  228. | Task type | No kit | Generic prompt | With the kit | |
  229. |---|---:|---:|---:|---|
  230. | New features (3 tasks) | 43.5% | 47.1% | **70.6%** | The biggest win: where data lives, navigation results, background work |
  231. | Bug fixes (2) | 80.0% | 88.6% | 82.9% | Every setup fixed the bugs; the kit adds little here |
  232. | Reviews (2) | 72.9% | 64.3% | **84.3%** | These two tasks had flawed setups, see the erratum below |
  233. | "Match our conventions" (1) | 68.0% | 60.0% | 44.0% | Worse with the kit: models restructured too much. v6.1 mostly fixed this |
  234. Computed from the frozen grades by [`v5-by-task-type.py`](evals-v2/method/tools/v5-by-task-type.py).
  235. > **Erratum (2026-10-01).** Two review tasks had flawed setups. No pre-registered outcome changes, but Opus's +9.3
  236. > depends on those two tasks (+3.4 without them). The Claude model was Sonnet 5.5, first published as "Sonnet 5".
  237. > Details: [VERDICT.md](evals-v2/results-v5/VERDICT.md#erratum-2026-10-01).
  238. <p align="center">
  239. <picture>
  240. <source media="(prefers-color-scheme: dark)" srcset="docs/assets/v5-kit-context-dark.svg">
  241. <img alt="Most kit tokens any single task loaded: Opus 5.5 9.9k, Sonnet 5.5 14.0k, GPT-6-Luna 18.3k, GPT-6-Sol 26.4k, Gemini 3.8 Flash 26.5k, against a 20k design budget." src="docs/assets/v5-kit-context-light.svg" width="760">
  242. </picture>
  243. </p>
  244. Every panel model opened `compose` first on every task. Kit context is an upper bound (the full size of every kit
  245. file opened). Details: [kit-tokens.txt](evals-v2/results-v5/kit-tokens.txt).
  246. ## How we test
  247. - **New tasks every round.** A separate model writes them in a new app domain without ever seeing the kit, and they're
  248. sealed before the first run.
  249. - **A gate, not a promise.** A script builds every task's setup and runs its hidden behaviour tests. Tasks that fail
  250. the gate or a manual content check go back to the author or get repaired, and every repair is written down.
  251. - **Real agents on a real app.** Claude Code, Codex, Antigravity and OpenCode, each driving its own model headless on
  252. a Kotlin Multiplatform project.
  253. - **Blind, cross-vendor grading.** Answers are shuffled and stripped of anything that gives away the model or the
  254. setup. Two graders from the other vendors score each one.
  255. - **Pre-registered.** Pass/fail rules come before the tasks, grades are frozen before scoring, and mistakes get a
  256. public erratum.
  257. Method and tools: [evals-v2/method](evals-v2/method/) · Results: [evals-v2/results-v5](evals-v2/results-v5/) ·
  258. Tasks and base app: [evals-v2/heldout-v5](evals-v2/heldout-v5/)
  259. ### Testing so far
  260. | Round | Domain | Format | What it showed |
  261. |---|---|---|---|
  262. | v1-v2 | Habits, expenses | Single-shot scenarios | The early kit made models lecture and over-escalate reviews, and engineering on new tasks fell (83% → 72%). Fixed. |
  263. | v3 | Workout log | 8 single-shot scenarios, 2 graders | Muse Spark 1.3: 54% → 91% with the kit; DeepSeek V4.1 Flash: 57% → 77% *(earlier candidate)* |
  264. | v4 | Plant care | 12 scenarios | Fed the redesign into 7 skills with an entry tree (results not published) |
  265. | **v5** | **Workout log** | **8 agentic tasks, 150 runs incl. add-ons** | **The results above: release evidence for v6.0** |
  266. | **v6** | **Reading log** | **6 agentic tasks, 90 runs** | **v6.0 vs v6.1 vs no kit: v6.1 released as stable ([results](#results-v61))** |
  267. Checks on the kit itself:
  268. - **Guard precision on a real app:** run read-only on a ~1,600-file production KMP codebase, the guard scripts went
  269. from 539 hits (527 false) to 12 hits, all true.
  270. - **Templates build:** following the bootstrap guide literally gave a zero-patch build for Android, desktop and iOS,
  271. and 36/36 scaffolded tests passed *(earlier candidate)*.
  272. ## Roadmap
  273. - [x] **v6.0.0-preview.1:** 7 skills, entry tree, 4 install channels *(released 2026-10-01)*
  274. - [x] **v6.1.0, stable:** the v6.1 fixes passed every pre-registered rule, and no model scored below its
  275. no-kit result. Conform over-scoping fixed (no-new-UI item 2/10 → 8/10) *(released 2026-10-05, [results](#results-v61))*
  276. - [x] **v6.1.1:** security fix for the guard scripts: `.composekit.conf` is read as data, never run as
  277. shell code, and a config line the guards cannot use fails the run *(released 2026-10-11)*
  278. - [x] **v6.1.2 (not published):** guard fixes prepared 2026-10-16 but never tagged; they ship in v6.2.0
  279. - [x] **v6.2.0:** adaptive layouts: one Screen per destination, layout variants picked by window size
  280. class (never by device), pane content sized by its own width, and the app's targets recorded at bootstrap. Guard
  281. fixes: `check-file-level-state` finds `lateinit` and annotated `var`s, mutable top-level `val`s and mutable `object`
  282. members in every source set; `check-contract-shape` reads same-line annotations *(released 2026-10-23)*
  283. - [x] **v6.3.0, now "Latest":** run and see: agents run a visible change and look at it before calling it done, or say
  284. it was not run, on Android, desktop, iOS or browser. Template pins refreshed; Android Studio install note
  285. *(released 2026-11-08)*
  286. - [x] **Cheap-model add-ons:** DeepSeek V4.1 Flash 32 → 38 and DeepSeek V4 Pro 29 → 34 of 39 with v6.1
  287. *(2026-10-05, [results](#add-ons-cheap-deepseek-models))*. MiniMax M3 was dropped; Kimi K3, GLM 5.3 and Qwen 3.8 Max are not planned
  288. - [ ] **Journey page** on the site: the full test history
  289. - [x] **Project site:** [compose.avinya.dev](https://compose.avinya.dev) *(live 2026-10-02)*
  290. - [ ] *Maybe:* a kit CLI for project tooling (new project, add feature, run guards), if users ask for it
  291. After the cheap-model add-ons, new benchmark rounds start only for a concrete reason: a reported problem, a major new
  292. model, or a large change to the kit.
  293. ## Known issues in v6.1
  294. - Muse Spark 1.3 gains little from the kit (+2 items) and can still over-scope "make this match our conventions" tasks.
  295. - New features can exceed 20k kit tokens when a model reads past the entry tree (5 of 10 v6.1 new-feature runs).
  296. - The v6 conform grading did not show graders the hidden-test result; the sensitivity check leaves the outcome
  297. unchanged, and future packets will state it.
  298. - One kit-shaped base project, so behaviour on a coherent non-kit project is not tested.
  299. - Installing only some of the skills isn't supported yet; skills cross-reference each other, so install all 7.
  300. ## Feedback
  301. Found a problem, or want a model benchmarked? [Open an issue](https://github.com/Meet-Miyani/compose-skill/issues).
  302. A reported problem is exactly what kicks off the next test round.
  303. ## Star history
  304. <a href="https://star-history.com/#Meet-Miyani/compose-skill&Date">
  305. <picture>
  306. <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=Meet-Miyani/compose-skill&type=Date&theme=dark">
  307. <img alt="Star history of Meet-Miyani/compose-skill" src="https://api.star-history.com/svg?repos=Meet-Miyani/compose-skill&type=Date" width="600">
  308. </picture>
  309. </a>
  310. ## License
  311. MIT, see [LICENSE](LICENSE). Attribution for the external sources used in the kit is in [NOTICE.md](NOTICE.md).
evals-v2/method/tools/v5-run-opencode.sh · Bundled script · 93 lines · 1 finding marked
  1. #!/bin/bash
  2. # Plan step 9 (v5) runner for OpenCode add-on models (copy of v5-run.sh plus an opencode branch; owner, 2026-10-01).
  3. # Add-on results are reported separately from the 5-model panel (m9.md).
  4. # Usage: v5-run-opencode.sh opencode <provider/model> <T1..T8> <nokit|generic|kit>
  5. # Clones the finished base app, installs the arm, applies the task setup, runs the agent headless, then adds the
  6. # hidden test (bug fixes) and runs the task's checks. Outputs in evals-v2/method/work/v5/runs/<model>/<task>-<arm>/.
  7. set -u
  8. export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-23.jdk/Contents/Home
  9. REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
  10. W=$REPO/evals-v2/method/work/v5
  11. V5=$REPO/evals-v2/heldout-v5
  12. TOOL=$1; MODEL=$2; TASK=$3; ARM=$4
  13. # Pause point between cells: while PAUSE exists, no new cell starts (running cells finish normally).
  14. while [ -f "$W/PAUSE" ]; do sleep 30; done
  15. OUT=$W/runs/${MODEL//\//_}/$TASK-$ARM
  16. [ -f "$OUT/meta.txt" ] && { echo "done already: $OUT"; exit 0; }
  17. rm -rf "$OUT"; mkdir -p "$OUT"
  18. git clone -q "$W/base" "$OUT/project"
  19. cp "$W/base/local.properties" "$OUT/project/"
  20. cd "$OUT/project" || exit 1
  21. field() { python3 - "$V5/tasks.md" "$TASK" "$1" <<'EOF'
  22. import re, sys
  23. t = open(sys.argv[1]).read()
  24. sec = re.search(rf"^## {sys.argv[2]}\b.*?(?=^## |\Z)", t, re.S | re.M).group(0)
  25. m = re.search(rf"^{sys.argv[3]}:\s*(.*?)(?=^\w[\w ]*:|\Z)", sec, re.S | re.M)
  26. print(m.group(1).strip() if m else "")
  27. EOF
  28. }
  29. PROMPT=$(field Prompt); SETUP=$(field Setup); HIDDEN=$(field "Hidden test"); CHECKS=$(field Checks)
  30. printf '%s\n' "$PROMPT" > "$OUT/prompt.txt"
  31. # 1. arm
  32. if [ "$ARM" != kit ]; then rm -rf scripts/composekit .composekit.conf .github/workflows/composekit.yml; fi
  33. case "$TOOL" in claude) INSTR=CLAUDE.md; SK=.claude/skills ;; *) INSTR=AGENTS.md; SK=.agents/skills ;; esac # opencode reads AGENTS.md and .agents/skills
  34. case "$ARM" in
  35. generic) cp "$REPO/evals-v2/control/generic-senior-prompt.md" "$INSTR" ;;
  36. kit) printf '%s\n' 'Compose/CMP work: load the compose skill first; it picks the path and the files to read.' > "$INSTR"
  37. mkdir -p "$SK"
  38. for s in compose compose-architecture compose-feature compose-ui compose-data compose-project compose-platform; do
  39. cp -R "$REPO/skills/$s" "$SK/"; done ;;
  40. esac
  41. [ "$TOOL" = agy ] && [ -f AGENTS.md ] && cp AGENTS.md GEMINI.md
  42. git add -A && git -c user.name=dev -c user.email=dev@example.com commit -qm "Project setup"
  43. # 2. task setup, as its own commit so a review task's "PR" is visible in git history
  44. TITLE=$(field Commit); [ -n "$TITLE" ] || TITLE="Update"
  45. case "$SETUP" in none|"") ;; *) bash "$V5/setup/$TASK.sh" > "$OUT/setup.log" 2>&1 || { echo "setup failed" >&2; exit 3; }
  46. git add -A && git -c user.name=dev -c user.email=dev@example.com commit -qm "$TITLE" ;; esac
  47. START_SHA=$(git rev-parse HEAD)
  48. # 3. agent (60 min hard cap)
  49. cap() { perl -e 'alarm 3600; exec @ARGV' "$@"; }
  50. T0=$(date +%s)
  51. case "$TOOL" in
  52. claude) cap claude -p "$PROMPT" --model "$MODEL" --setting-sources project,local --dangerously-skip-permissions \
  53. --output-format stream-json --verbose < /dev/null > "$OUT/agent.jsonl" 2> "$OUT/agent.err" ;;
  54. codex) cap codex exec --disable memories -m "$MODEL" -s workspace-write --add-dir "$HOME/.gradle" --add-dir "$HOME/.konan" \
  55. -c sandbox_workspace_write.network_access=true --json -o "$OUT/final.md" "$PROMPT" < /dev/null \
  56. > "$OUT/agent.jsonl" 2> "$OUT/agent.err" ;;
  57. opencode) cap opencode run --pure --auto --format json -m "$MODEL" "$PROMPT" < /dev/null \
  58. > "$OUT/agent.jsonl" 2> "$OUT/agent.err" ;;
  59. agy) cap agy --print "$PROMPT" --model "$MODEL" --dangerously-skip-permissions \
  60. --output-format stream-json --print-timeout 60m < /dev/null > "$OUT/agent.jsonl" 2> "$OUT/agent.err" ;;
  61. esac
  62. echo "agent_exit=$? seconds=$(( $(date +%s) - T0 ))" > "$OUT/meta.tmp"
  63. # 4. final message and diff (kit files excluded so graders stay blind to the arm)
  64. python3 - "$TOOL" "$OUT" <<'EOF'
  65. import json, sys, os
  66. tool, out = sys.argv[1], sys.argv[2]
  67. if tool == "codex": sys.exit(0)
  68. text = ""
  69. for l in open(os.path.join(out, "agent.jsonl"), errors="replace"):
  70. try: e = json.loads(l)
  71. except ValueError: continue
  72. if tool == "claude" and e.get("type") == "result": text = str(e.get("result", ""))
  73. if tool == "agy" and e.get("event") == "result": text = str((e.get("result") or {}).get("response", ""))
  74. if tool == "opencode" and e.get("type") == "text":
  75. p = e.get("part") or {}
  76. if p.get("messageID") != globals().get("_mid"): globals()["_mid"] = p.get("messageID"); text = ""
  77. text += p.get("text", "")
  78. open(os.path.join(out, "final.md"), "w").write(text)
  79. EOF
  80. git add -A >/dev/null 2>&1
  81. git diff --cached "$START_SHA" -- . ':(exclude).claude' ':(exclude).agents' ':(exclude)CLAUDE.md' \
  82. ':(exclude)AGENTS.md' ':(exclude)GEMINI.md' ':(exclude)**/build/**' > "$OUT/answer.diff"
  83. # 5. hidden test + checks
  84. case "$HIDDEN" in none|"") ;; *) D=$(cat "$V5/hidden/$TASK/DEST"); mkdir -p "$(dirname "$D")"
  85. cp "$(find "$V5/hidden/$TASK" -name '*.kt' | head -1)" "$D" ;; esac
  86. case "$CHECKS" in none|"") echo "checks=none" >> "$OUT/meta.tmp" ;;
  87. *) ./gradlew --console=plain $CHECKS < /dev/null > "$OUT/checks.log" 2>&1; echo "checks_exit=$?" >> "$OUT/meta.tmp" ;; esac
  88. mv "$OUT/meta.tmp" "$OUT/meta.txt"; cat "$OUT/meta.txt"
evals-v2/method/tools/v5-run.sh · Bundled script · 86 lines · 1 finding marked
  1. #!/bin/bash
  2. # Plan step 9 (v5) runner, pre-registered in evals-v2/method/preregistration-v5.md.
  3. # Usage: v5-run.sh <claude|codex|agy> <model> <T1..T8> <nokit|generic|kit>
  4. # Clones the finished base app, installs the arm, applies the task setup, runs the agent headless, then adds the
  5. # hidden test (bug fixes) and runs the task's checks. Outputs in evals-v2/method/work/v5/runs/<model>/<task>-<arm>/.
  6. set -u
  7. export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-23.jdk/Contents/Home
  8. REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"
  9. W=$REPO/evals-v2/method/work/v5
  10. V5=$REPO/evals-v2/heldout-v5
  11. TOOL=$1; MODEL=$2; TASK=$3; ARM=$4
  12. # Pause point between cells: while PAUSE exists, no new cell starts (running cells finish normally).
  13. while [ -f "$W/PAUSE" ]; do sleep 30; done
  14. OUT=$W/runs/$MODEL/$TASK-$ARM
  15. [ -f "$OUT/meta.txt" ] && { echo "done already: $OUT"; exit 0; }
  16. rm -rf "$OUT"; mkdir -p "$OUT"
  17. git clone -q "$W/base" "$OUT/project"
  18. cp "$W/base/local.properties" "$OUT/project/"
  19. cd "$OUT/project" || exit 1
  20. field() { python3 - "$V5/tasks.md" "$TASK" "$1" <<'EOF'
  21. import re, sys
  22. t = open(sys.argv[1]).read()
  23. sec = re.search(rf"^## {sys.argv[2]}\b.*?(?=^## |\Z)", t, re.S | re.M).group(0)
  24. m = re.search(rf"^{sys.argv[3]}:\s*(.*?)(?=^\w[\w ]*:|\Z)", sec, re.S | re.M)
  25. print(m.group(1).strip() if m else "")
  26. EOF
  27. }
  28. PROMPT=$(field Prompt); SETUP=$(field Setup); HIDDEN=$(field "Hidden test"); CHECKS=$(field Checks)
  29. printf '%s\n' "$PROMPT" > "$OUT/prompt.txt"
  30. # 1. arm
  31. if [ "$ARM" != kit ]; then rm -rf scripts/composekit .composekit.conf .github/workflows/composekit.yml; fi
  32. case "$TOOL" in claude) INSTR=CLAUDE.md; SK=.claude/skills ;; *) INSTR=AGENTS.md; SK=.agents/skills ;; esac
  33. case "$ARM" in
  34. generic) cp "$REPO/evals-v2/control/generic-senior-prompt.md" "$INSTR" ;;
  35. kit) printf '%s\n' 'Compose/CMP work: load the compose skill first; it picks the path and the files to read.' > "$INSTR"
  36. mkdir -p "$SK"
  37. for s in compose compose-architecture compose-feature compose-ui compose-data compose-project compose-platform; do
  38. cp -R "$REPO/skills/$s" "$SK/"; done ;;
  39. esac
  40. [ "$TOOL" = agy ] && [ -f AGENTS.md ] && cp AGENTS.md GEMINI.md
  41. git add -A && git -c user.name=dev -c user.email=dev@example.com commit -qm "Project setup"
  42. # 2. task setup, as its own commit so a review task's "PR" is visible in git history
  43. TITLE=$(field Commit); [ -n "$TITLE" ] || TITLE="Update"
  44. case "$SETUP" in none|"") ;; *) bash "$V5/setup/$TASK.sh" > "$OUT/setup.log" 2>&1 || { echo "setup failed" >&2; exit 3; }
  45. git add -A && git -c user.name=dev -c user.email=dev@example.com commit -qm "$TITLE" ;; esac
  46. START_SHA=$(git rev-parse HEAD)
  47. # 3. agent (60 min hard cap)
  48. cap() { perl -e 'alarm 3600; exec @ARGV' "$@"; }
  49. T0=$(date +%s)
  50. case "$TOOL" in
  51. claude) cap claude -p "$PROMPT" --model "$MODEL" --setting-sources project,local --dangerously-skip-permissions \
  52. --output-format stream-json --verbose < /dev/null > "$OUT/agent.jsonl" 2> "$OUT/agent.err" ;;
  53. codex) cap codex exec --disable memories -m "$MODEL" -s workspace-write --add-dir "$HOME/.gradle" --add-dir "$HOME/.konan" \
  54. -c sandbox_workspace_write.network_access=true --json -o "$OUT/final.md" "$PROMPT" < /dev/null \
  55. > "$OUT/agent.jsonl" 2> "$OUT/agent.err" ;;
  56. agy) cap agy --print "$PROMPT" --model "$MODEL" --dangerously-skip-permissions \
  57. --output-format stream-json --print-timeout 60m < /dev/null > "$OUT/agent.jsonl" 2> "$OUT/agent.err" ;;
  58. esac
  59. echo "agent_exit=$? seconds=$(( $(date +%s) - T0 ))" > "$OUT/meta.tmp"
  60. # 4. final message and diff (kit files excluded so graders stay blind to the arm)
  61. python3 - "$TOOL" "$OUT" <<'EOF'
  62. import json, sys, os
  63. tool, out = sys.argv[1], sys.argv[2]
  64. if tool == "codex": sys.exit(0)
  65. text = ""
  66. for l in open(os.path.join(out, "agent.jsonl"), errors="replace"):
  67. try: e = json.loads(l)
  68. except ValueError: continue
  69. if tool == "claude" and e.get("type") == "result": text = str(e.get("result", ""))
  70. if tool == "agy" and e.get("event") == "result": text = str((e.get("result") or {}).get("response", ""))
  71. open(os.path.join(out, "final.md"), "w").write(text)
  72. EOF
  73. git add -A >/dev/null 2>&1
  74. git diff --cached "$START_SHA" -- . ':(exclude).claude' ':(exclude).agents' ':(exclude)CLAUDE.md' \
  75. ':(exclude)AGENTS.md' ':(exclude)GEMINI.md' ':(exclude)**/build/**' > "$OUT/answer.diff"
  76. # 5. hidden test + checks
  77. case "$HIDDEN" in none|"") ;; *) D=$(cat "$V5/hidden/$TASK/DEST"); mkdir -p "$(dirname "$D")"
  78. cp "$(find "$V5/hidden/$TASK" -name '*.kt' | head -1)" "$D" ;; esac
  79. case "$CHECKS" in none|"") echo "checks=none" >> "$OUT/meta.tmp" ;;
  80. *) ./gradlew --console=plain $CHECKS < /dev/null > "$OUT/checks.log" 2>&1; echo "checks_exit=$?" >> "$OUT/meta.tmp" ;; esac
  81. mv "$OUT/meta.tmp" "$OUT/meta.txt"; cat "$OUT/meta.txt"
scripts/validate-skill.sh · Bundled script · 2,221 lines · 1 finding marked
  1. #!/usr/bin/env bash
  2. set -euo pipefail
  3. ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
  4. SKILL_PATH=""
  5. # ╔══════════════════════════════════════════════════════════════════╗
  6. # ║ Agent Skill Scanner v4 ║
  7. # ║ Validates skill packages against the agentskills.io spec ║
  8. # ║ and best practices. Runs locally and in CI (GitHub Actions). ║
  9. # ║ ║
  10. # ║ Spec: https://agentskills.io/specification ║
  11. # ║ Guide: https://agentskills.io/skill-creation ║
  12. # ║ ║
  13. # ║ Token Estimation: ║
  14. # ║ - Uses character/4 approximation (industry standard) ║
  15. # ║ - Accurate within ~10% for English text ║
  16. # ║ - Based on OpenAI tiktoken cl100k_base encoding ║
  17. # ║ ║
  18. # ║ Quality Score: 0-100 across 5 dimensions ║
  19. # ║ - Description Quality (30) · Spec Compliance (20) ║
  20. # ║ - Instruction Clarity (25) · Progressive Disclosure (15) ║
  21. # ║ - Security (10) ║
  22. # ╚══════════════════════════════════════════════════════════════════╝
  23. #
  24. # Usage:
  25. # ./scripts/validate-skill.sh # full scan of skills/compose
  26. # ./scripts/validate-skill.sh skills/compose-ui # scan another skill
  27. # ./scripts/validate-skill.sh --help # show usage
  28. #
  29. # Environment:
  30. # CI=true — emits GitHub Actions annotations (auto-detected)
  31. # NO_COLOR=1 — disable colored output
  32. SCAN_START=$SECONDS
  33. ERRORS=0
  34. WARNINGS=0
  35. INFO_COUNT=0
  36. CHECKS_RUN=0
  37. TOTAL_CHECKS=15
  38. CURRENT_SECTION=""
  39. CI="${CI:-false}"
  40. NO_COLOR="${NO_COLOR:-0}"
  41. OUTPUT_MODE="terminal"
  42. QUICK_MODE=false
  43. SKILL_FILE="SKILL.md"
  44. SKILL_DIR=""
  45. FINDINGS_FILE=$(mktemp)
  46. trap 'rm -f "$FINDINGS_FILE"' EXIT
  47. # ── Spec limits (agentskills.io/specification) ─────────────────────
  48. NAME_MAX_LEN=64
  49. DESC_MAX_LEN=1024
  50. DESC_MIN_USEFUL=30
  51. BODY_MAX_LINES=500
  52. TOKEN_BUDGET=5000
  53. COMPAT_MAX_LEN=500
  54. # ── Token budget zones ────────────────────────────────────────────
  55. TOKEN_SAFE_ZONE=3500
  56. TOKEN_WARN_ZONE=5000
  57. TOKEN_DANGER_ZONE=8000
  58. REF_TOKEN_SAFE=2000
  59. REF_TOKEN_WARN=3500
  60. REF_TOKEN_DANGER=4500
  61. TOTAL_TOKEN_SAFE=50000
  62. TOTAL_TOKEN_WARN=100000
  63. TOTAL_TOKEN_DANGER=200000
  64. # ── Score trackers (populated during checks, consumed by scoring) ──
  65. _S_HAS_NAME=false
  66. _S_NAME_OK=false
  67. _S_HAS_DESC=false
  68. _S_DESC_LEN=0
  69. _S_DESC_WHAT=false
  70. _S_DESC_WHEN=false
  71. _S_DESC_FIRST_PERSON=false
  72. _S_DESC_GENERIC=false
  73. _S_DESC_NEGATIVES=false
  74. _S_CODE_BLOCKS=0
  75. _S_HEADINGS=0
  76. _S_REF_LINKS=0
  77. _S_BODY_LINES=0
  78. _S_BODY_TOKENS=0
  79. _S_HAS_REFS_DIR=false
  80. _S_REFS_LINKED=0
  81. _S_REFS_TOTAL=0
  82. _S_FENCES_OK=true
  83. _S_SECRETS=0
  84. _S_HARDCODED=0
  85. _S_DANGEROUS=0
  86. QUALITY_SCORE=0
  87. QUALITY_GRADE="F"
  88. QUALITY_DESC=0
  89. QUALITY_CLARITY=0
  90. QUALITY_SPEC=0
  91. QUALITY_PROGRESSIVE=0
  92. QUALITY_SECURITY=0
  93. # ── Colors & formatting ───────────────────────────────────────────
  94. if [ "$NO_COLOR" = "1" ]; then
  95. _red() { printf '%s' "$*"; }
  96. _green() { printf '%s' "$*"; }
  97. _yellow() { printf '%s' "$*"; }
  98. _blue() { printf '%s' "$*"; }
  99. _cyan() { printf '%s' "$*"; }
  100. _magenta() { printf '%s' "$*"; }
  101. _bold() { printf '%s' "$*"; }
  102. _dim() { printf '%s' "$*"; }
  103. _bold_green() { printf '%s' "$*"; }
  104. _bold_red() { printf '%s' "$*"; }
  105. _bold_yellow() { printf '%s' "$*"; }
  106. _bold_cyan() { printf '%s' "$*"; }
  107. else
  108. _red() { printf "\033[0;31m%s\033[0m" "$*"; }
  109. _green() { printf "\033[0;32m%s\033[0m" "$*"; }
  110. _yellow() { printf "\033[0;33m%s\033[0m" "$*"; }
  111. _blue() { printf "\033[0;34m%s\033[0m" "$*"; }
  112. _cyan() { printf "\033[0;36m%s\033[0m" "$*"; }
  113. _magenta() { printf "\033[0;35m%s\033[0m" "$*"; }
  114. _bold() { printf "\033[1m%s\033[0m" "$*"; }
  115. _dim() { printf "\033[2m%s\033[0m" "$*"; }
  116. _bold_green() { printf "\033[1;32m%s\033[0m" "$*"; }
  117. _bold_red() { printf "\033[1;31m%s\033[0m" "$*"; }
  118. _bold_yellow() { printf "\033[1;33m%s\033[0m" "$*"; }
  119. _bold_cyan() { printf "\033[1;36m%s\033[0m" "$*"; }
  120. fi
  121. # ── Logging ────────────────────────────────────────────────────────
  122. _error() {
  123. ERRORS=$((ERRORS + 1))
  124. [ "$OUTPUT_MODE" = "terminal" ] && echo " $(_red "✗ ERROR") $*" || true
  125. echo "ERROR|${CURRENT_SECTION}|$*" >> "$FINDINGS_FILE"
  126. }
  127. _warn() {
  128. WARNINGS=$((WARNINGS + 1))
  129. [ "$OUTPUT_MODE" = "terminal" ] && echo " $(_yellow "! WARN ") $*" || true
  130. echo "WARN|${CURRENT_SECTION}|$*" >> "$FINDINGS_FILE"
  131. }
  132. _pass() {
  133. [ "$OUTPUT_MODE" = "terminal" ] && echo " $(_green "✓ PASS ") $*" || true
  134. }
  135. _info() {
  136. INFO_COUNT=$((INFO_COUNT + 1))
  137. [ "$OUTPUT_MODE" = "terminal" ] && echo " $(_blue "ℹ INFO ") $*" || true
  138. echo "INFO|${CURRENT_SECTION}|$*" >> "$FINDINGS_FILE"
  139. }
  140. _detail() {
  141. [ "$OUTPUT_MODE" = "terminal" ] && echo " $(_dim "$*")" || true
  142. }
  143. ci_annotate() {
  144. local level="$1"; shift
  145. if [ "$CI" = "true" ]; then
  146. echo "::${level} $*"
  147. fi
  148. }
  149. section() {
  150. CHECKS_RUN=$((CHECKS_RUN + 1))
  151. CURRENT_SECTION="$1"
  152. if [ "$OUTPUT_MODE" = "terminal" ]; then
  153. echo ""
  154. echo " $(_bold_cyan "[$CHECKS_RUN/$TOTAL_CHECKS]") $(_bold "$1")"
  155. echo " $(_dim "$(printf '%.0s─' $(seq 1 60))")"
  156. fi
  157. }
  158. file_lines() { wc -l < "$1" | tr -d ' '; }
  159. file_chars() { wc -c < "$1" | tr -d ' '; }
  160. file_words() { wc -w < "$1" | tr -d ' '; }
  161. estimate_tokens() {
  162. local chars
  163. chars=$(file_chars "$1")
  164. echo $(( chars / 4 ))
  165. }
  166. estimate_tokens_detailed() {
  167. local file="$1"
  168. local chars words lines char_tokens word_tokens
  169. chars=$(file_chars "$file")
  170. words=$(file_words "$file")
  171. lines=$(file_lines "$file")
  172. char_tokens=$(( chars / 4 ))
  173. word_tokens=$(( (words * 13) / 10 ))
  174. echo "$char_tokens|$chars|$words|$lines|$word_tokens"
  175. }
  176. token_zone_color() {
  177. local tokens="$1" safe="$2" warn="$3"
  178. if [ "$tokens" -le "$safe" ]; then
  179. echo "green"
  180. elif [ "$tokens" -le "$warn" ]; then
  181. echo "yellow"
  182. else
  183. echo "red"
  184. fi
  185. }
  186. format_tokens_colored() {
  187. local tokens="$1" safe="$2" warn="$3" label="${4:-tokens}"
  188. local zone
  189. zone=$(token_zone_color "$tokens" "$safe" "$warn")
  190. case "$zone" in
  191. green) printf "%s" "$(_green "~$tokens $label")" ;;
  192. yellow) printf "%s" "$(_yellow "~$tokens $label")" ;;
  193. red) printf "%s" "$(_red "~$tokens $label")" ;;
  194. esac
  195. }
  196. token_zone_indicator() {
  197. local tokens="$1" safe="$2" warn="$3"
  198. local zone pct
  199. zone=$(token_zone_color "$tokens" "$safe" "$warn")
  200. pct=$(( (tokens * 100) / warn ))
  201. case "$zone" in
  202. green)
  203. if [ "$NO_COLOR" = "1" ]; then echo "[SAFE $pct%]"
  204. else echo "$(_green "[SAFE $pct%]")"; fi ;;
  205. yellow)
  206. if [ "$NO_COLOR" = "1" ]; then echo "[WARN $pct%]"
  207. else echo "$(_yellow "[WARN $pct%]")"; fi ;;
  208. red)
  209. if [ "$NO_COLOR" = "1" ]; then echo "[HIGH $pct%]"
  210. else echo "$(_red "[HIGH $pct%]")"; fi ;;
  211. esac
  212. }
  213. # ── Visual helpers ─────────────────────────────────────────────────
  214. bar_gauge() {
  215. local value="$1" max="$2" width="${3:-20}"
  216. local pct filled empty bar=""
  217. if [ "$max" -le 0 ]; then pct=0
  218. else pct=$(( (value * 100) / max )); fi
  219. [ "$pct" -gt 100 ] && pct=100
  220. filled=$(( (pct * width) / 100 ))
  221. empty=$(( width - filled ))
  222. local i
  223. for ((i=0; i<filled; i++)); do bar+="█"; done
  224. for ((i=0; i<empty; i++)); do bar+="░"; done
  225. echo "$bar"
  226. }
  227. bar_gauge_colored() {
  228. local value="$1" max="$2" width="${3:-20}" safe="${4:-0}" warn="${5:-0}"
  229. local pct gauge
  230. if [ "$max" -le 0 ]; then pct=0
  231. else pct=$(( (value * 100) / max )); fi
  232. [ "$pct" -gt 100 ] && pct=100
  233. gauge=$(bar_gauge "$value" "$max" "$width")
  234. if [ "$safe" -gt 0 ] && [ "$warn" -gt 0 ]; then
  235. local zone
  236. zone=$(token_zone_color "$value" "$safe" "$warn")
  237. case "$zone" in
  238. green) printf "%s %s" "$(_green "$gauge")" "$(_green "${pct}%")" ;;
  239. yellow) printf "%s %s" "$(_yellow "$gauge")" "$(_yellow "${pct}%")" ;;
  240. red) printf "%s %s" "$(_red "$gauge")" "$(_red "${pct}%")" ;;
  241. esac
  242. else
  243. printf "%s %d%%" "$gauge" "$pct"
  244. fi
  245. }
  246. letter_grade() {
  247. local score="$1"
  248. if [ "$score" -ge 95 ]; then echo "A+"
  249. elif [ "$score" -ge 90 ]; then echo "A"
  250. elif [ "$score" -ge 85 ]; then echo "A-"
  251. elif [ "$score" -ge 80 ]; then echo "B+"
  252. elif [ "$score" -ge 75 ]; then echo "B"
  253. elif [ "$score" -ge 70 ]; then echo "B-"
  254. elif [ "$score" -ge 65 ]; then echo "C+"
  255. elif [ "$score" -ge 60 ]; then echo "C"
  256. elif [ "$score" -ge 55 ]; then echo "C-"
  257. elif [ "$score" -ge 50 ]; then echo "D"
  258. else echo "F"
  259. fi
  260. }
  261. grade_color() {
  262. local score="$1" grade
  263. grade=$(letter_grade "$score")
  264. if [ "$score" -ge 85 ]; then printf "%s" "$(_bold_green "$grade")"
  265. elif [ "$score" -ge 65 ]; then printf "%s" "$(_bold_yellow "$grade")"
  266. else printf "%s" "$(_bold_red "$grade")"
  267. fi
  268. }
  269. safe_count() {
  270. local result
  271. result=$("$@" 2>/dev/null | tr -d ' ' || true)
  272. if [ -z "$result" ] || ! [[ "$result" =~ ^[0-9]+$ ]]; then
  273. echo "0"
  274. else
  275. echo "$result"
  276. fi
  277. }
  278. # ════════════════════════════════════════════════════════════════════
  279. # CHECKS
  280. # ════════════════════════════════════════════════════════════════════
  281. # ── [1] Skill Structure ───────────────────────────────────────────
  282. check_structure() {
  283. section "Skill Structure"
  284. if [ ! -f "$SKILL_FILE" ]; then
  285. _error "SKILL.md not found — required by agentskills.io spec"
  286. ci_annotate "error" "file=SKILL.md::SKILL.md not found"
  287. return
  288. fi
  289. _pass "SKILL.md ($(file_lines "$SKILL_FILE") lines, $(file_chars "$SKILL_FILE") bytes)"
  290. if [ ! -f "README.md" ]; then
  291. _warn "README.md missing — recommended for discoverability and GitHub rendering"
  292. ci_annotate "warning" "file=README.md::README.md missing (recommended)"
  293. else
  294. _pass "README.md ($(file_lines README.md) lines)"
  295. fi
  296. local dirs=("references" "scripts" "assets" "agents")
  297. for dir in "${dirs[@]}"; do
  298. if [ -d "$dir" ]; then
  299. local count
  300. count=$(find "$dir" -type f | wc -l | tr -d ' ')
  301. _pass "$dir/ ($count files)"
  302. if [ "$dir" = "references" ]; then
  303. _S_HAS_REFS_DIR=true
  304. fi
  305. fi
  306. done
  307. if [ ! -d "references" ]; then
  308. _info "No references/ directory"
  309. _detail "Add reference docs for progressive disclosure of complex content"
  310. fi
  311. if [ -f "LICENSE" ] || [ -f "LICENSE.md" ] || [ -f "LICENSE.txt" ]; then
  312. _pass "License file found"
  313. fi
  314. }
  315. # ── [2] Frontmatter ───────────────────────────────────────────────
  316. check_frontmatter() {
  317. [ ! -f "$SKILL_FILE" ] && return
  318. section "Frontmatter & Description Quality"
  319. local first_line
  320. first_line=$(head -1 "$SKILL_FILE")
  321. if [ "$first_line" != "---" ]; then
  322. _error "Line 1: Expected '---' delimiter, found: '$first_line'"
  323. ci_annotate "error" "file=$SKILL_FILE,line=1::Missing YAML frontmatter delimiter"
  324. return
  325. fi
  326. local frontmatter
  327. frontmatter=$(sed -n '2,/^---$/p' "$SKILL_FILE" | sed '$d')
  328. # ── name ──
  329. local name
  330. name=$(echo "$frontmatter" | grep '^name:' | head -1 | sed 's/^name:[[:space:]]*//')
  331. if [ -z "$name" ]; then
  332. _error "Required field 'name' missing"
  333. ci_annotate "error" "file=$SKILL_FILE::Missing required 'name' field"
  334. else
  335. _S_HAS_NAME=true
  336. local name_len=${#name}
  337. local name_ok=true
  338. if [ "$name_len" -gt "$NAME_MAX_LEN" ]; then
  339. _error "name '$name' is $name_len chars (max $NAME_MAX_LEN)"
  340. ci_annotate "error" "file=$SKILL_FILE::name exceeds $NAME_MAX_LEN chars"
  341. name_ok=false
  342. fi
  343. if echo "$name" | grep -qE '[A-Z]'; then
  344. _error "name '$name' has uppercase — spec requires lowercase only"
  345. ci_annotate "error" "file=$SKILL_FILE::name must be lowercase"
  346. name_ok=false
  347. fi
  348. if echo "$name" | grep -qE '[^a-z0-9-]'; then
  349. _error "name '$name' has invalid chars — only a-z, 0-9, hyphens allowed"
  350. ci_annotate "error" "file=$SKILL_FILE::name contains invalid characters"
  351. name_ok=false
  352. fi
  353. if echo "$name" | grep -qE '^-|-$'; then
  354. _error "name '$name' starts or ends with hyphen"
  355. ci_annotate "error" "file=$SKILL_FILE::name starts/ends with hyphen"
  356. name_ok=false
  357. fi
  358. if echo "$name" | grep -qF -- '--'; then
  359. _error "name '$name' has consecutive hyphens (--)"
  360. ci_annotate "error" "file=$SKILL_FILE::name has consecutive hyphens"
  361. name_ok=false
  362. fi
  363. if [ "$name_ok" = true ]; then
  364. _pass "name: '$name' ($name_len chars)"
  365. _S_NAME_OK=true
  366. fi
  367. if [ "$name" != "$SKILL_DIR" ]; then
  368. _warn "name '$name' ≠ directory '$SKILL_DIR'"
  369. _detail "Spec: name should match parent directory name"
  370. ci_annotate "warning" "file=$SKILL_FILE::name doesn't match directory name"
  371. else
  372. _pass "name matches directory name"
  373. fi
  374. fi
  375. # ── description ──
  376. local desc_line desc=""
  377. desc_line=$(echo "$frontmatter" | grep -n '^description:' | head -1 | cut -d: -f1)
  378. if [ -z "$desc_line" ]; then
  379. _error "Required field 'description' missing"
  380. ci_annotate "error" "file=$SKILL_FILE::Missing required 'description' field"
  381. else
  382. _S_HAS_DESC=true
  383. local inline_desc
  384. inline_desc=$(echo "$frontmatter" | sed -n "${desc_line}p" | sed 's/^description:[[:space:]]*//')
  385. if [ -n "$inline_desc" ] && ! echo "$inline_desc" | grep -qE '^\s*[>|][-+]?\s*$'; then
  386. desc="$inline_desc"
  387. else
  388. desc=$(echo "$frontmatter" | sed -n "$((desc_line+1)),\$p" | sed '/^[a-zA-Z_-]*:/,$d' | tr '\n' ' ' | sed 's/^[[:space:]]*//' | sed 's/[[:space:]]*$//')
  389. fi
  390. local desc_len=${#desc}
  391. _S_DESC_LEN=$desc_len
  392. if [ "$desc_len" -eq 0 ]; then
  393. _error "description is empty"
  394. ci_annotate "error" "file=$SKILL_FILE::description is empty"
  395. else
  396. if [ "$desc_len" -gt "$DESC_MAX_LEN" ]; then
  397. _warn "description is $desc_len chars (spec max: $DESC_MAX_LEN)"
  398. ci_annotate "warning" "file=$SKILL_FILE::description exceeds $DESC_MAX_LEN chars"
  399. else
  400. _pass "description length: $desc_len chars (limit: $DESC_MAX_LEN)"
  401. fi
  402. if [ "$desc_len" -lt "$DESC_MIN_USEFUL" ]; then
  403. _warn "description is very short ($desc_len chars) — likely won't trigger well"
  404. _detail "Good: 'Extract text and tables from PDF files, fill forms, merge"
  405. _detail " documents. Use when working with PDF documents.'"
  406. _detail "Bad: 'Helps with PDFs.'"
  407. ci_annotate "warning" "file=$SKILL_FILE::description too short for reliable triggering"
  408. fi
  409. # WHAT and WHEN analysis
  410. local has_what=false has_when=false
  411. if echo "$desc" | grep -qiE 'build|create|generate|extract|analyze|process|manage|handle|configure|review|refactor|optimize|test|debug|deploy|format|validate|convert|transform|monitor|implement'; then
  412. has_what=true
  413. _S_DESC_WHAT=true
  414. fi
  415. if echo "$desc" | grep -qiE 'use when|when working|when the user|when handling|if the user|for tasks|for working|designed for|use this|use for'; then
  416. has_when=true
  417. _S_DESC_WHEN=true
  418. fi
  419. if [ "$has_what" = true ] && [ "$has_when" = true ]; then
  420. _pass "description covers WHAT and WHEN to use"
  421. elif [ "$has_what" = false ] && [ "$has_when" = false ]; then
  422. _warn "description may lack WHAT the skill does and WHEN to use it"
  423. _detail "Spec: 'Describes what the skill does and when to use it'"
  424. elif [ "$has_when" = false ]; then
  425. _info "description explains WHAT but could clarify WHEN to trigger"
  426. _detail "Adding 'Use when...' helps agents decide when to activate"
  427. fi
  428. # ── [NEW] First-person voice ──
  429. if echo "$desc" | grep -qiE '\bI can\b|\bI will\b|\bI help\b|\bI am\b|\bI provide\b|\bmy skill\b'; then
  430. _warn "description uses first-person voice ('I can', 'I will', etc.)"
  431. _detail "Descriptions are injected into the agent system prompt — first-person"
  432. _detail "mixes viewpoints and confuses agent reasoning. Use third person instead"
  433. ci_annotate "warning" "file=$SKILL_FILE::description uses first-person voice"
  434. _S_DESC_FIRST_PERSON=true
  435. else
  436. _pass "description uses correct voice (not first-person)"
  437. fi
  438. # ── [NEW] Generic verb detection ──
  439. local generic_matches
  440. generic_matches=$(echo "$desc" | grep -oiE '\b(manage|handle|deal with|work with|help with|assist with|take care of)\b' 2>/dev/null | head -3 || true)
  441. if [ -n "$generic_matches" ]; then
  442. local generic_list
  443. generic_list=$(echo "$generic_matches" | tr '\n' ', ' | sed 's/,$//')
  444. _info "description contains generic verbs: $generic_list"
  445. _detail "Specific verbs (extract, generate, validate) improve trigger precision"
  446. _S_DESC_GENERIC=true
  447. else
  448. _pass "description uses specific action verbs"
  449. fi
  450. # ── [NEW] Negative trigger detection ──
  451. if echo "$desc" | grep -qiE 'not for|not designed for|do not use|does not|don.t use'; then
  452. _pass "description includes boundary markers (negative triggers)"
  453. _S_DESC_NEGATIVES=true
  454. else
  455. _info "no negative triggers ('NOT for...', 'Do not use when...')"
  456. _detail "Negative triggers prevent mis-activation when many skills are loaded"
  457. fi
  458. fi
  459. fi
  460. # ── optional fields ──
  461. echo ""
  462. _detail "Optional fields:"
  463. if echo "$frontmatter" | grep -q '^license:'; then
  464. local license_val
  465. license_val=$(echo "$frontmatter" | grep '^license:' | sed 's/^license:[[:space:]]*//')
  466. _pass "license: $license_val"
  467. else
  468. _info "No license field — recommended for shared/public skills"
  469. fi
  470. if echo "$frontmatter" | grep -q '^compatibility:'; then
  471. local compat
  472. compat=$(echo "$frontmatter" | grep '^compatibility:' | sed 's/^compatibility:[[:space:]]*//')
  473. local compat_len=${#compat}
  474. if [ "$compat_len" -gt "$COMPAT_MAX_LEN" ]; then
  475. _warn "compatibility is $compat_len chars (spec max: $COMPAT_MAX_LEN)"
  476. else
  477. _pass "compatibility field present ($compat_len chars)"
  478. fi
  479. fi
  480. if echo "$frontmatter" | grep -q '^metadata:'; then
  481. _pass "metadata field present"
  482. if echo "$frontmatter" | grep -q '^\s*version:'; then
  483. _pass "metadata.version set"
  484. fi
  485. if echo "$frontmatter" | grep -q '^\s*author:'; then
  486. _pass "metadata.author set"
  487. fi
  488. fi
  489. if echo "$frontmatter" | grep -q '^allowed-tools:'; then
  490. _pass "allowed-tools field present (experimental)"
  491. fi
  492. }
  493. # ── [3] Body Content & Progressive Disclosure ─────────────────────
  494. check_body() {
  495. [ ! -f "$SKILL_FILE" ] && return
  496. section "Body Content & Progressive Disclosure"
  497. local body_start
  498. body_start=$(grep -n '^---$' "$SKILL_FILE" | sed -n '2p' | cut -d: -f1)
  499. if [ -z "$body_start" ]; then
  500. _error "No closing frontmatter delimiter (---) found"
  501. ci_annotate "error" "file=$SKILL_FILE::Missing closing frontmatter delimiter"
  502. return
  503. fi
  504. local body_lines token_est
  505. body_lines=$(tail -n +"$((body_start + 1))" "$SKILL_FILE" | wc -l | tr -d ' ')
  506. token_est=$(estimate_tokens "$SKILL_FILE")
  507. _S_BODY_LINES=$body_lines
  508. _S_BODY_TOKENS=$token_est
  509. # Line count with bar gauge
  510. if [ "$body_lines" -eq 0 ]; then
  511. _error "SKILL.md body is empty — agents need instructions"
  512. ci_annotate "error" "file=$SKILL_FILE::Empty body"
  513. elif [ "$body_lines" -gt "$BODY_MAX_LINES" ]; then
  514. _warn "Body: $body_lines lines (spec recommends <$BODY_MAX_LINES)"
  515. _detail "Move detailed content to references/ for on-demand loading"
  516. ci_annotate "warning" "file=$SKILL_FILE::Body exceeds $BODY_MAX_LINES lines"
  517. else
  518. _pass "Body: $body_lines lines (limit: $BODY_MAX_LINES)"
  519. fi
  520. local lines_gauge
  521. lines_gauge=$(bar_gauge_colored "$body_lines" "$BODY_MAX_LINES" 20 "$(( BODY_MAX_LINES * 70 / 100 ))" "$BODY_MAX_LINES")
  522. [ "$OUTPUT_MODE" = "terminal" ] && echo " $lines_gauge" || true
  523. # Token budget with bar gauge
  524. if [ "$token_est" -gt "$TOKEN_BUDGET" ]; then
  525. _warn "~$token_est tokens (spec recommends <$TOKEN_BUDGET on activation)"
  526. _detail "Progressive disclosure: SKILL.md body loads fully on activation"
  527. _detail "Keep instructions concise, move reference material to references/"
  528. ci_annotate "warning" "file=$SKILL_FILE::Estimated $token_est tokens exceeds $TOKEN_BUDGET budget"
  529. else
  530. _pass "~$token_est tokens (budget: $TOKEN_BUDGET)"
  531. fi
  532. local token_gauge
  533. token_gauge=$(bar_gauge_colored "$token_est" "$TOKEN_DANGER_ZONE" 20 "$TOKEN_SAFE_ZONE" "$TOKEN_WARN_ZONE")
  534. [ "$OUTPUT_MODE" = "terminal" ] && echo " $token_gauge" || true
  535. # Structure analysis
  536. local heading_count
  537. heading_count=$(tail -n +"$((body_start + 1))" "$SKILL_FILE" | grep -c '^#' 2>/dev/null || true)
  538. heading_count=${heading_count:-0}
  539. _S_HEADINGS=$heading_count
  540. if [ "$heading_count" -eq 0 ]; then
  541. _warn "No headings in body — add structure for readability"
  542. else
  543. _pass "$heading_count heading(s) providing structure"
  544. fi
  545. # Code examples
  546. local fence_pattern='```'
  547. local code_fence_count
  548. code_fence_count=$(grep -cF "$fence_pattern" "$SKILL_FILE" 2>/dev/null || true)
  549. code_fence_count=${code_fence_count:-0}
  550. local code_pairs=$(( code_fence_count / 2 ))
  551. _S_CODE_BLOCKS=$code_pairs
  552. local inline_code_count
  553. inline_code_count=$(grep -c '`[^`]' "$SKILL_FILE" 2>/dev/null || true)
  554. inline_code_count=${inline_code_count:-0}
  555. if [ "$code_pairs" -gt 0 ]; then
  556. _pass "$code_pairs fenced code example(s) in body"
  557. elif [ "$inline_code_count" -gt 0 ]; then
  558. _pass "$inline_code_count line(s) with inline code references"
  559. else
  560. _info "No code examples in body — consider adding for clarity"
  561. fi
  562. # Reference links (progressive disclosure pattern)
  563. local ref_link_count
  564. ref_link_count=$(tail -n +"$((body_start + 1))" "$SKILL_FILE" | grep -cE '\]\(references/' 2>/dev/null || true)
  565. ref_link_count=${ref_link_count:-0}
  566. _S_REF_LINKS=$ref_link_count
  567. if [ "$ref_link_count" -gt 0 ]; then
  568. _pass "$ref_link_count reference link(s) — using progressive disclosure"
  569. elif [ -d "references" ]; then
  570. local ref_file_count
  571. ref_file_count=$(find references -name '*.md' -type f | wc -l | tr -d ' ')
  572. if [ "$ref_file_count" -gt 0 ]; then
  573. _warn "references/ has $ref_file_count files but body has no links to them"
  574. _detail "Link references from SKILL.md body so agents can load them on demand"
  575. fi
  576. fi
  577. }
  578. # ── [4] Internal Links ────────────────────────────────────────────
  579. check_links() {
  580. [ ! -f "$SKILL_FILE" ] && return
  581. section "Internal Links"
  582. local link_data
  583. link_data=$(grep -nF '](' "$SKILL_FILE" || true)
  584. if [ -z "$link_data" ]; then
  585. _info "No internal links in SKILL.md"
  586. return
  587. fi
  588. local checked=0 broken=0
  589. while IFS= read -r match; do
  590. local line_num line_content
  591. line_num=$(echo "$match" | cut -d: -f1)
  592. line_content=$(echo "$match" | cut -d: -f2-)
  593. local targets
  594. targets=$(echo "$line_content" | tr ']' '\n' | grep '^(' | sed 's/^(\([^)]*\)).*/\1/' | sed '/^$/d')
  595. [ -z "$targets" ] && continue
  596. while IFS= read -r link_path; do
  597. case "$link_path" in http*|https*|"#"*|"") continue ;; esac
  598. local file_path
  599. file_path=$(echo "$link_path" | cut -d'#' -f1)
  600. [ -z "$file_path" ] && continue
  601. checked=$((checked + 1))
  602. if [ -f "$file_path" ]; then
  603. _pass "Line $line_num: $link_path → $(file_lines "$file_path") lines"
  604. else
  605. _error "Line $line_num: $link_path → FILE NOT FOUND"
  606. ci_annotate "error" "file=$SKILL_FILE,line=$line_num::Broken link: $link_path"
  607. broken=$((broken + 1))
  608. fi
  609. done <<< "$targets"
  610. done <<< "$link_data"
  611. echo ""
  612. _detail "Checked $checked link(s), $broken broken"
  613. }
  614. # ── [5] Reference Files ──────────────────────────────────────────
  615. check_references() {
  616. [ ! -d "references" ] && return
  617. section "Reference Files"
  618. local ref_links=""
  619. if [ -f "$SKILL_FILE" ]; then
  620. ref_links=$(grep -oE '\]\(references/[^)]+\)' "$SKILL_FILE" | sed 's/\](\(.*\))/\1/' | cut -d'#' -f1 | sort -u || true)
  621. fi
  622. local total=0 linked=0 orphaned=0 empty=0
  623. echo ""
  624. printf " $(_dim " %-35s %-8s %-8s %s")\n" "FILE" "LINES" "STATUS" ""
  625. echo " $(_dim " $(printf '%.0s─' $(seq 1 56))")"
  626. while IFS= read -r file; do
  627. total=$((total + 1))
  628. local lines
  629. lines=$(file_lines "$file")
  630. if [ "$lines" -eq 0 ]; then
  631. printf " $(_yellow " %-35s %-8s %-8s")\n" "$(basename "$file")" "0" "EMPTY"
  632. _warn "$file is empty (0 lines)"
  633. ci_annotate "warning" "file=$file::Empty reference file"
  634. empty=$((empty + 1))
  635. continue
  636. fi
  637. if echo "$ref_links" | grep -qF "$file"; then
  638. linked=$((linked + 1))
  639. printf " $(_green " %-35s %-8s %-8s")\n" "$(basename "$file")" "$lines" "LINKED"
  640. else
  641. orphaned=$((orphaned + 1))
  642. printf " $(_yellow " %-35s %-8s %-8s")\n" "$(basename "$file")" "$lines" "ORPHAN"
  643. _warn "$file — $(_yellow "orphaned")"
  644. _detail "Not linked from SKILL.md — agents won't discover this file"
  645. ci_annotate "warning" "file=$file::Not linked from SKILL.md (orphaned)"
  646. fi
  647. done < <(find references -name '*.md' -type f | sort)
  648. _S_REFS_TOTAL=$total
  649. _S_REFS_LINKED=$linked
  650. # Missing references (linked in SKILL.md but don't exist)
  651. local missing=0
  652. if [ -n "$ref_links" ]; then
  653. for ref in $ref_links; do
  654. if [ ! -f "$ref" ]; then
  655. _error "$ref → linked in SKILL.md but file is missing"
  656. ci_annotate "error" "file=$ref::Referenced in SKILL.md but does not exist"
  657. missing=$((missing + 1))
  658. fi
  659. done
  660. fi
  661. echo ""
  662. echo " $(_dim " ┌──────────────────────────────────────────┐")"
  663. printf " $(_dim " │") Total: %-4s Linked: %-4s Orphaned: %-4s$(_dim "│")\n" "$total" "$linked" "$orphaned"
  664. printf " $(_dim " │") Empty: %-4s Missing: %-4s $(_dim "│")\n" "$empty" "$missing"
  665. echo " $(_dim " └──────────────────────────────────────────┘")"
  666. }
  667. # ── [6] Markdown Syntax ──────────────────────────────────────────
  668. check_markdown() {
  669. section "Markdown Syntax"
  670. local pattern file_count=0 issues=0
  671. pattern='```'
  672. while IFS= read -r file; do
  673. file_count=$((file_count + 1))
  674. local count
  675. count=$(grep -cF "$pattern" "$file" 2>/dev/null || true)
  676. count=${count:-0}
  677. if [ "$count" -gt 0 ] && [ $((count % 2)) -ne 0 ]; then
  678. _warn "$file — unclosed code block ($count fences)"
  679. grep -nF "$pattern" "$file" 2>/dev/null | while IFS= read -r m; do
  680. _detail "Line $(echo "$m" | cut -d: -f1): $(echo "$m" | cut -d: -f2-)"
  681. done
  682. ci_annotate "warning" "file=$file::Unclosed code block ($count fences)"
  683. issues=$((issues + 1))
  684. fi
  685. done < <(find . -name '*.md' -not -path './.git/*' | sort)
  686. if [ "$issues" -eq 0 ]; then
  687. _pass "All $file_count markdown files have balanced code fences"
  688. else
  689. _S_FENCES_OK=false
  690. fi
  691. }
  692. # ── [7] Reference Nesting ────────────────────────────────────────
  693. check_reference_depth() {
  694. [ ! -d "references" ] && return
  695. section "Reference Nesting"
  696. local deep_refs=0
  697. while IFS= read -r file; do
  698. local nested
  699. nested=$(grep -cE '\]\(references/' "$file" 2>/dev/null || true)
  700. if [ "$nested" -gt 0 ]; then
  701. _warn "$file → $nested cross-reference(s) to other reference files"
  702. _detail "Spec: keep file references one level deep from SKILL.md"
  703. ci_annotate "warning" "file=$file::Cross-references between reference files"
  704. deep_refs=$((deep_refs + 1))
  705. fi
  706. done < <(find references -name '*.md' -type f | sort)
  707. if [ "$deep_refs" -eq 0 ]; then
  708. _pass "No nested reference chains — clean one-level structure"
  709. else
  710. _detail "$deep_refs file(s) with cross-references"
  711. fi
  712. }
  713. # ── [8] Scripts Validation ───────────────────────────────────────
  714. check_scripts() {
  715. [ ! -d "scripts" ] && return
  716. section "Scripts"
  717. local total=0 executable=0 not_executable=0 documented=0
  718. while IFS= read -r script; do
  719. total=$((total + 1))
  720. local basename_script
  721. basename_script=$(basename "$script")
  722. if [ -x "$script" ]; then
  723. executable=$((executable + 1))
  724. _pass "$script (executable, $(file_lines "$script") lines)"
  725. else
  726. not_executable=$((not_executable + 1))
  727. _warn "$script is not executable"
  728. _detail "Run: chmod +x $script"
  729. ci_annotate "warning" "file=$script::Script is not executable"
  730. fi
  731. if [ -f "$SKILL_FILE" ] && grep -qF "$basename_script" "$SKILL_FILE"; then
  732. documented=$((documented + 1))
  733. fi
  734. done < <(find scripts -type f | sort)
  735. if [ "$total" -gt 0 ] && [ "$documented" -eq 0 ] && [ -f "$SKILL_FILE" ]; then
  736. _info "No scripts referenced in SKILL.md body"
  737. _detail "Document available scripts so agents know what tools they can run"
  738. elif [ "$documented" -gt 0 ]; then
  739. _pass "$documented of $total script(s) documented in SKILL.md"
  740. fi
  741. }
  742. # ── [9] Package Integrity ────────────────────────────────────────
  743. check_repo_hygiene() {
  744. section "Repository Hygiene"
  745. local sensitive
  746. sensitive=$(find . -maxdepth 3 \
  747. \( -name '.env' -o -name '*.key' -o -name '*.pem' -o -name 'credentials*' \) \
  748. -not -path './.git/*' 2>/dev/null || true)
  749. if [ -n "$sensitive" ]; then
  750. _warn "Possible sensitive files detected"
  751. echo "$sensitive" | while IFS= read -r f; do
  752. _detail " $f"
  753. done
  754. else
  755. _pass "No sensitive files (.env, .key, .pem, credentials)"
  756. fi
  757. local artifacts
  758. artifacts=$(find . -maxdepth 3 \
  759. \( -name 'node_modules' -o -name '__pycache__' -o -name '.DS_Store' \) \
  760. -not -path './.git/*' 2>/dev/null || true)
  761. if [ -n "$artifacts" ]; then
  762. _warn "Development artifacts found — add to .gitignore"
  763. echo "$artifacts" | while IFS= read -r f; do
  764. _detail " $f"
  765. done
  766. else
  767. _pass "No development artifacts (node_modules, __pycache__, .DS_Store)"
  768. fi
  769. }
  770. # ── [10] Token Budget Analysis ────────────────────────────────────
  771. check_token_budget() {
  772. section "Token Budget Analysis"
  773. if [ "$OUTPUT_MODE" = "terminal" ]; then echo ""; fi
  774. _detail "Token estimation: chars/4 (tiktoken cl100k_base approximation)"
  775. _detail "Zone thresholds based on agentskills.io and context engineering best practices"
  776. if [ "$OUTPUT_MODE" = "terminal" ]; then echo ""; fi
  777. local total_tokens=0
  778. local total_chars=0
  779. local total_words=0
  780. local total_lines=0
  781. local file_count=0
  782. # ── SKILL.md analysis ──
  783. if [ -f "$SKILL_FILE" ]; then
  784. local skill_data skill_tokens skill_chars skill_words skill_lines
  785. skill_data=$(estimate_tokens_detailed "$SKILL_FILE")
  786. skill_tokens=$(echo "$skill_data" | cut -d'|' -f1)
  787. skill_chars=$(echo "$skill_data" | cut -d'|' -f2)
  788. skill_words=$(echo "$skill_data" | cut -d'|' -f3)
  789. skill_lines=$(echo "$skill_data" | cut -d'|' -f4)
  790. local zone_indicator
  791. zone_indicator=$(token_zone_indicator "$skill_tokens" "$TOKEN_SAFE_ZONE" "$TOKEN_WARN_ZONE")
  792. if [ "$OUTPUT_MODE" = "terminal" ]; then
  793. echo " $(_bold "SKILL.md") $zone_indicator"
  794. echo " ├─ $(format_tokens_colored "$skill_tokens" "$TOKEN_SAFE_ZONE" "$TOKEN_WARN_ZONE" "tokens")"
  795. local skill_gauge
  796. skill_gauge=$(bar_gauge_colored "$skill_tokens" "$TOKEN_DANGER_ZONE" 20 "$TOKEN_SAFE_ZONE" "$TOKEN_WARN_ZONE")
  797. echo " ├─ $skill_gauge"
  798. echo " ├─ $skill_chars chars | $skill_words words | $skill_lines lines"
  799. echo " └─ Budget: $TOKEN_WARN_ZONE tokens recommended"
  800. fi
  801. total_tokens=$((total_tokens + skill_tokens))
  802. total_chars=$((total_chars + skill_chars))
  803. total_words=$((total_words + skill_words))
  804. total_lines=$((total_lines + skill_lines))
  805. file_count=$((file_count + 1))
  806. if [ "$skill_tokens" -gt "$TOKEN_DANGER_ZONE" ]; then
  807. _warn "SKILL.md exceeds danger zone (~$skill_tokens tokens > $TOKEN_DANGER_ZONE)"
  808. _detail "Consider moving content to references/ for on-demand loading"
  809. fi
  810. fi
  811. # ── Reference files analysis ──
  812. if [ -d "references" ]; then
  813. if [ "$OUTPUT_MODE" = "terminal" ]; then
  814. echo ""
  815. echo " $(_bold "Reference Files:")"
  816. echo ""
  817. printf " $(_dim "%-35s %-12s %-6s %-6s")\n" "FILE" "ZONE" "TOKENS" "LINES"
  818. echo " $(_dim "$(printf '%.0s─' $(seq 1 62))")"
  819. fi
  820. local ref_total=0
  821. local ref_files=0
  822. local largest_ref=""
  823. local largest_ref_tokens=0
  824. while IFS= read -r file; do
  825. local ref_data ref_tokens ref_chars ref_words ref_lines
  826. ref_data=$(estimate_tokens_detailed "$file")
  827. ref_tokens=$(echo "$ref_data" | cut -d'|' -f1)
  828. ref_chars=$(echo "$ref_data" | cut -d'|' -f2)
  829. ref_words=$(echo "$ref_data" | cut -d'|' -f3)
  830. ref_lines=$(echo "$ref_data" | cut -d'|' -f4)
  831. local zone_indicator
  832. zone_indicator=$(token_zone_indicator "$ref_tokens" "$REF_TOKEN_SAFE" "$REF_TOKEN_WARN")
  833. local basename_file
  834. basename_file=$(basename "$file")
  835. [ "$OUTPUT_MODE" = "terminal" ] && printf " %-35s %-12s ~%-6d %d\n" "$basename_file" "$(echo "$zone_indicator" | sed 's/\x1b\[[0-9;]*m//g')" "$ref_tokens" "$ref_lines" || true
  836. ref_total=$((ref_total + ref_tokens))
  837. ref_files=$((ref_files + 1))
  838. total_tokens=$((total_tokens + ref_tokens))
  839. total_chars=$((total_chars + ref_chars))
  840. total_words=$((total_words + ref_words))
  841. total_lines=$((total_lines + ref_lines))
  842. file_count=$((file_count + 1))
  843. if [ "$ref_tokens" -gt "$largest_ref_tokens" ]; then
  844. largest_ref_tokens=$ref_tokens
  845. largest_ref="$basename_file"
  846. fi
  847. if [ "$ref_tokens" -gt "$REF_TOKEN_DANGER" ]; then
  848. _warn "$basename_file is too large (~$ref_tokens tokens > $REF_TOKEN_DANGER)"
  849. _detail "Split into base + advanced files (e.g., topic.md + topic-advanced.md)"
  850. elif [ "$ref_tokens" -gt "$REF_TOKEN_WARN" ]; then
  851. _warn "$basename_file is getting large (~$ref_tokens tokens > $REF_TOKEN_WARN)"
  852. _detail "Tighten prose: tables over paragraphs, cross-link shared patterns, trim tutorial content"
  853. fi
  854. done < <(find references -name '*.md' -type f | sort)
  855. if [ "$OUTPUT_MODE" = "terminal" ]; then
  856. echo ""
  857. echo " $(_dim "Reference subtotal: ~$ref_total tokens across $ref_files files")"
  858. if [ -n "$largest_ref" ]; then
  859. echo " $(_dim "Largest reference: $largest_ref (~$largest_ref_tokens tokens)")"
  860. fi
  861. fi
  862. fi
  863. # ── README analysis ──
  864. if [ -f "README.md" ]; then
  865. local readme_data readme_tokens readme_lines
  866. readme_data=$(estimate_tokens_detailed "README.md")
  867. readme_tokens=$(echo "$readme_data" | cut -d'|' -f1)
  868. readme_lines=$(echo "$readme_data" | cut -d'|' -f4)
  869. if [ "$OUTPUT_MODE" = "terminal" ]; then
  870. echo ""
  871. echo " $(_bold "README.md") $(_dim "(not loaded by agents, for humans)")"
  872. echo " └─ ~$readme_tokens tokens | $readme_lines lines"
  873. fi
  874. fi
  875. # ── Total package summary ──
  876. if [ "$OUTPUT_MODE" = "terminal" ]; then
  877. echo ""
  878. echo " $(_bold "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━")"
  879. local total_zone_indicator
  880. total_zone_indicator=$(token_zone_indicator "$total_tokens" "$TOTAL_TOKEN_SAFE" "$TOTAL_TOKEN_WARN")
  881. echo " $(_bold "TOTAL SKILL PACKAGE") $total_zone_indicator"
  882. echo ""
  883. echo " $(format_tokens_colored "$total_tokens" "$TOTAL_TOKEN_SAFE" "$TOTAL_TOKEN_WARN" "tokens") (estimated)"
  884. local total_gauge
  885. total_gauge=$(bar_gauge_colored "$total_tokens" "$TOTAL_TOKEN_DANGER" 25 "$TOTAL_TOKEN_SAFE" "$TOTAL_TOKEN_WARN")
  886. echo " $total_gauge"
  887. echo " $total_chars chars | $total_words words | $total_lines lines | $file_count files"
  888. echo ""
  889. fi
  890. local uses_progressive=false
  891. if [ -d "references" ] && [ -f "$SKILL_FILE" ]; then
  892. local ref_link_count
  893. ref_link_count=$(grep -cE '\]\(references/' "$SKILL_FILE" 2>/dev/null || true)
  894. ref_link_count=${ref_link_count:-0}
  895. if [ "$ref_link_count" -gt 0 ]; then
  896. uses_progressive=true
  897. fi
  898. fi
  899. local ctx_200k_pct ctx_128k_pct skill_only_200k_pct
  900. ctx_200k_pct=$(( (total_tokens * 100) / 200000 ))
  901. ctx_128k_pct=$(( (total_tokens * 100) / 128000 ))
  902. if [ -f "$SKILL_FILE" ]; then
  903. skill_only_200k_pct=$(( (_S_BODY_TOKENS * 100) / 200000 ))
  904. else
  905. skill_only_200k_pct=0
  906. fi
  907. if [ "$OUTPUT_MODE" = "terminal" ]; then
  908. echo " $(_bold "Context Window Impact:")"
  909. echo " $(_dim "Agent tools (Cursor, Windsurf, Copilot) typically use ~200K context")"
  910. echo " $(_dim "regardless of the underlying model's max window.")"
  911. echo ""
  912. echo " ├─ Agent context (200K): ~${ctx_200k_pct}% if all files loaded"
  913. echo " ├─ Agent context (128K): ~${ctx_128k_pct}% if all files loaded"
  914. if [ "$uses_progressive" = true ]; then
  915. echo " ├─ $(_green "SKILL.md only: ~${skill_only_200k_pct}% of 200K") (initial activation cost)"
  916. echo " └─ $(_dim "Progressive disclosure — references load on demand, not upfront")"
  917. else
  918. echo " └─ $(_yellow "Monolithic — entire skill loads at once (no references/ linked)")"
  919. fi
  920. echo ""
  921. fi
  922. if [ "$uses_progressive" = true ]; then
  923. if [ "$total_tokens" -le "$TOTAL_TOKEN_SAFE" ]; then
  924. _pass "Package is well-optimized for token efficiency"
  925. elif [ "$total_tokens" -le "$TOTAL_TOKEN_WARN" ]; then
  926. _pass "Package size is fine — progressive disclosure loads files on demand"
  927. else
  928. _info "Large package (~$total_tokens tokens) — ensure no dead/orphaned references"
  929. _detail "Total size is informational with progressive disclosure; individual file sizes are the real quality gate"
  930. fi
  931. else
  932. if [ "$total_tokens" -le "$TOKEN_BUDGET" ]; then
  933. _pass "Monolithic skill is within token budget"
  934. elif [ "$total_tokens" -le "$TOKEN_DANGER_ZONE" ]; then
  935. _warn "Monolithic skill (~$total_tokens tokens) exceeds $TOKEN_BUDGET budget"
  936. _detail "Move detailed content to references/ for on-demand loading"
  937. else
  938. _warn "Monolithic skill (~$total_tokens tokens) is very large"
  939. _detail "Split into SKILL.md + references/ for progressive disclosure"
  940. fi
  941. fi
  942. }
  943. # ── [11] Content Quality Metrics ─────────────────────────────────
  944. check_content_quality() {
  945. section "Content Quality Metrics"
  946. local total_code_blocks=0
  947. local total_headings=0
  948. if [ -f "$SKILL_FILE" ]; then
  949. local code_blocks headings internal_links external_links
  950. code_blocks=$(safe_count grep -c '```' "$SKILL_FILE")
  951. code_blocks=$((code_blocks / 2))
  952. headings=$(safe_count grep -c '^#' "$SKILL_FILE")
  953. internal_links=$(safe_count grep -c '\](references/' "$SKILL_FILE")
  954. external_links=$(safe_count grep -cE '\]\(https?://' "$SKILL_FILE")
  955. echo ""
  956. echo " $(_bold "SKILL.md Structure:")"
  957. echo " ├─ $headings heading(s) providing navigation"
  958. echo " ├─ $code_blocks code example(s)"
  959. echo " ├─ $internal_links internal link(s) to references"
  960. echo " └─ $external_links external link(s)"
  961. total_code_blocks=$code_blocks
  962. total_headings=$headings
  963. if [ "$headings" -lt 3 ]; then
  964. _info "Consider adding more headings for better navigation"
  965. fi
  966. if [ "$code_blocks" -lt 2 ]; then
  967. _info "Consider adding code examples for clarity"
  968. fi
  969. fi
  970. if [ -d "references" ]; then
  971. echo ""
  972. echo " $(_bold "Reference Files Quality:")"
  973. local ref_with_code=0
  974. local ref_without_code=0
  975. local total_ref_headings=0
  976. while IFS= read -r file; do
  977. local file_code_blocks file_headings
  978. file_code_blocks=$(safe_count grep -c '```' "$file")
  979. file_code_blocks=$((file_code_blocks / 2))
  980. file_headings=$(safe_count grep -c '^#' "$file")
  981. total_code_blocks=$((total_code_blocks + file_code_blocks))
  982. total_ref_headings=$((total_ref_headings + file_headings))
  983. if [ "$file_code_blocks" -gt 0 ]; then
  984. ref_with_code=$((ref_with_code + 1))
  985. else
  986. ref_without_code=$((ref_without_code + 1))
  987. fi
  988. done < <(find references -name '*.md' -type f)
  989. local ref_count
  990. ref_count=$(find references -name '*.md' -type f | wc -l | tr -d ' ')
  991. echo " ├─ $ref_with_code of $ref_count files contain code examples"
  992. echo " ├─ $total_ref_headings total headings across references"
  993. echo " └─ $total_code_blocks total code blocks in package"
  994. if [ "$ref_without_code" -gt 0 ]; then
  995. _info "$ref_without_code reference file(s) have no code examples"
  996. _detail "Code examples help agents understand expected patterns"
  997. fi
  998. fi
  999. # Keyword density analysis
  1000. if [ -f "$SKILL_FILE" ]; then
  1001. echo ""
  1002. echo " $(_bold "Trigger Keyword Analysis:")"
  1003. local compose_mentions kotlin_mentions android_mentions kmp_mentions
  1004. compose_mentions=$(grep -ioE '(compose|composable|@Composable|jetpack)' "$SKILL_FILE" 2>/dev/null | wc -l | tr -d ' ' || true)
  1005. kotlin_mentions=$(grep -ioE '(kotlin|coroutine|flow|stateflow|channel)' "$SKILL_FILE" 2>/dev/null | wc -l | tr -d ' ' || true)
  1006. android_mentions=$(grep -ioE '(android|viewmodel|hilt|koin|room|datastore)' "$SKILL_FILE" 2>/dev/null | wc -l | tr -d ' ' || true)
  1007. kmp_mentions=$(grep -ioE '(multiplatform|kmp|cmp|commonmain|ios|desktop)' "$SKILL_FILE" 2>/dev/null | wc -l | tr -d ' ' || true)
  1008. compose_mentions=${compose_mentions:-0}
  1009. kotlin_mentions=${kotlin_mentions:-0}
  1010. android_mentions=${android_mentions:-0}
  1011. kmp_mentions=${kmp_mentions:-0}
  1012. echo " ├─ Compose/UI: $compose_mentions mentions"
  1013. echo " ├─ Kotlin/Flow: $kotlin_mentions mentions"
  1014. echo " ├─ Android/DI: $android_mentions mentions"
  1015. echo " └─ Multiplatform: $kmp_mentions mentions"
  1016. local total_keywords=$((compose_mentions + kotlin_mentions + android_mentions + kmp_mentions))
  1017. if [ "$total_keywords" -lt 10 ]; then
  1018. _info "Low keyword density may reduce skill trigger accuracy"
  1019. _detail "Ensure description and body mention key terms agents should recognize"
  1020. else
  1021. _pass "Good keyword coverage for skill triggering ($total_keywords mentions)"
  1022. fi
  1023. fi
  1024. }
  1025. # ── [12] Agent Metadata ──────────────────────────────────────────
  1026. check_agents_metadata() {
  1027. section "Agent Metadata (Codex)"
  1028. if [ ! -f "agents/openai.yaml" ]; then
  1029. _info "No agents/openai.yaml — optional, configures Codex app UI and policy"
  1030. return
  1031. fi
  1032. _pass "agents/openai.yaml present"
  1033. if grep -q '^interface:' agents/openai.yaml; then
  1034. if grep -q 'display_name:' agents/openai.yaml; then
  1035. local display_name
  1036. display_name=$(grep 'display_name:' agents/openai.yaml | sed 's/.*display_name:[[:space:]]*//' | tr -d '"')
  1037. _pass "display_name: '$display_name'"
  1038. else
  1039. _info "No display_name — Codex uses skill name"
  1040. fi
  1041. if grep -q 'short_description:' agents/openai.yaml; then
  1042. _pass "short_description set"
  1043. else
  1044. _info "No short_description — Codex uses SKILL.md description"
  1045. fi
  1046. if grep -q 'default_prompt:' agents/openai.yaml; then
  1047. _pass "default_prompt set"
  1048. fi
  1049. if grep -q 'brand_color:' agents/openai.yaml; then
  1050. _pass "brand_color set"
  1051. fi
  1052. local icon_fields
  1053. icon_fields=$(grep -oE '(icon_small|icon_large):[[:space:]]*"[^"]+"' agents/openai.yaml 2>/dev/null || true)
  1054. if [ -n "$icon_fields" ]; then
  1055. echo "$icon_fields" | while IFS= read -r line; do
  1056. local field icon_path
  1057. field=$(echo "$line" | cut -d: -f1 | tr -d ' ')
  1058. icon_path=$(echo "$line" | sed 's/.*"\(.*\)"/\1/')
  1059. if [ -f "$icon_path" ]; then
  1060. _pass "$field: $icon_path"
  1061. else
  1062. _warn "$field: '$icon_path' → file not found"
  1063. ci_annotate "warning" "file=agents/openai.yaml::$field file missing: $icon_path"
  1064. fi
  1065. done
  1066. fi
  1067. fi
  1068. if grep -q '^policy:' agents/openai.yaml; then
  1069. if grep -q 'allow_implicit_invocation:' agents/openai.yaml; then
  1070. local implicit
  1071. implicit=$(grep 'allow_implicit_invocation:' agents/openai.yaml | sed 's/.*allow_implicit_invocation:[[:space:]]*//')
  1072. _pass "allow_implicit_invocation: $implicit"
  1073. fi
  1074. fi
  1075. if grep -q '^dependencies:' agents/openai.yaml; then
  1076. _pass "dependencies declared"
  1077. fi
  1078. }
  1079. # ── [13] Security Scan ───────────────────────────────────────────
  1080. check_security() {
  1081. section "Security Scan"
  1082. local secret_count=0
  1083. local path_count=0
  1084. local cmd_count=0
  1085. # ── Secrets detection (11 patterns from skill-tools / skill-validator) ──
  1086. local -a secret_patterns=(
  1087. 'sk-[a-zA-Z0-9]{20,}'
  1088. 'sk_live_[a-zA-Z0-9]+'
  1089. 'sk_test_[a-zA-Z0-9]+'
  1090. 'ghp_[a-zA-Z0-9]{36}'
  1091. 'gho_[a-zA-Z0-9]{36}'
  1092. 'ghu_[a-zA-Z0-9]{36}'
  1093. 'ghs_[a-zA-Z0-9]{36}'
  1094. 'ghr_[a-zA-Z0-9]{36}'
  1095. 'xoxb-[a-zA-Z0-9-]+'
  1096. 'xoxp-[a-zA-Z0-9-]+'
  1097. 'AKIA[0-9A-Z]{16}'
  1098. )
  1099. while IFS= read -r file; do
  1100. for pattern in "${secret_patterns[@]}"; do
  1101. local matches
  1102. matches=$(grep -nE "$pattern" "$file" 2>/dev/null | head -1 || true)
  1103. if [ -n "$matches" ]; then
  1104. secret_count=$((secret_count + 1))
  1105. local line_num
  1106. line_num=$(echo "$matches" | cut -d: -f1)
  1107. _error "$file:$line_num — potential secret/API key detected"
  1108. _detail "Pattern: $pattern"
  1109. ci_annotate "error" "file=$file,line=$line_num::Potential secret detected"
  1110. fi
  1111. done
  1112. # PEM private key headers
  1113. local pem_match
  1114. pem_match=$(grep -n 'BEGIN.*PRIVATE KEY' "$file" 2>/dev/null | head -1 || true)
  1115. if [ -n "$pem_match" ]; then
  1116. secret_count=$((secret_count + 1))
  1117. local line_num
  1118. line_num=$(echo "$pem_match" | cut -d: -f1)
  1119. _error "$file:$line_num — private key header detected"
  1120. ci_annotate "error" "file=$file,line=$line_num::Private key detected"
  1121. fi
  1122. # JWT tokens
  1123. local jwt_match
  1124. jwt_match=$(grep -nE 'eyJ[a-zA-Z0-9_-]{10,}\.[a-zA-Z0-9_-]{10,}\.' "$file" 2>/dev/null | head -1 || true)
  1125. if [ -n "$jwt_match" ]; then
  1126. secret_count=$((secret_count + 1))
  1127. local line_num
  1128. line_num=$(echo "$jwt_match" | cut -d: -f1)
  1129. _error "$file:$line_num — JWT token detected"
  1130. ci_annotate "error" "file=$file,line=$line_num::JWT token detected"
  1131. fi
  1132. done < <(find . -name '*.md' -not -path './.git/*' | sort)
  1133. _S_SECRETS=$secret_count
  1134. if [ "$secret_count" -eq 0 ]; then
  1135. _pass "No API keys or secrets detected (13 patterns checked)"
  1136. fi
  1137. # ── Hardcoded absolute paths ──
  1138. echo ""
  1139. _detail "Checking for hardcoded paths..."
  1140. while IFS= read -r file; do
  1141. local path_matches
  1142. path_matches=$(grep -nE '(/U[s]ers/[a-zA-Z]|/home/[a-zA-Z]|C:\\Users\\)' "$file" 2>/dev/null || true)
  1143. if [ -n "$path_matches" ]; then
  1144. while IFS= read -r match; do
  1145. path_count=$((path_count + 1))
  1146. local line_num
  1147. line_num=$(echo "$match" | cut -d: -f1)
  1148. _warn "$file:$line_num — hardcoded absolute path"
  1149. _detail "$(echo "$match" | cut -d: -f2- | sed 's/^[[:space:]]*//' | head -c 80)"
  1150. done <<< "$path_matches"
  1151. fi
  1152. done < <(find . -name '*.md' -not -path './.git/*' | sort)
  1153. _S_HARDCODED=$path_count
  1154. if [ "$path_count" -eq 0 ]; then
  1155. _pass "No hardcoded absolute paths"
  1156. fi
  1157. # ── Dangerous commands ──
  1158. echo ""
  1159. _detail "Checking for dangerous commands..."
  1160. local -a dangerous_patterns=(
  1161. 'rm -rf /'
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 1,376 on GitHub
  1162. 'sudo rm '
  1163. 'DROP TABLE'
  1164. 'DROP DATABASE'
  1165. 'chmod 777'
  1166. )
  1167. local -a dangerous_regex=(
  1168. 'curl .*\| *sh'
  1169. 'wget .*\| *sh'
  1170. )
  1171. while IFS= read -r file; do
  1172. for pattern in "${dangerous_patterns[@]}"; do
  1173. local matches
  1174. matches=$(grep -nF "$pattern" "$file" 2>/dev/null | head -1 || true)
  1175. if [ -n "$matches" ]; then
  1176. cmd_count=$((cmd_count + 1))
  1177. local line_num
  1178. line_num=$(echo "$matches" | cut -d: -f1)
  1179. _warn "$file:$line_num — potentially dangerous command"
  1180. _detail "Matched: $pattern"
  1181. ci_annotate "warning" "file=$file,line=$line_num::Dangerous command pattern"
  1182. fi
  1183. done
  1184. for pattern in "${dangerous_regex[@]}"; do
  1185. local matches
  1186. matches=$(grep -nE "$pattern" "$file" 2>/dev/null | head -1 || true)
  1187. if [ -n "$matches" ]; then
  1188. cmd_count=$((cmd_count + 1))
  1189. local line_num
  1190. line_num=$(echo "$matches" | cut -d: -f1)
  1191. _warn "$file:$line_num — potentially dangerous command"
  1192. _detail "Matched: $pattern"
  1193. ci_annotate "warning" "file=$file,line=$line_num::Dangerous command pattern"
  1194. fi
  1195. done
  1196. done < <(find . -name '*.md' -not -path './.git/*' | sort)
  1197. _S_DANGEROUS=$cmd_count
  1198. if [ "$cmd_count" -eq 0 ]; then
  1199. _pass "No dangerous shell commands detected"
  1200. fi
  1201. echo ""
  1202. echo " $(_dim " ┌─────────────────────────────────────────────┐")"
  1203. printf " $(_dim " │") Secrets: %-4s Paths: %-4s Commands: %-4s $(_dim "│")\n" "$secret_count" "$path_count" "$cmd_count"
  1204. echo " $(_dim " └─────────────────────────────────────────────┘")"
  1205. }
  1206. # ── [14] Heading Hierarchy & Duplicates ──────────────────────────
  1207. check_heading_hierarchy() {
  1208. section "Heading Hierarchy & Duplicates"
  1209. local skip_issues=0
  1210. local dup_issues=0
  1211. local files_checked=0
  1212. while IFS= read -r file; do
  1213. files_checked=$((files_checked + 1))
  1214. local prev_level=0
  1215. local line_num=0
  1216. local headings_seen=""
  1217. while IFS= read -r line; do
  1218. line_num=$((line_num + 1))
  1219. case "$line" in
  1220. '#'*)
  1221. local stripped_prefix
  1222. stripped_prefix="${line%%[^#]*}"
  1223. local level=${#stripped_prefix}
  1224. local text
  1225. text=$(echo "$line" | sed 's/^#* *//')
  1226. # MD001: heading level increment check
  1227. if [ "$prev_level" -gt 0 ] && [ "$level" -gt "$((prev_level + 1))" ]; then
  1228. skip_issues=$((skip_issues + 1))
  1229. _warn "$file:$line_num — heading skip H$prev_level → H$level"
  1230. _detail "$text"
  1231. ci_annotate "warning" "file=$file,line=$line_num::Heading level skip (H$prev_level to H$level)"
  1232. fi
  1233. prev_level=$level
  1234. # MD024: duplicate heading check (same level, same text)
  1235. local key="${level}:${text}"
  1236. if echo "$headings_seen" | grep -qF "<<${key}>>"; then
  1237. dup_issues=$((dup_issues + 1))
  1238. _warn "$file:$line_num — duplicate heading: '$text' (H$level)"
  1239. ci_annotate "warning" "file=$file,line=$line_num::Duplicate heading '$text'"
  1240. fi
  1241. headings_seen="${headings_seen}<<${key}>>"
  1242. ;;
  1243. esac
  1244. done < "$file"
  1245. done < <(find . -name '*.md' -not -path './.git/*' | sort)
  1246. if [ "$skip_issues" -eq 0 ]; then
  1247. _pass "No heading level skips across $files_checked files (MD001)"
  1248. fi
  1249. if [ "$dup_issues" -eq 0 ]; then
  1250. _pass "No duplicate sibling headings across $files_checked files (MD024)"
  1251. fi
  1252. echo ""
  1253. _detail "Checked $files_checked files | Level skips: $skip_issues | Duplicates: $dup_issues"
  1254. }
  1255. # ── [15] Quality Score ───────────────────────────────────────────
  1256. compute_quality_score() {
  1257. section "Quality Score"
  1258. local desc_score=0
  1259. local clarity_score=0
  1260. local spec_score=0
  1261. local progressive_score=0
  1262. local security_score=10
  1263. _check_mark() {
  1264. if [ "$1" = "1" ]; then printf "%s" "$(_green "✓")"; else printf "%s" "$(_red "✗")"; fi
  1265. }
  1266. # ── Description Quality (30 pts) ──
  1267. local d1=0 d2=0 d3=0 d4=0 d5=0 d6=0 d7=0
  1268. [ "$_S_HAS_DESC" = true ] && { desc_score=$((desc_score + 5)); d1=1; }
  1269. [ "$_S_DESC_LEN" -ge "$DESC_MIN_USEFUL" ] && { desc_score=$((desc_score + 3)); d2=1; }
  1270. [ "$_S_DESC_WHAT" = true ] && { desc_score=$((desc_score + 5)); d3=1; }
  1271. [ "$_S_DESC_WHEN" = true ] && { desc_score=$((desc_score + 5)); d4=1; }
  1272. [ "$_S_DESC_FIRST_PERSON" = false ] && { desc_score=$((desc_score + 4)); d5=1; }
  1273. [ "$_S_DESC_GENERIC" = false ] && { desc_score=$((desc_score + 4)); d6=1; }
  1274. [ "$_S_DESC_NEGATIVES" = true ] && { desc_score=$((desc_score + 4)); d7=1; }
  1275. # ── Instruction Clarity (25 pts) ──
  1276. local c1=0 c2=0 c3=0 c4=0 c5=0 c6=0
  1277. [ "$_S_CODE_BLOCKS" -ge 1 ] && { clarity_score=$((clarity_score + 5)); c1=1; }
  1278. [ "$_S_CODE_BLOCKS" -ge 3 ] && { clarity_score=$((clarity_score + 5)); c2=1; }
  1279. [ "$_S_HEADINGS" -ge 3 ] && { clarity_score=$((clarity_score + 5)); c3=1; }
  1280. [ "$_S_HEADINGS" -ge 5 ] && { clarity_score=$((clarity_score + 3)); c4=1; }
  1281. [ "$_S_REF_LINKS" -gt 0 ] && { clarity_score=$((clarity_score + 4)); c5=1; }
  1282. [ "$_S_REF_LINKS" -ge 5 ] && { clarity_score=$((clarity_score + 3)); c6=1; }
  1283. # ── Spec Compliance (20 pts) ──
  1284. local s1=0 s2=0 s3=0 s4=0 s5=0 s6=0
  1285. [ "$_S_HAS_NAME" = true ] && { spec_score=$((spec_score + 4)); s1=1; }
  1286. [ "$_S_NAME_OK" = true ] && { spec_score=$((spec_score + 4)); s2=1; }
  1287. [ "$_S_HAS_DESC" = true ] && { spec_score=$((spec_score + 4)); s3=1; }
  1288. [ "$_S_BODY_TOKENS" -le "$TOKEN_BUDGET" ] && { spec_score=$((spec_score + 4)); s4=1; }
  1289. [ "$_S_BODY_LINES" -le "$BODY_MAX_LINES" ] && { spec_score=$((spec_score + 2)); s5=1; }
  1290. [ "$_S_FENCES_OK" = true ] && { spec_score=$((spec_score + 2)); s6=1; }
  1291. # ── Progressive Disclosure (15 pts) ──
  1292. local p1=0 p2=0 p3=0 p4=0
  1293. [ "$_S_HAS_REFS_DIR" = true ] && { progressive_score=$((progressive_score + 5)); p1=1; }
  1294. [ "$_S_REFS_LINKED" -gt 0 ] && { progressive_score=$((progressive_score + 5)); p2=1; }
  1295. if [ "$_S_REFS_TOTAL" -gt 0 ] && [ "$_S_REFS_LINKED" -eq "$_S_REFS_TOTAL" ]; then
  1296. progressive_score=$((progressive_score + 3)); p3=1
  1297. fi
  1298. [ "$_S_BODY_LINES" -lt "$BODY_MAX_LINES" ] && { progressive_score=$((progressive_score + 2)); p4=1; }
  1299. # ── Security (10 pts — start at 10, deduct for issues) ──
  1300. local x1=1 x2=1 x3=1
  1301. if [ "$_S_SECRETS" -gt 0 ]; then security_score=$((security_score - 5)); x1=0; fi
  1302. if [ "$_S_HARDCODED" -gt 0 ]; then security_score=$((security_score - 3)); x2=0; fi
  1303. if [ "$_S_DANGEROUS" -gt 0 ]; then security_score=$((security_score - 2)); x3=0; fi
  1304. [ "$security_score" -lt 0 ] && security_score=0
  1305. local total_score=$((desc_score + clarity_score + spec_score + progressive_score + security_score))
  1306. QUALITY_SCORE=$total_score
  1307. QUALITY_GRADE=$(letter_grade "$total_score")
  1308. QUALITY_DESC=$desc_score
  1309. QUALITY_CLARITY=$clarity_score
  1310. QUALITY_SPEC=$spec_score
  1311. QUALITY_PROGRESSIVE=$progressive_score
  1312. QUALITY_SECURITY=$security_score
  1313. # ── Display: clean bar chart overview (terminal only) ──
  1314. if [ "$OUTPUT_MODE" = "terminal" ]; then
  1315. _dim_bar() {
  1316. local name="$1" score="$2" max="$3"
  1317. local gauge pct
  1318. if [ "$max" -le 0 ]; then pct=0
  1319. else pct=$(( (score * 100) / max )); fi
  1320. gauge=$(bar_gauge "$score" "$max" 15)
  1321. local colored_gauge
  1322. if [ "$pct" -ge 80 ]; then
  1323. colored_gauge=$(_green "$gauge")
  1324. elif [ "$pct" -ge 60 ]; then
  1325. colored_gauge=$(_yellow "$gauge")
  1326. else
  1327. colored_gauge=$(_red "$gauge")
  1328. fi
  1329. if [ "$score" -eq "$max" ]; then
  1330. printf " %-24s %s %2d / %-2d $(_green "✓")\n" "$name" "$colored_gauge" "$score" "$max"
  1331. else
  1332. printf " %-24s %s %2d / %-2d\n" "$name" "$colored_gauge" "$score" "$max"
  1333. fi
  1334. }
  1335. echo ""
  1336. echo " $(_bold "Score Overview:")"
  1337. echo ""
  1338. _dim_bar "Description Quality" "$desc_score" "30"
  1339. _dim_bar "Instruction Clarity" "$clarity_score" "25"
  1340. _dim_bar "Spec Compliance" "$spec_score" "20"
  1341. _dim_bar "Progressive Disclosure" "$progressive_score" "15"
  1342. _dim_bar "Security" "$security_score" "10"
  1343. echo ""
  1344. echo " $(_dim "$(printf '%.0s─' $(seq 1 52))")"
  1345. local total_gauge
  1346. total_gauge=$(bar_gauge "$total_score" 100 20)
  1347. local colored_total
  1348. if [ "$total_score" -ge 85 ]; then
  1349. colored_total=$(_bold_green "$total_gauge")
  1350. elif [ "$total_score" -ge 65 ]; then
  1351. colored_total=$(_bold_yellow "$total_gauge")
  1352. else
  1353. colored_total=$(_bold_red "$total_gauge")
  1354. fi
  1355. printf " %-24s %s %d/100 Grade: %s\n" "$(_bold "TOTAL")" "$colored_total" "$total_score" "$(grade_color "$total_score")"
  1356. fi # end terminal-only display
  1357. # ── Collect failed sub-checks with sources and actionable fixes ──
  1358. # Sources verified against:
  1359. # [spec] agentskills.io/specification
  1360. # [official] agentskills.io/skill-creation/best-practices
  1361. # agentskills.io/skill-creation/optimizing-descriptions
  1362. # [community] mdskills.ai/docs/skill-best-practices
  1363. local fix_count=0
  1364. local -a fix_labels=()
  1365. local -a fix_sources=()
  1366. local -a fix_dims=()
  1367. [ "$d1" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Description"); fix_labels+=("Add a 'description:' field to frontmatter (+5)"); fix_sources+=("spec: required field — agentskills.io/specification#description-field"); }
  1368. [ "$d2" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Description"); fix_labels+=("Write a longer description (>=${DESC_MIN_USEFUL} chars) (+3)"); fix_sources+=("official: 'A few sentences to a short paragraph' — agentskills.io/skill-creation/optimizing-descriptions"); }
  1369. [ "$d3" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Description"); fix_labels+=("Add action verbs: build, generate, validate, extract, etc. (+5)"); fix_sources+=("spec: 'Describes what the skill does' — agentskills.io/specification#description-field"); }
  1370. [ "$d4" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Description"); fix_labels+=("Add 'Use when...' trigger phrase to description (+5)"); fix_sources+=("official: 'Use imperative phrasing: Use this skill when...' — agentskills.io/skill-creation/optimizing-descriptions"); }
  1371. [ "$d5" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Description"); fix_labels+=("Rewrite in third person — remove 'I can/will/help' (+4)"); fix_sources+=("community: 'first person causes discovery problems' — mdskills.ai/docs/skill-best-practices"); }
  1372. [ "$d6" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Description"); fix_labels+=("Replace generic verbs (manage, handle) with specific ones (+4)"); fix_sources+=("community: 'Be specific about format, operation, trigger' — mdskills.ai/docs/skill-best-practices"); }
  1373. [ "$d7" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Description"); fix_labels+=("Add boundary markers: 'Not for...', 'Do not use when...' (+4)"); fix_sources+=("official: 'Add specificity about what the skill does not do' — agentskills.io/skill-creation/optimizing-descriptions"); }
  1374. [ "$c1" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Clarity"); fix_labels+=("Add at least 1 fenced code block to SKILL.md body (+5)"); fix_sources+=("spec: recommended section 'Examples of inputs and outputs' — agentskills.io/specification#body-content"); }
  1375. [ "$c2" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Clarity"); fix_labels+=("Add 3+ code examples to SKILL.md (+5)"); fix_sources+=("official: 'a working example tends to outperform exhaustive documentation' — agentskills.io/skill-creation/best-practices"); }
  1376. [ "$c3" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Clarity"); fix_labels+=("Add 3+ markdown headings for navigation (+5)"); fix_sources+=("community: 'structure for agent scanning' — mdskills.ai/docs/skill-best-practices"); }
  1377. [ "$c4" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Clarity"); fix_labels+=("Add 5+ headings for deeper structure (+3)"); fix_sources+=("community: structured content aids agent navigation — mdskills.ai/docs/skill-best-practices"); }
  1378. [ "$c5" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Clarity"); fix_labels+=("Link to at least one reference file from SKILL.md body (+4)"); fix_sources+=("official: 'tell the agent when to load each file' — agentskills.io/skill-creation/best-practices"); }
  1379. [ "$c6" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Clarity"); fix_labels+=("Link to 5+ reference files for full coverage (+3)"); fix_sources+=("official: progressive disclosure via on-demand loading — agentskills.io/specification#progressive-disclosure"); }
  1380. [ "$s4" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Spec"); fix_labels+=("Reduce SKILL.md to <$TOKEN_BUDGET tokens — move content to references/ (+4)"); fix_sources+=("spec: '<5000 tokens recommended' — agentskills.io/specification#progressive-disclosure"); }
  1381. [ "$s5" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Spec"); fix_labels+=("Reduce SKILL.md body to <$BODY_MAX_LINES lines (+2)"); fix_sources+=("spec: 'under 500 lines' — agentskills.io/specification#progressive-disclosure"); }
  1382. [ "$s6" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Spec"); fix_labels+=("Fix unclosed code fences in markdown files (+2)"); fix_sources+=("spec: valid markdown required for agent parsing"); }
  1383. [ "$p1" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Disclosure"); fix_labels+=("Create a references/ directory for detailed docs (+5)"); fix_sources+=("spec: optional directory for on-demand loading — agentskills.io/specification#references"); }
  1384. [ "$p2" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Disclosure"); fix_labels+=("Add links to reference files from SKILL.md body (+5)"); fix_sources+=("official: 'tell the agent when to load each file' — agentskills.io/skill-creation/best-practices"); }
  1385. [ "$p3" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Disclosure"); fix_labels+=("Link all reference files from SKILL.md — orphaned files found (+3)"); fix_sources+=("spec: 'use relative paths from skill root' — agentskills.io/specification#file-references"); }
  1386. [ "$p4" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Disclosure"); fix_labels+=("Keep body under $BODY_MAX_LINES lines (+2)"); fix_sources+=("spec: 'under 500 lines' — agentskills.io/specification#progressive-disclosure"); }
  1387. [ "$x1" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Security"); fix_labels+=("Remove detected API keys/secrets from markdown files (+5)"); fix_sources+=("community: 'Never hardcode credentials' — mdskills.ai/docs/skill-best-practices"); }
  1388. [ "$x2" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Security"); fix_labels+=("Replace hardcoded user home paths with relative paths (+3)"); fix_sources+=("spec: 'use relative paths from the skill root' — agentskills.io/specification#file-references"); }
  1389. [ "$x3" = "0" ] && { fix_count=$((fix_count+1)); fix_dims+=("Security"); fix_labels+=("Remove or guard dangerous commands (rm -rf, chmod 777) (+2)"); fix_sources+=("community: 'Shell commands need guardrails' — mdskills.ai/docs/skill-best-practices"); }
  1390. # ── Display: "How to Improve" section with sourced fixes (terminal only) ──
  1391. if [ "$OUTPUT_MODE" = "terminal" ]; then
  1392. if [ "$fix_count" -gt 0 ]; then
  1393. echo ""
  1394. echo ""
  1395. echo " $(_bold "┌────────────────────────────────────────────────────────┐")"
  1396. echo " $(_bold "│") $(_bold "HOW TO REACH 100/100") $(_dim "($fix_count items, by impact)") $(_bold "│")"
  1397. echo " $(_bold "└────────────────────────────────────────────────────────┘")"
  1398. local prev_dim=""
  1399. local i
  1400. for ((i=0; i<fix_count; i++)); do
  1401. local dim="${fix_dims[$i]}"
  1402. local label="${fix_labels[$i]}"
  1403. local source="${fix_sources[$i]}"
  1404. if [ "$dim" != "$prev_dim" ]; then
  1405. echo ""
  1406. echo " $(_bold_cyan "$dim:")"
  1407. prev_dim="$dim"
  1408. fi
  1409. echo " $(_yellow "→") $label"
  1410. echo " $(_dim "$source")"
  1411. done
  1412. echo ""
  1413. echo " $(_dim "Sources: spec = agentskills.io/specification")"
  1414. echo " $(_dim " official = agentskills.io/skill-creation/*")"
  1415. echo " $(_dim " community = mdskills.ai/docs/skill-best-practices")"
  1416. else
  1417. echo ""
  1418. _pass "Perfect score — no improvements needed"
  1419. fi
  1420. fi
  1421. }
  1422. # ════════════════════════════════════════════════════════════════════
  1423. # OUTPUT MODES
  1424. # ════════════════════════════════════════════════════════════════════
  1425. print_json() {
  1426. local elapsed=$(( SECONDS - SCAN_START ))
  1427. # Collect findings into JSON arrays
  1428. local errors_json="[]" warnings_json="[]" info_json="[]"
  1429. local err_items=""
  1430. while IFS='|' read -r sev sec msg; do
  1431. [ -z "$sev" ] && continue
  1432. msg=$(echo "$msg" | sed 's/"/\\"/g')
  1433. sec=$(echo "$sec" | sed 's/"/\\"/g')
  1434. case "$sev" in
  1435. ERROR) err_items="${err_items}{\"section\":\"$sec\",\"message\":\"$msg\"}," ;;
  1436. esac
  1437. done < "$FINDINGS_FILE"
  1438. local warn_items=""
  1439. while IFS='|' read -r sev sec msg; do
  1440. [ -z "$sev" ] && continue
  1441. msg=$(echo "$msg" | sed 's/"/\\"/g')
  1442. sec=$(echo "$sec" | sed 's/"/\\"/g')
  1443. case "$sev" in
  1444. WARN) warn_items="${warn_items}{\"section\":\"$sec\",\"message\":\"$msg\"}," ;;
  1445. esac
  1446. done < "$FINDINGS_FILE"
  1447. local info_items=""
  1448. while IFS='|' read -r sev sec msg; do
  1449. [ -z "$sev" ] && continue
  1450. msg=$(echo "$msg" | sed 's/"/\\"/g')
  1451. sec=$(echo "$sec" | sed 's/"/\\"/g')
  1452. case "$sev" in
  1453. INFO) info_items="${info_items}{\"section\":\"$sec\",\"message\":\"$msg\"}," ;;
  1454. esac
  1455. done < "$FINDINGS_FILE"
  1456. # Strip trailing commas and wrap
  1457. err_items="${err_items%,}"
  1458. warn_items="${warn_items%,}"
  1459. info_items="${info_items%,}"
  1460. [ -n "$err_items" ] && errors_json="[$err_items]"
  1461. [ -n "$warn_items" ] && warnings_json="[$warn_items]"
  1462. [ -n "$info_items" ] && info_json="[$info_items]"
  1463. local result="PASS"
  1464. [ "$WARNINGS" -gt 0 ] && result="PASS_WITH_WARNINGS"
  1465. [ "$ERRORS" -gt 0 ] && result="FAIL"
  1466. cat <<ENDJSON
  1467. {
  1468. "version": "4.0",
  1469. "skill": "$(pwd)",
  1470. "timestamp": "$(date -u '+%Y-%m-%dT%H:%M:%SZ')",
  1471. "duration_seconds": $elapsed,
  1472. "result": "$result",
  1473. "counts": {
  1474. "checks": $CHECKS_RUN,
  1475. "errors": $ERRORS,
  1476. "warnings": $WARNINGS,
  1477. "info": $INFO_COUNT
  1478. },
  1479. "quality_score": {
  1480. "total": $QUALITY_SCORE,
  1481. "grade": "$QUALITY_GRADE",
  1482. "dimensions": {
  1483. "description_quality": { "score": $QUALITY_DESC, "max": 30 },
  1484. "instruction_clarity": { "score": $QUALITY_CLARITY, "max": 25 },
  1485. "spec_compliance": { "score": $QUALITY_SPEC, "max": 20 },
  1486. "progressive_disclosure": { "score": $QUALITY_PROGRESSIVE, "max": 15 },
  1487. "security": { "score": $QUALITY_SECURITY, "max": 10 }
  1488. }
  1489. },
  1490. "token_analysis": {
  1491. "skill_md_tokens": $_S_BODY_TOKENS,
  1492. "body_lines": $_S_BODY_LINES,
  1493. "code_blocks": $_S_CODE_BLOCKS,
  1494. "headings": $_S_HEADINGS,
  1495. "reference_links": $_S_REF_LINKS,
  1496. "reference_files_total": $_S_REFS_TOTAL,
  1497. "reference_files_linked": $_S_REFS_LINKED
  1498. },
  1499. "findings": {
  1500. "errors": $errors_json,
  1501. "warnings": $warnings_json,
  1502. "info": $info_json
  1503. }
  1504. }
  1505. ENDJSON
  1506. }
  1507. print_markdown() {
  1508. local elapsed=$(( SECONDS - SCAN_START ))
  1509. local result_icon result_text
  1510. if [ "$ERRORS" -gt 0 ]; then
  1511. result_icon="x" result_text="FAIL — $ERRORS error(s), $WARNINGS warning(s)"
  1512. elif [ "$WARNINGS" -gt 0 ]; then
  1513. result_icon="!" result_text="PASS with $WARNINGS warning(s)"
  1514. else
  1515. result_icon="+" result_text="All checks passed"
  1516. fi
  1517. cat <<ENDMD
  1518. # Agent Skill Scan Report
  1519. **Score: $QUALITY_SCORE/100 ($QUALITY_GRADE)** | Result: $result_text | Duration: ${elapsed}s
  1520. ## Score Breakdown
  1521. | Dimension | Score | Max | % |
  1522. |---|---|---|---|
  1523. | Description Quality | $QUALITY_DESC | 30 | $(( (QUALITY_DESC * 100) / 30 ))% |
  1524. | Instruction Clarity | $QUALITY_CLARITY | 25 | $(( (QUALITY_CLARITY * 100) / 25 ))% |
  1525. | Spec Compliance | $QUALITY_SPEC | 20 | $(( (QUALITY_SPEC * 100) / 20 ))% |
  1526. | Progressive Disclosure | $QUALITY_PROGRESSIVE | 15 | $(( (QUALITY_PROGRESSIVE * 100) / 15 ))% |
  1527. | Security | $QUALITY_SECURITY | 10 | $(( (QUALITY_SECURITY * 100) / 10 ))% |
  1528. | **Total** | **$QUALITY_SCORE** | **100** | **${QUALITY_SCORE}%** |
  1529. ## Summary
  1530. | Metric | Count |
  1531. |---|---|
  1532. | Sections checked | $CHECKS_RUN |
  1533. | Errors | $ERRORS |
  1534. | Warnings | $WARNINGS |
  1535. | Suggestions | $INFO_COUNT |
  1536. ## Token Analysis
  1537. | File | Tokens | Lines |
  1538. |---|---|---|
  1539. | SKILL.md | ~$_S_BODY_TOKENS | $_S_BODY_LINES |
  1540. | Reference files | $_S_REFS_TOTAL files | $_S_REFS_LINKED linked |
  1541. ENDMD
  1542. # Findings
  1543. local total_findings
  1544. total_findings=$(wc -l < "$FINDINGS_FILE" | tr -d ' ')
  1545. if [ "$total_findings" -gt 0 ]; then
  1546. echo ""
  1547. echo "## Findings"
  1548. echo ""
  1549. local has_errors has_warnings has_info
  1550. has_errors=$(grep -c "^ERROR|" "$FINDINGS_FILE" 2>/dev/null || true)
  1551. has_warnings=$(grep -c "^WARN|" "$FINDINGS_FILE" 2>/dev/null || true)
  1552. has_info=$(grep -c "^INFO|" "$FINDINGS_FILE" 2>/dev/null || true)
  1553. if [ "${has_errors:-0}" -gt 0 ]; then
  1554. echo "### Errors"
  1555. echo ""
  1556. grep "^ERROR|" "$FINDINGS_FILE" | while IFS='|' read -r _ sec msg; do
  1557. echo "- **$sec**: $msg"
  1558. done
  1559. echo ""
  1560. fi
  1561. if [ "${has_warnings:-0}" -gt 0 ]; then
  1562. echo "### Warnings"
  1563. echo ""
  1564. grep "^WARN|" "$FINDINGS_FILE" | while IFS='|' read -r _ sec msg; do
  1565. echo "- **$sec**: $msg"
  1566. done
  1567. echo ""
  1568. fi
  1569. if [ "${has_info:-0}" -gt 0 ]; then
  1570. echo "### Suggestions"
  1571. echo ""
  1572. grep "^INFO|" "$FINDINGS_FILE" | while IFS='|' read -r _ sec msg; do
  1573. echo "- **$sec**: $msg"
  1574. done
  1575. echo ""
  1576. fi
  1577. fi
  1578. echo "---"
  1579. echo ""
  1580. echo "_Validated against [agentskills.io/specification](https://agentskills.io/specification) | Scanner v4.0 | $(date '+%Y-%m-%d %H:%M:%S')_"
  1581. }
  1582. print_score_only() {
  1583. if [ "$NO_COLOR" = "1" ]; then
  1584. echo "$QUALITY_SCORE/100 $QUALITY_GRADE"
  1585. else
  1586. echo "$QUALITY_SCORE/100 $(grade_color "$QUALITY_SCORE")"
  1587. fi
  1588. [ "$ERRORS" -gt 0 ] && exit 1
  1589. exit 0
  1590. }
  1591. # ════════════════════════════════════════════════════════════════════
  1592. # REPORT (terminal)
  1593. # ════════════════════════════════════════════════════════════════════
  1594. print_report() {
  1595. echo ""
  1596. echo ""
  1597. local elapsed=$(( SECONDS - SCAN_START ))
  1598. _pad() {
  1599. local text="$1" width="$2"
  1600. local visible
  1601. visible=$(echo "$text" | sed 's/\x1b\[[0-9;]*m//g')
  1602. local pad_len=$(( width - ${#visible} ))
  1603. if [ "$pad_len" -lt 0 ]; then pad_len=0; fi
  1604. printf '%s%*s' "$text" "$pad_len" ""
  1605. }
  1606. local W=58
  1607. local line
  1608. line=$(printf '%.0s═' $(seq 1 $W))
  1609. local thin_line
  1610. thin_line=$(printf '%.0s─' $(seq 1 $W))
  1611. # ── Summary box ──
  1612. echo " $(_bold "╔${line}╗")"
  1613. echo " $(_bold "║")$(_pad "" $W)$(_bold "║")"
  1614. echo " $(_bold "║")$(_pad " AGENT SKILL SCAN REPORT " $W)$(_bold "║")"
  1615. echo " $(_bold "║")$(_pad " Scanner v4.0 " $W)$(_bold "║")"
  1616. echo " $(_bold "║")$(_pad "" $W)$(_bold "║")"
  1617. echo " $(_bold "╠${line}╣")"
  1618. echo " ║$(_pad "" $W)║"
  1619. echo " ║ $(_pad "Checks run : $CHECKS_RUN sections" $(( W - 2 )))║"
  1620. echo " ║ $(_pad "$(_red "Errors") : $ERRORS" $(( W - 2 )))║"
  1621. echo " ║ $(_pad "$(_yellow "Warnings") : $WARNINGS" $(( W - 2 )))║"
  1622. echo " ║ $(_pad "$(_blue "Info") : $INFO_COUNT" $(( W - 2 )))║"
  1623. echo " ║ $(_pad "Duration : ${elapsed}s" $(( W - 2 )))║"
  1624. echo " ║$(_pad "" $W)║"
  1625. echo " $(_bold "╠${line}╣")"
  1626. echo " ║$(_pad "" $W)║"
  1627. local result_text
  1628. if [ "$ERRORS" -gt 0 ]; then
  1629. result_text=$(_bold_red "FAIL")
  1630. elif [ "$WARNINGS" -gt 0 ]; then
  1631. result_text=$(_bold_yellow "PASS with warnings")
  1632. else
  1633. result_text=$(_bold_green "ALL CLEAR")
  1634. fi
  1635. echo " ║ $(_pad "Result : $result_text" $(( W - 2 )))║"
  1636. echo " ║ $(_pad "Quality Score : $QUALITY_SCORE/100 ($(grade_color "$QUALITY_SCORE"))" $(( W - 2 )))║"
  1637. echo " ║$(_pad "" $W)║"
  1638. echo " $(_bold "╠${line}╣")"
  1639. echo " ║$(_pad "" $W)║"
  1640. echo " ║ $(_pad "Description : $QUALITY_DESC/30" $(( W - 2 )))║"
  1641. echo " ║ $(_pad "Clarity : $QUALITY_CLARITY/25" $(( W - 2 )))║"
  1642. echo " ║ $(_pad "Spec : $QUALITY_SPEC/20" $(( W - 2 )))║"
  1643. echo " ║ $(_pad "Progressive : $QUALITY_PROGRESSIVE/15" $(( W - 2 )))║"
  1644. echo " ║ $(_pad "Security : $QUALITY_SECURITY/10" $(( W - 2 )))║"
  1645. echo " ║$(_pad "" $W)║"
  1646. echo " $(_bold "╚${line}╝")"
  1647. # ── Detailed findings ──
  1648. local total_findings
  1649. total_findings=$(wc -l < "$FINDINGS_FILE" | tr -d ' ')
  1650. if [ "$total_findings" -gt 0 ]; then
  1651. echo ""
  1652. echo " $(_bold "┌${thin_line}┐")"
  1653. echo " $(_bold "│")$(_pad " DETAILED FINDINGS " $W)$(_bold "│")"
  1654. echo " $(_bold "└${thin_line}┘")"
  1655. _print_findings_by_severity() {
  1656. local severity="$1" label="$2" color_fn="$3"
  1657. local matches
  1658. matches=$(grep "^${severity}|" "$FINDINGS_FILE" 2>/dev/null || true)
  1659. [ -z "$matches" ] && return
  1660. local count
  1661. count=$(echo "$matches" | wc -l | tr -d ' ')
  1662. echo ""
  1663. echo " $($color_fn "$label ($count)")"
  1664. echo " $(_dim "$(printf '%.0s─' $(seq 1 56))")"
  1665. local prev_section=""
  1666. while IFS='|' read -r _ sec msg; do
  1667. if [ "$sec" != "$prev_section" ]; then
  1668. echo ""
  1669. echo " $(_dim "[$sec]")"
  1670. prev_section="$sec"
  1671. fi
  1672. echo " $($color_fn "▸") $msg"
  1673. done <<< "$matches"
  1674. }
  1675. _print_findings_by_severity "ERROR" "ERRORS" "_red"
  1676. _print_findings_by_severity "WARN" "WARNINGS" "_yellow"
  1677. _print_findings_by_severity "INFO" "SUGGESTIONS" "_blue"
  1678. echo ""
  1679. echo " $(_dim "$(printf '%.0s─' $(seq 1 60))")"
  1680. else
  1681. echo ""
  1682. echo " $(_bold_green "No findings — skill package is in perfect shape.")"
  1683. fi
  1684. echo ""
  1685. echo " $(_dim "Validated against agentskills.io/specification")"
  1686. echo " $(_dim "Scored using skill-tools 5-dimension rubric (0-100)")"
  1687. echo ""
  1688. # ── CI job summary ──
  1689. if [ "$CI" = "true" ]; then
  1690. {
  1691. echo "## Agent Skill Scan Report"
  1692. echo ""
  1693. if [ "$ERRORS" -gt 0 ]; then
  1694. echo "**FAIL** — $ERRORS error(s), $WARNINGS warning(s)"
  1695. elif [ "$WARNINGS" -gt 0 ]; then
  1696. echo "**PASS with $WARNINGS warning(s)**"
  1697. else
  1698. echo "**All checks passed**"
  1699. fi
  1700. echo ""
  1701. echo "**Quality Score: $QUALITY_SCORE/100 (Grade: $QUALITY_GRADE)**"
  1702. echo ""
  1703. echo "| Metric | Count |"
  1704. echo "|--------|-------|"
  1705. echo "| Checks | $CHECKS_RUN |"
  1706. echo "| Errors | $ERRORS |"
  1707. echo "| Warnings | $WARNINGS |"
  1708. echo "| Info | $INFO_COUNT |"
  1709. echo "| Duration | ${elapsed}s |"
  1710. echo ""
  1711. echo "| Dimension | Score |"
  1712. echo "|-----------|-------|"
  1713. echo "| Description Quality | $QUALITY_DESC/30 |"
  1714. echo "| Instruction Clarity | $QUALITY_CLARITY/25 |"
  1715. echo "| Spec Compliance | $QUALITY_SPEC/20 |"
  1716. echo "| Progressive Disclosure | $QUALITY_PROGRESSIVE/15 |"
  1717. echo "| Security | $QUALITY_SECURITY/10 |"
  1718. echo "| **Total** | **$QUALITY_SCORE/100 ($QUALITY_GRADE)** |"
  1719. if [ "$total_findings" -gt 0 ]; then
  1720. echo ""
  1721. echo "### Findings"
  1722. echo ""
  1723. echo "<details>"
  1724. echo "<summary>Click to expand ($total_findings findings)</summary>"
  1725. echo ""
  1726. _ci_findings() {
  1727. local severity="$1" icon="$2"
  1728. local matches
  1729. matches=$(grep "^${severity}|" "$FINDINGS_FILE" 2>/dev/null || true)
  1730. [ -z "$matches" ] && return
  1731. while IFS='|' read -r _ sec msg; do
  1732. echo "- ${icon} **${sec}**: ${msg}"
  1733. done <<< "$matches"
  1734. }
  1735. _ci_findings "ERROR" "x"
  1736. _ci_findings "WARN" "!"
  1737. _ci_findings "INFO" "i"
  1738. echo ""
  1739. echo "</details>"
  1740. fi
  1741. echo ""
  1742. echo "_Validated against [agentskills.io/specification](https://agentskills.io/specification) | Score: $QUALITY_SCORE/100 ($QUALITY_GRADE)_"
  1743. } >> "${GITHUB_STEP_SUMMARY:-/dev/null}"
  1744. fi
  1745. }
  1746. # ════════════════════════════════════════════════════════════════════
  1747. # MAIN
  1748. # ════════════════════════════════════════════════════════════════════
  1749. show_help() {
  1750. echo ""
  1751. echo " $(_bold "Agent Skill Scanner v4")"
  1752. echo ""
  1753. echo " Validates skill packages against the agentskills.io specification"
  1754. echo " and community best practices for AI agent skills."
  1755. echo " Computes a quality score (0-100) across 5 dimensions."
  1756. echo ""
  1757. echo " $(_bold "Usage:")"
  1758. echo " ./scripts/validate-skill.sh Run full scan of skills/compose (terminal)"
  1759. echo " ./scripts/validate-skill.sh <skill-dir> Scan another skill, e.g. skills/compose-ui"
  1760. echo " ./scripts/validate-skill.sh --json Output as JSON"
  1761. echo " ./scripts/validate-skill.sh --md Output as Markdown report"
  1762. echo " ./scripts/validate-skill.sh --score-only Print score and grade only"
  1763. echo " ./scripts/validate-skill.sh --help Show this help"
  1764. echo ""
  1765. echo " $(_bold "Output Modes:")"
  1766. echo " $(_cyan "--json") Machine-readable JSON (pipe to jq, feed to web tools)"
  1767. echo " $(_cyan "--md") Markdown report (save to file, paste in PRs/docs)"
  1768. echo " $(_cyan "--score-only") Quick score check (e.g., in pre-commit hooks)"
  1769. echo " $(_dim "(default)") Rich terminal output with colors and bar charts"
  1770. echo ""
  1771. echo " $(_bold "Environment:")"
  1772. echo " CI=true Emit GitHub Actions annotations + job summary"
  1773. echo " NO_COLOR=1 Disable colored output"
  1774. echo ""
  1775. echo " $(_bold "Checks (15 sections, 20+ individual checks):")"
  1776. echo " $(_cyan " 1.") Skill structure Required/optional files and directories"
  1777. echo " $(_cyan " 2.") Frontmatter & desc name, description, voice, verbs, negative triggers"
  1778. echo " $(_cyan " 3.") Body & disclosure Line count, token budget, structure, code examples"
  1779. echo " $(_cyan " 4.") Internal links All links resolve to existing files"
  1780. echo " $(_cyan " 5.") Reference files Linked, orphaned, empty, and missing detection"
  1781. echo " $(_cyan " 6.") Markdown syntax Unclosed code blocks"
  1782. echo " $(_cyan " 7.") Reference nesting No deep cross-reference chains"
  1783. echo " $(_cyan " 8.") Scripts Executable permissions, documentation"
  1784. echo " $(_cyan " 9.") Repository hygiene Sensitive files, development artifacts"
  1785. echo " $(_cyan "10.") Token budget Per-file and total token analysis with zones"
  1786. echo " $(_cyan "11.") Content quality Code examples, headings, keyword density"
  1787. echo " $(_cyan "12.") Agent metadata agents/openai.yaml validation"
  1788. echo " $(_cyan "13.") Security scan API keys, secrets, hardcoded paths, dangerous commands"
  1789. echo " $(_cyan "14.") Heading hierarchy Level skip detection (MD001), duplicate headings (MD024)"
  1790. echo " $(_cyan "15.") Quality score 0-100 score across 5 dimensions with letter grade"
  1791. echo ""
  1792. echo " $(_bold "Quality Score Dimensions:")"
  1793. echo " Description Quality (30 pts) Length, specificity, voice, verbs, triggers"
  1794. echo " Instruction Clarity (25 pts) Code blocks, headings, reference links"
  1795. echo " Spec Compliance (20 pts) Required fields, name format, token/line limits"
  1796. echo " Progressive Discl. (15 pts) References dir, linking, body size"
  1797. echo " Security (10 pts) No secrets, paths, or dangerous commands"
  1798. echo ""
  1799. echo " $(_bold "Letter Grades:")"
  1800. echo " $(_bold_green "A+") 95+ $(_bold_green "A") 90+ $(_bold_green "A-") 85+ $(_bold_yellow "B+") 80+ $(_bold_yellow "B") 75+"
  1801. echo " $(_bold_yellow "B-") 70+ $(_bold_yellow "C+") 65+ $(_bold_red "C") 60+ $(_bold_red "C-") 55+ $(_bold_red "D") 50+ $(_bold_red "F") <50"
  1802. echo ""
  1803. echo " $(_bold "Token Zones:")"
  1804. echo " $(_green "[SAFE]") Well within budget, efficient"
  1805. echo " $(_yellow "[WARN]") At or near recommended limits"
  1806. echo " $(_red "[HIGH]") Exceeding recommendations, consider optimization"
  1807. echo ""
  1808. echo " $(_bold "Token Estimation:")"
  1809. echo " Uses chars/4 approximation (industry standard for tiktoken cl100k_base)"
  1810. echo " SKILL.md budget: <5000 tokens | Reference: <4000 tokens each"
  1811. echo " Total package: <50000 tokens recommended"
  1812. echo ""
  1813. echo " $(_bold "Severity:")"
  1814. echo " $(_red "✗ ERROR") Spec violation or broken content — must fix"
  1815. echo " $(_yellow "! WARN ") Best practice issue — should fix"
  1816. echo " $(_blue "ℹ INFO ") Suggestion — nice to have"
  1817. echo " $(_green "✓ PASS ") Check passed"
  1818. echo ""
  1819. echo " $(_bold "References:")"
  1820. echo " Spec: https://agentskills.io/specification"
  1821. echo " Scoring: https://agentskills.io/skill-creation/best-practices"
  1822. echo " Tokens: https://agentpatterns.ai/context-engineering/context-budget-allocation/"
  1823. echo ""
  1824. }
  1825. _suppress_terminal() {
  1826. [ "$OUTPUT_MODE" != "terminal" ]
  1827. }
  1828. main() {
  1829. # Parse CLI flags
  1830. while [ $# -gt 0 ]; do
  1831. case "$1" in
  1832. --help|-h) show_help; exit 0 ;;
  1833. --json) OUTPUT_MODE="json"; NO_COLOR=1 ;;
  1834. --md|--markdown) OUTPUT_MODE="markdown"; NO_COLOR=1 ;;
  1835. --score-only|--score) OUTPUT_MODE="score-only" ;;
  1836. --quick) QUICK_MODE=true ;;
  1837. -*) echo "Unknown flag: $1"; echo "Run with --help for usage."; exit 2 ;;
  1838. *)
  1839. if [ -n "$SKILL_PATH" ]; then echo "Only one skill path per run."; exit 2; fi
  1840. SKILL_PATH="$1" ;;
  1841. esac
  1842. shift
  1843. done
  1844. if ! cd "${SKILL_PATH:-$ROOT/skills/compose}" 2>/dev/null; then
  1845. echo "Not a directory: $SKILL_PATH"; exit 2
  1846. fi
  1847. SKILL_DIR="$(basename "$(pwd)")"
  1848. if ! _suppress_terminal; then
  1849. echo ""
  1850. echo " $(_bold "╔══════════════════════════════════════════════════════════╗")"
  1851. echo " $(_bold "║") $(_bold "║")"
  1852. echo " $(_bold "║") $(_bold_cyan "A G E N T S K I L L S C A N N E R") $(_bold "║")"
  1853. echo " $(_bold "║") $(_dim "v 4 . 0") $(_bold "║")"
  1854. echo " $(_bold "║") $(_bold "║")"
  1855. echo " $(_bold "╚══════════════════════════════════════════════════════════╝")"
  1856. echo ""
  1857. if [ "$CI" = "true" ]; then
  1858. _detail "Mode : CI (GitHub Actions annotations + job summary)"
  1859. else
  1860. _detail "Mode : Local"
  1861. fi
  1862. _detail "Skill: $(pwd)"
  1863. _detail "Spec : agentskills.io/specification"
  1864. _detail "Date : $(date '+%Y-%m-%d %H:%M:%S')"
  1865. echo ""
  1866. echo " $(_dim "Running $TOTAL_CHECKS sections with 20+ individual checks...")"
  1867. fi
  1868. if [ "$OUTPUT_MODE" != "terminal" ]; then
  1869. exec 3>&1 1>/dev/null
  1870. fi
  1871. check_structure
  1872. check_frontmatter
  1873. check_body
  1874. check_links
  1875. if [ "$QUICK_MODE" = false ]; then
  1876. check_references
  1877. check_reference_depth
  1878. check_token_budget
  1879. check_content_quality
  1880. check_heading_hierarchy
  1881. fi
  1882. check_markdown
  1883. check_scripts
  1884. check_repo_hygiene
  1885. check_agents_metadata
  1886. check_security
  1887. compute_quality_score
  1888. if [ "$OUTPUT_MODE" != "terminal" ]; then
  1889. exec 1>&3 3>&-
  1890. fi
  1891. case "$OUTPUT_MODE" in
  1892. json) print_json ;;
  1893. markdown) print_markdown ;;
  1894. score-only) print_score_only ;;
  1895. *) print_report ;;
  1896. esac
  1897. [ "$ERRORS" -gt 0 ] && exit 1
  1898. exit 0
  1899. }
  1900. main "$@"

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.