Skip to content
Back to search

yao-meta-skill

SkillWorks with Claude CodeGitHub

Create, improve, or evaluate an existing skill from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install/trust checks, 优化已有 skill, 补 trigger 评测. Exclude summary/translation/docs, brainstorming, one-off prompts, copy-only edits, outline-only requests.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/yaojingang--yao-meta-skill/badge.svg)](/listings/yaojingang--yao-meta-skill)

HTML

<a href="/listings/yaojingang--yao-meta-skill"><img src="/api/listings/yaojingang--yao-meta-skill/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Install the skill globally for Codex first:

npx -y skills add yaojingang/yao-meta-skill -a codex -g -y

To install it for every supported agent, replace `-a codex` with `-a '*'`:

npx -y skills add yaojingang/yao-meta-skill -a '*' -g -y

After installation, restart the client. Then ask for tasks such as "create a skill from this workflow", "improve this existing skill", "evaluate this skill", or "add evals to this skill" to trigger `yao-meta-skill`.

WorkBuddy

The generic Skill package supports WorkBuddy's documented Skill format. Version 2.1.1 adds the bilingual display fields, author, and version required by the WorkBuddy open platform. The existing CLI, Skill IR, and packaging targets remain the shared implementation.

Build the existing generic ZIP with `python3 scripts/yao.py package . --platform generic --output-dir dist --zip --self`, then import `dist/yao-meta-skill.zip` through WorkBuddy's **Add Skill → Upload Skill** flow. See [WorkBuddy compatibility](references/workbuddy.md) for workspace paths, Python prerequisites, creating WorkBuddy skills, and update handling.

Run `make workbuddy-compat-check` to check the actual ZIP metadata, bundled resources, isolated CLI execution, and explicit output directory. Native WorkBuddy import and conversation routing remain client acceptance checks.

Update notifications and one-confirmation update

Each activation can run a cached update preflight. It checks the official GitHub `VERSION` at most once every 24 hours and shows each newer stable version once. Network failures leave the active Skill task unchanged.

python3 scripts/yao.py check-update --notice --self
python3 scripts/yao.py check-update --force --self
python3 scripts/yao.py self-update --self
python3 scripts/yao.py self-update --self --yes

`self-update` recognizes verified Agent Skills CLI and installed Codex plugin channels. The command displays its plan first and requires `--yes` before changing an installation. Development checkouts, unmanaged copies, non-official sources, and ambiguous multi-channel installs stay read-only. Restart Codex or the active AI client after a successful update. See [Update Delivery](references/update-delivery.md) for the channel and recovery contract.

1. Describe the workflow, prompt set, or repeated task you want to turn into a skill. 2. Start with a short, human intent dialogue so the real job, outputs, exclusions, constraints, and standards are explicit. 3. Let `quickstart` clarify intent first, then run silent benchmark scan and reference synthesis; it only surfaces explicit questions when intent is still unclear or when there is a real design conflict. 4. Use the archetype-aware `quickstart` or the full authoring flow to generate or improve the package in scaffold, production, library, or governed mode. 5. Review the generated `reports/skill-interpretation.html` first for the bilingual interpretation report. It defaults to Simplified Chinese and provides an English switch in the top right. Then open `reports/skill-overview.html` for the audit scorecard and `reports/review-studio.html` to inspect release blockers, permission approvals, and evidence paths in one page before adding more structure.

Target-specific CLI commands require an explicit Skill path. Commands that operate on Yao Meta Skill itself also require `--self`. Runtime update cache and opt-in CLI telemetry use user-level cache/state directories, so external Skill work leaves the Yao source and installed package unchanged.

Or use the unified authoring CLI:

python3 scripts/yao.py quickstart --output-dir .
python3 scripts/yao.py github-benchmark-scan my-skill --query "release workflow portability"
python3 scripts/yao.py reference-scan my-skill \
  --external-reference "World Class Method::method::Borrow a tight evaluation loop.::Do not copy heavy process." \

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

SKILL.md · Manifest · 60 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. ---
  2. name: yao-meta-skill
  3. display_name: Yao 技能工厂
  4. display_name_en: Yao Meta Skill
  5. description: Create, improve, or evaluate an existing skill from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install/trust checks, 优化已有 skill, 补 trigger 评测. Exclude summary/translation/docs, brainstorming, one-off prompts, copy-only edits, outline-only requests.
  6. description_zh: 将重复流程整理为可复用技能,支持创建、优化、评测、打包和安装检查。
  7. description_en: Create, improve, evaluate, and package reusable agent skills.
  8. version: "2.1.1"
  9. author: Yao Team
  10. metadata:
  11. author: Yao Team
  12. ---
  13. # Yao Meta Skill
  14. ## Router Rules
  15. - Route by frontmatter `description` using the lightest mode.
  16. - Keep `SKILL.md` lean: guidance in `references/`, logic in `scripts/`, evidence in `reports/`.
  17. ## Modes
  18. - `Scaffold`: exploratory; `Production`: team; `Library`: shared; `Governed`: release-critical.
  19. - Rules: [Method](references/skill-engineering-method.md), [Modes](references/operating-modes.md), [Boundaries](references/resource-boundaries.md).
  20. ## Compact Workflow
  21. 1. Require `repeated use` + `reusable output contract`; skip one-offs.
  22. 2. Lock target path + identity before writes; self-edits require `--self`.
  23. 3. Capture job, output, exclusions, constraints, standards, mode.
  24. 4. Scan `3-5` useful external/user/local references.
  25. 5. Write `description` early; route edits need `trigger_eval.py`; release gates follow risk.
  26. 6. Add deeper profiles only when earned.
  27. Playbooks: [Target Safety](references/target-safety.md), [Intent](references/intent-dialogue.md), [Skill IR](references/skill-ir-method.md), [Output Eval](references/output-eval-method.md), [Review Studio](references/review-studio-method.md).
  28. ## WorkBuddy
  29. Use [WorkBuddy](references/workbuddy.md) for import, runtime, output metadata and updates.
  30. ## Updates
  31. Activation: `scripts/yao.py check-update --notice --self`; show `notice_text` if requested; continue. WorkBuddy updates use ZIP re-import. For supported managed Agent Skills/Codex installs, “更新”: `scripts/yao.py self-update --self --yes`.
  32. ## Skill OS 2.0 Gates
  33. For production/library/governed releases, run Skill IR, compiler, trigger/output eval, Atlas, conformance, trust, package/install, upgrade, drift, waiver, Review Studio.
  34. ## Governed Package Boundary
  35. Governed/file-backed packages require `owner`, `review cadence`, `input_files` (`file-backed fixture`), `output contract`, `rollback boundary`, `trust report`, and `reports/output_quality_scorecard.md`. Preserve these labels; mark unavailable evidence as `missing evidence`; never fabricate evidence.
  36. ## First-Turn Style
  37. - Infer non-core gaps; ask one core-fork question per round, max two, then use recorded preferred inference.
  38. - In Chinese, sound soft and companion-like; use [Intent Dialogue](references/intent-dialogue.md).
  39. ## Output Contract
  40. Builds: `SKILL.md`, aligned interface, justified assets, boundary/gate summary. Audits: findings + fixes; edit only when asked. No-skill: no files.
README.md · README · 754 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Yao Meta Skill
  2. [![CI](https://github.com/yaojingang/yao-meta-skill/actions/workflows/test.yml/badge.svg)](https://github.com/yaojingang/yao-meta-skill/actions/workflows/test.yml)
  3. [![License: MIT](https://img.shields.io/badge/License-MIT-black.svg)](LICENSE)
  4. [![English](https://img.shields.io/badge/Docs-English-black)](README.md)
  5. [![中文](https://img.shields.io/badge/Docs-%E4%B8%AD%E6%96%87-red)](docs/README.zh-CN.md)
  6. [![日本語](https://img.shields.io/badge/Docs-%E6%97%A5%E6%9C%AC%E8%AA%9E-blue)](docs/README.ja-JP.md)
  7. [![Français](https://img.shields.io/badge/Docs-Fran%C3%A7ais-green)](docs/README.fr-FR.md)
  8. [![Русский](https://img.shields.io/badge/Docs-%D0%A0%D1%83%D1%81%D1%81%D0%BA%D0%B8%D0%B9-purple)](docs/README.ru-RU.md)
  9. `YAO` stands for `Yielding AI Outcomes`: the goal is not to generate more prompt text, but to produce reusable AI assets and real operational outcomes.
  10. `yao-meta-skill` creates, evaluates, packages, and governs reusable agent skills. The 1.0 line focused on turning repeated workflows into installable, readable, cross-platform skill packages. The 2.0 line expands that factory into a Skill OS: a governed system for modeling a skill once, compiling it for multiple targets, testing its behavior, reviewing its release evidence, and tracking the next iteration.
  11. [Quick Start](#quick-start) · [Skill OS 2.0](#skill-os-20-upgrade) · [1.0 vs 2.0](#from-10-to-20) · [Operator UX](#operator-ux-commands) · [Benchmark](#weighted-quality-benchmark) · [Examples](examples/README.md) · [Evals](evals/README.md) · [Failure Library](failures/README.md) · [Method Doctrine](#method-doctrine)
  12. ## Skill OS 2.0 Upgrade
  13. Skill OS 2.0 keeps the original promise of `yao-meta-skill`, but makes the package lifecycle more explicit. Instead of stopping at `SKILL.md`, it adds a semantic contract, target compilers, evaluation evidence, release gates, and operation reports around the skill.
  14. - **Skill IR**: a platform-neutral intermediate representation for intent, triggers, inputs, outputs, boundaries, references, and expected artifacts.
  15. - **Target compilers and adapters**: generated surfaces for OpenAI, Claude, generic agent skills, Agent Skills compatible packages, and VS Code-oriented workflows.
  16. - **Output Eval Lab**: trigger checks, output assertions, execution evidence, timing and token evidence, benchmark reproducibility, blind-review packs, answer keys, and adjudication reports.
  17. - **Review Studio 2.0**: a single HTML gate page for intent, triggers, output eval, context cost, runtime checks, trust, Skill Atlas signals, adoption drift, waivers, annotations, release evidence, warnings, blockers, and fix actions.
  18. - **Evidence and release governance**: evidence consistency checks, package verification, install simulation, runtime permission probes, world-class evidence intake, world-class ledger, operator runbook, and public claim guard.
  19. - **SkillOps loop**: metadata-only adoption drift, telemetry hooks, adaptive proposals, daily and weekly curator reports, and portfolio-level drift signals.
  20. Current posture: the repository is ready for beta and external testing, while stronger public "world-class" claims remain evidence-gated. Provider-backed production evidence, human blind-review evidence, native permission execution, and real-client telemetry are tracked as separate evidence tasks instead of being treated as completed work.
  21. See the companion artifacts:
  22. - [Visual 1.0 vs 2.0 comparison report](.previews/yao-meta-skill-2-comparison/index.html)
  23. - [Chinese desktop preview](.previews/yao-meta-skill-2-comparison/yao-meta-skill-1-vs-2.png)
  24. - [English desktop preview](.previews/yao-meta-skill-2-comparison/yao-meta-skill-1-vs-2-en.png)
  25. ## From 1.0 to 2.0
  26. | Dimension | 1.0 focus | 2.0 upgrade |
  27. | --- | --- | --- |
  28. | Product role | Create, refactor, evaluate, and package reusable skills. | Govern the full lifecycle of a skill: creation, compilation, evaluation, review, release, telemetry, and iteration. |
  29. | Architecture | `SKILL.md`, `agents/interface.yaml`, manifest files, and report artifacts. | Skill IR, target compilers, adapters, gate contracts, evidence ledgers, release locks, and action-oriented review pages. |
  30. | Cross-platform delivery | OpenAI, Claude, and generic package targets. | Adds broader Agent Skills and VS Code-oriented compatibility, with registry-readable compatibility records. |
  31. | Quality model | Trigger and structure checks plus report-based review. | Output eval, benchmark reproducibility, execution evidence, failure disclosure, blind-review packs, and evidence consistency checks. |
  32. | Report experience | Overview HTML and first-pass review pages. | Bilingual Skill Overview v2, Review Studio 2.0, reviewer annotations, action cards, charts, and audit-oriented report contracts. |
  33. | Release boundary | Package output with basic validation. | Package verification, install simulation, runtime permission probes, release locks, public claim guard, and operator runbooks. |
  34. | Operating loop | Manual feedback and local iteration. | Adoption drift, metadata telemetry, SkillOps reports, adaptive proposals, and portfolio-level drift detection. |
  35. ## 2.0 Use Cases
  36. - **Create a new skill from repeated work**: start with a workflow note, prompt set, transcript, runbook, or document pattern, then generate a package with a lean entrypoint, explicit inputs and outputs, references, reports, and the lightest justified gates.
  37. - **Upgrade a personal skill into a team asset**: add interface contracts, manifests, target adapters, trust checks, output evals, reviewer waivers, release notes, and Review Studio evidence before other people depend on the skill.
  38. - **Prepare a skill for beta release**: run package verification, install simulation, compatibility checks, runtime permission probes, and evidence consistency checks, then separate beta readiness from stronger public claims.
  39. - **Keep a skill useful after release**: use metadata-only telemetry, adoption drift, feedback logs, SkillOps reports, and adaptive proposals to decide whether the next move should be documentation, an eval, a skill patch, or a governance update.
  40. - **Compare with other meta-skill approaches**: keep Anthropic/OpenAI-style conversational creation and lean instruction writing where they fit, then use `yao-meta-skill` when the package needs evidence, portability, release gates, and repeatable maintenance.
  41. ## Operator UX Commands
  42. These read-only helper commands turn common maintainer questions into repeatable diagnostics:
  43. ```bash
  44. python3 scripts/yao.py install-status --expected-source .
  45. python3 scripts/yao.py localized-doc-sync-check
  46. python3 scripts/yao.py pr-review-report 4 --repo yaojingang/yao-meta-skill
  47. ```
  48. - `install-status` explains whether the active skill is coming from `.codex/skills`, `.agents/skills`, or the disabled mirror, and flags duplicate active installs.
  49. - `localized-doc-sync-check` verifies that the Chinese README carries the public homepage sections that were added to the English README.
  50. - `pr-review-report` reads GitHub PR metadata, changed files, status checks, and suggested local commands without merging or mutating the PR.
  51. ## Capability Surface
  52. It turns rough workflows, transcripts, prompts, notes, and runbooks into reusable skill packages with:
  53. - a clear trigger surface
  54. - a lean `SKILL.md`
  55. - optional references, scripts, and evals
  56. - an inference-first intent dialogue that asks one personalized question only for core task, output, or direction forks, stops after two rounds, and records structured assumptions without retaining a full transcript
  57. - a silent-by-default GitHub benchmark scan plus reference synthesis that studies top public repositories and world-class pattern tracks, then surfaces only real conflicts or uncertainty to the user
  58. - a generated visual HTML overview for each newly initialized skill
  59. - a Review Studio 2.0 HTML gate page that combines intent, trigger, output eval, context, runtime, trust, atlas, adoption drift, reviewer waivers, reviewer annotations, release evidence, and per-warning fix actions
  60. - a Skill OS 2.0 audit that maps each world-class requirement to current evidence, human-required gaps, and external-required gaps
  61. - a Skill OS 2.0 blueprint coverage report that maps the upgrade plan's core modules and recommended PRs to concrete artifacts, commands, and tests
  62. - a world-class evidence plan that turns remaining provider, human, native-permission, and real-client telemetry gaps into executable evidence tasks
  63. - a world-class evidence ledger that records which external and human evidence is accepted or still pending without treating planned work as proof
  64. - a world-class evidence intake contract that validates external and human evidence packets for provenance, privacy, artifact refs, and anti-overclaim rules before ledger review
  65. - a redacted world-class preflight report that checks local files, environment readiness, human/external prerequisites, and source blockers before operators collect evidence
  66. - a world-class submission review queue that compares evidence packets, intake validation, source artifacts, and ledger state without accepting evidence
  67. - a world-class operator runbook that gives reviewers the exact commands, artifacts, and collection checklist needed to close remaining evidence gaps
  68. - a world-class claim guard that scans public claim surfaces and blocks premature completed/true claims while the evidence ledger still has pending external or human evidence
  69. - a benchmark reproducibility manifest that checks methodology sections, required artifacts, failure disclosure, and reproduction commands
  70. - an evidence consistency gate that compares generated reports against each other so benchmark, overview, interpretation, adoption, world-class ledger, coverage, and Review Studio facts do not drift silently
  71. - Output Eval Lab evidence with assertion grading, execution/timing/token evidence, a blind A/B review pack, a separate answer key, and reviewer adjudication reports
  72. - a runtime permission probe report that checks packaged target adapters for explicit permission metadata, native-enforcement flags, metadata fallback notes, and residual risks
  73. - a Python compatibility gate that catches supported-runtime syntax hazards before they reach GitHub Actions or packaged distribution
  74. - a side-by-side HTML review studio for first-pass human review
  75. - an artifact design profile that defines visual direction, layout patterns, and quality gates for reports, tutorials, dashboards, screenshots, and review pages
  76. - a prompt quality profile that abstracts need modeling, RTF mapping, complexity, and quality checks into reviewer-visible evidence instead of bloating `SKILL.md`
  77. - a systems-thinking model that maps boundaries, feedback loops, drift risks, recurring failure patterns, and highest-leverage quality moves
  78. - three high-value next iteration directions after the first package is created
  79. - a lightweight feedback log that does not require a full promotion cycle
  80. - a local-first metadata-only adoption and drift report that turns real usage signals into next iteration candidates, with optional `yao.py` CLI run capture, external client event emit hooks, hook recipes, and JSONL import that record command names and outcomes without arguments or raw content
  81. - an explicit-source adaptive proposal loop that summarizes redacted repeated user preferences and generates approval-gated adaptation proposals without scanning private logs or writing source files
  82. - a SkillOps opportunity scorer and decision policy that ranks redacted repeated signals, maps them to report-only, AGENTS update, existing-skill patch, or eval-addition actions, and keeps every durable write approval-gated
  83. - a weekly SkillOps curator report that aggregates daily opportunities, Skill Atlas portfolio signals, release lock state, and world-class evidence gaps into a proposal-only maintenance queue
  84. - a Browser/Chrome Native Messaging telemetry host that can receive length-prefixed metadata-only client events and generate a local launcher plus manifest without storing raw content
  85. - a Skill Atlas drift layer that reads aggregate adoption reports and surfaces portfolio-level drift signals without packaging raw telemetry logs
  86. - a baseline compare report for with-skill vs baseline review
  87. - a conversation-style, archetype-aware quickstart that steers new packages toward scaffold, production, library, or governed fits
  88. - Skill IR as the platform-neutral semantic contract, plus compiler reports and client-specific adapters
  89. - Registry audit metadata with package version, owner, license, checksum, and compatibility matrix
  90. - governance, promotion, and portability checks built into the default flow
  91. ## Architecture
  92. Hero view: Skill OS 2.0 turns messy operational input into a governed, reusable skill package through a model, compile, evaluate, release, and operate loop.
  93. ```mermaid
  94. flowchart LR
  95. A["Inputs<br/>workflow / prompt / transcript / docs / notes"] --> B["Intent model<br/>job / outputs / exclusions / standards"]
  96. B --> C["Skill IR<br/>trigger / contracts / resources / evidence"]
  97. C --> D["Skill package<br/>SKILL.md / references / scripts / reports"]
  98. C --> E["Target compilers<br/>OpenAI / Claude / generic / Agent Skills / VS Code"]
  99. D --> F["Eval Lab<br/>trigger / output / benchmark / runtime"]
  100. E --> F
  101. F --> G["Review Studio<br/>gates / warnings / actions / waivers"]
  102. G --> H["Release boundary<br/>package verification / install simulation / claim guard"]
  103. H --> I["SkillOps loop<br/>feedback / adoption drift / next iteration"]
  104. I --> B
  105. ```
  106. Read it in 10 seconds:
  107. - **Inputs**: start from rough operational material instead of a polished spec.
  108. - **Intent model**: make the job, outputs, exclusions, constraints, and standards explicit before generating files.
  109. - **Skill IR**: keep the semantic contract separate from any single platform format.
  110. - **Package and compile**: generate the lean skill package and the target-specific adapters from the same source model.
  111. - **Evaluate and review**: turn trigger behavior, output quality, runtime checks, and trust signals into reviewable evidence.
  112. - **Release and operate**: publish only within the current evidence boundary, then feed adoption drift and reviewer feedback into the next iteration.
  113. ## Weighted Quality Benchmark
  114. This benchmark is a project-level engineering review, scored from `0-10` per dimension and weighted to `100`. GitHub stars are intentionally excluded because they measure ecosystem heat, not meta-skill engineering quality.
  115. The score is local engineering evidence, not a claim of world-class readiness. Public superiority claims still depend on accepted external and human evidence in the world-class ledger.
  116. Weighted score formula: `sum(score / 10 * weight)`.
  117. | Meta Skill | Method Depth 15 | Context Discipline 10 | Toolchain 15 | Eval/Test Rigor 20 | Governance 15 | Portability 10 | Onboarding/Review 5 | Local Reliability 10 | Weighted Score |
  118. | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
  119. | Yao Meta Skill | 9.5 | 8.0 | 9.5 | 9.5 | 9.5 | 9.0 | 6.5 | 9.5 | 91.5 |
  120. | Anthropic Skill Creator | 9.0 | 6.5 | 8.5 | 7.5 | 4.0 | 5.0 | 7.5 | 5.0 | 67.5 |
  121. | OpenAI Skill Creator | 8.5 | 9.5 | 5.0 | 2.0 | 3.0 | 4.0 | 8.5 | 4.0 | 50.5 |
  122. | Rank | Meta Skill | Score | Core Positioning |
  123. | ---: | --- | ---: | --- |
  124. | 1 | Yao Meta Skill | 91.5 | A complete engineering, evaluation, governance, and portability system for reusable skills. |
  125. | 2 | Anthropic Skill Creator | 67.5 | Strong methodology and iteration loop, with weaker local execution reliability and governance coverage. |
  126. | 3 | OpenAI Skill Creator | 50.5 | Best treated as a concise skill-writing method guide rather than a full engineering system. |
  127. ## Human Blind A/B Review Snapshot
  128. On 2026-06-29, a single human reviewer compared `yao-meta-skill` with the bundled OpenAI `skill-creator` across five realistic skill-creation scenarios: support triage, revenue reconciliation, webinar repurposing, incident postmortems, and PR review follow-up. The reviewer confirmed decisions were completed before the answer key was opened.
  129. Result: `yao-meta-skill` was selected in `5/5` cases.
  130. Evidence:
  131. - Review entrypoint: [reports/blind-human-review-2026-06-29/index.html](reports/blind-human-review-2026-06-29/index.html)
  132. - Adjudication summary: [reports/blind-human-review-2026-06-29/adjudication.md](reports/blind-human-review-2026-06-29/adjudication.md)
  133. - Recorded decisions: [reports/blind-human-review-2026-06-29/review-decisions.recorded.json](reports/blind-human-review-2026-06-29/review-decisions.recorded.json)
  134. Boundary: this is single-reviewer blind preference evidence. It is not provider-backed independent model execution evidence, and the per-case rationale fields are still empty.
  135. ## Best-Fit Scenarios
  136. - Choose **Yao Meta Skill** when the target is a reusable team asset with explicit boundaries, trigger evaluation, governance, packaging, portability, and local execution checks.
  137. - Choose **Anthropic Skill Creator** when the target is a conversation-first creation loop and the priority is human-guided iteration over repository-level governance.
  138. - Choose **OpenAI Skill Creator** when the target is a compact reference for writing lean skill instructions and keeping context small.
  139. - A practical hybrid pattern is still useful: draft conversationally, then use `yao-meta-skill` to harden the package, add evidence, and make it team-ready.
  140. ## Quick Start
  141. Install the skill globally for Codex first:
  142. ```bash
  143. npx -y skills add yaojingang/yao-meta-skill -a codex -g -y
  144. ```
  145. To install it for every supported agent, replace `-a codex` with `-a '*'`:
  146. ```bash
  147. npx -y skills add yaojingang/yao-meta-skill -a '*' -g -y
  148. ```
  149. After installation, restart the client. Then ask for tasks such as "create a skill from this workflow", "improve this existing skill", "evaluate this skill", or "add evals to this skill" to trigger `yao-meta-skill`.
  150. ### WorkBuddy
  151. The generic Skill package supports WorkBuddy's documented Skill format. Version 2.1.1 adds the bilingual display fields, author, and version required by the WorkBuddy open platform. The existing CLI, Skill IR, and packaging targets remain the shared implementation.
  152. Build the existing generic ZIP with `python3 scripts/yao.py package . --platform generic --output-dir dist --zip --self`, then import `dist/yao-meta-skill.zip` through WorkBuddy's **Add Skill → Upload Skill** flow. See [WorkBuddy compatibility](references/workbuddy.md) for workspace paths, Python prerequisites, creating WorkBuddy skills, and update handling.
  153. Run `make workbuddy-compat-check` to check the actual ZIP metadata, bundled resources, isolated CLI execution, and explicit output directory. Native WorkBuddy import and conversation routing remain client acceptance checks.
  154. ### Update notifications and one-confirmation update
  155. Each activation can run a cached update preflight. It checks the official GitHub `VERSION` at most once every 24 hours and shows each newer stable version once. Network failures leave the active Skill task unchanged.
  156. ```bash
  157. python3 scripts/yao.py check-update --notice --self
  158. python3 scripts/yao.py check-update --force --self
  159. python3 scripts/yao.py self-update --self
  160. python3 scripts/yao.py self-update --self --yes
  161. ```
  162. `self-update` recognizes verified Agent Skills CLI and installed Codex plugin channels. The command displays its plan first and requires `--yes` before changing an installation. Development checkouts, unmanaged copies, non-official sources, and ambiguous multi-channel installs stay read-only. Restart Codex or the active AI client after a successful update. See [Update Delivery](references/update-delivery.md) for the channel and recovery contract.
  163. 1. Describe the workflow, prompt set, or repeated task you want to turn into a skill.
  164. 2. Start with a short, human intent dialogue so the real job, outputs, exclusions, constraints, and standards are explicit.
  165. 3. Let `quickstart` clarify intent first, then run silent benchmark scan and reference synthesis; it only surfaces explicit questions when intent is still unclear or when there is a real design conflict.
  166. 4. Use the archetype-aware `quickstart` or the full authoring flow to generate or improve the package in scaffold, production, library, or governed mode.
  167. 5. Review the generated `reports/skill-interpretation.html` first for the bilingual interpretation report. It defaults to Simplified Chinese and provides an English switch in the top right. Then open `reports/skill-overview.html` for the audit scorecard and `reports/review-studio.html` to inspect release blockers, permission approvals, and evidence paths in one page before adding more structure.
  168. Target-specific CLI commands require an explicit Skill path. Commands that operate on Yao Meta Skill itself also require `--self`. Runtime update cache and opt-in CLI telemetry use user-level cache/state directories, so external Skill work leaves the Yao source and installed package unchanged.
  169. Or use the unified authoring CLI:
  170. ```bash
  171. python3 scripts/yao.py quickstart --output-dir .
  172. python3 scripts/yao.py github-benchmark-scan my-skill --query "release workflow portability"
  173. python3 scripts/yao.py reference-scan my-skill \
  174. --external-reference "World Class Method::method::Borrow a tight evaluation loop.::Do not copy heavy process." \
  175. --user-reference "A product or repo I admire::taste::Learn the clarity and operating standard.::Do not copy wording." \
  176. --local-constraint "Current Library Naming::structure::Keep naming aligned with the local skill library.::Do not inherit private references."
  177. python3 scripts/yao.py skill-interpretation my-skill
  178. python3 scripts/yao.py review-viewer my-skill
  179. python3 scripts/yao.py review-studio my-skill
  180. python3 scripts/yao.py artifact-design-profile my-skill
  181. python3 scripts/yao.py prompt-quality-profile my-skill
  182. python3 scripts/yao.py system-model my-skill
  183. python3 scripts/yao.py feedback my-skill --note "Tighten exclusions before adding scripts." --rating 4 --category boundary
  184. python3 scripts/yao.py adapt-scan my-skill --source ./curated-user-signals.jsonl
  185. python3 scripts/yao.py adapt-propose my-skill
  186. python3 scripts/yao.py daily-skillops my-skill --source ./curated-user-signals.jsonl
  187. python3 scripts/yao.py weekly-curator my-skill
  188. python3 scripts/yao.py adoption-drift my-skill --record-event skill_activation --activation-type explicit --outcome accepted
  189. YAO_CLI_TELEMETRY=1 python3 scripts/yao.py validate my-skill
  190. python3 scripts/yao.py telemetry-emit my-skill --event skill_activation --activation-type explicit --outcome accepted --command browser-extension
  191. python3 scripts/yao.py telemetry-hooks my-skill
  192. python3 scripts/telemetry_native_host.py my-skill --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/
  193. python3 scripts/yao.py telemetry-import my-skill --input-jsonl /tmp/external-client-events.jsonl --command browser-extension
  194. python3 scripts/yao.py review-waivers my-skill --add-waiver --gate-key trust-report --reviewer "Yao Team" --reason "Known warning accepted for this release with bounded follow-up." --expires-at 2026-09-30
  195. python3 scripts/yao.py review-waivers my-skill --add-waiver --gate-key permission-gates --reviewer "Yao Team" --reason "Permission warning accepted only for this non-governed release window." --expires-at 2026-09-30
  196. python3 scripts/yao.py review-annotations my-skill --add-annotation --gate-key output-lab --target-path reports/output_quality_scorecard.md --line 1 --body "Clarify recorded fixture vs model-executed evidence before release."
  197. python3 scripts/yao.py baseline-compare --self
  198. python3 scripts/yao.py check-update --notice --self
  199. python3 scripts/yao.py self-update --self
  200. python3 scripts/yao.py skill-ir . --output-json skill-ir/examples/yao-meta-skill.json --self
  201. python3 scripts/yao.py compile-skill . --target openai --target claude --target generic --target vscode --self
  202. python3 scripts/yao.py package . --platform generic --output-dir dist --self
  203. python3 scripts/yao.py output-eval --self
  204. python3 scripts/yao.py output-exec --self
  205. python3 scripts/yao.py output-review --self
  206. python3 scripts/yao.py conformance . --self
  207. python3 scripts/yao.py trust . --self
  208. python3 scripts/yao.py python-compat . --self
  209. python3 scripts/yao.py runtime-permissions . --package-dir dist --self
  210. python3 scripts/yao.py skill-atlas --workspace-root . --self
  211. python3 scripts/yao.py registry-audit . --self
  212. python3 scripts/yao.py package-verify . --package-dir dist --require-zip --self
  213. python3 scripts/yao.py install-simulate . --package-dir dist --self
  214. python3 scripts/yao.py upgrade-check . --previous-package-json registry/examples/yao-meta-skill-1.0.0.json --self
  215. python3 scripts/yao.py world-class-evidence . --self
  216. SUBMISSIONS_DIR="${SUBMISSIONS_DIR:-evidence/world_class/submissions}"
  217. python3 scripts/yao.py world-class-preflight . --submissions-dir "$SUBMISSIONS_DIR" --self
  218. python3 scripts/yao.py world-class-submission-kit . --output-dir "$SUBMISSIONS_DIR" --self
  219. # Alternative: prefill artifact SHA-256 digests while keeping drafts template-only.
  220. python3 scripts/yao.py world-class-submission-kit . --output-dir "$SUBMISSIONS_DIR" --prefill-artifacts --self
  221. python3 scripts/yao.py world-class-intake . --submissions-dir "$SUBMISSIONS_DIR" --self
  222. python3 scripts/yao.py world-class-submission-review . --submissions-dir "$SUBMISSIONS_DIR" --self
  223. python3 scripts/yao.py world-class-ledger . --submissions-dir "$SUBMISSIONS_DIR" --self
  224. python3 scripts/yao.py world-class-runbook . --submissions-dir "$SUBMISSIONS_DIR" --self
  225. python3 scripts/yao.py world-class-claim-guard . --self
  226. python3 scripts/yao.py benchmark-reproducibility . --self
  227. python3 scripts/yao.py evidence-consistency . --self
  228. ```
  229. ## Local Development Source
  230. Development source: this repository is the source of truth for authoring and review.
  231. Use Python 3.11 or newer for local development. GitHub Actions runs the test suite on Python 3.11, and the Makefile checks the active interpreter before running `make test` or `make ci-test`.
  232. ```bash
  233. python3.11 -m venv .venv
  234. source .venv/bin/activate
  235. python -m pip install --upgrade pip
  236. python -m pip install --requirement requirements-ci.txt
  237. make ci-test
  238. ```
  239. If `python3` points to an older system interpreter, pass the interpreter explicitly:
  240. ```bash
  241. make PYTHON=python3.11 ci-test
  242. ```
  243. Disabled mirror: `~/.agents/skills.disabled/yao-meta-skill` is the local backup mirror for this source. Keeping the mirror outside `~/.agents/skills` prevents Codex from showing a duplicate `Yao Meta Skill` while this repository is also visible in the active workspace.
  244. Sync the current source into the disabled mirror:
  245. ```bash
  246. make sync-local-install
  247. ```
  248. The sync command first rebuilds the package and runs install preflight against `dist/yao-meta-skill.zip`. It refuses to sync when package extraction, adapter readability, or installer permission enforcement fails. After the preflight passes, it copies Git-tracked files plus new source files in code and guidance directories such as `scripts/`, `tests/`, `references/`, and `docs/`. It skips untracked business-skill folders and untracked private reports by default, so local experiments do not leak into the mirror.
  249. Restore an active global Codex install only when you intentionally want this skill discoverable outside the development workspace:
  250. ```bash
  251. make sync-active-install
  252. ```
  253. That active install writes to `~/.agents/skills/yao-meta-skill` and can make Codex show a second `Yao Meta Skill` entry while this repository is open as a skills workspace.
  254. ## Generated Artifact Boundaries
  255. Keep this repository focused on the meta-skill factory.
  256. - Put reusable factory examples in `examples/`.
  257. - Name embedded example entrypoints `SKILL.example.md`; reserve the exact `SKILL.md` filename for the installable root skill so recursive agent discovery does not activate examples or test fixtures.
  258. - Put reusable benchmark evidence, regression results, and release evidence in `reports/`.
  259. - Keep private analysis reports, customer-specific outputs, and one-off generated business skills outside this repository unless they are intentionally promoted into an example or regression fixture.
  260. - Place real generated skills as sibling skill directories under the local skill workspace, not as top-level folders inside `yao-meta-skill`.
  261. ## 5-Minute Workflow
  262. 1. Start from a raw workflow note.
  263. 2. Turn it into a skill package with `SKILL.md`, `agents/interface.yaml`, and only the folders the workflow actually needs.
  264. 3. Validate the trigger description with `evals/trigger_cases.json`.
  265. 4. Export compatibility artifacts for the clients you care about.
  266. 5. Compare the result against the examples in `examples/`.
  267. Minimum commands:
  268. ```bash
  269. python3 scripts/trigger_eval.py --description-file evals/improved_description.txt --cases evals/trigger_cases.json
  270. python3 scripts/run_description_optimization_suite.py
  271. python3 scripts/judge_blind_eval.py --description-file SKILL.md --cases evals/blind_holdout/trigger_cases.json --semantic-config evals/semantic_config.json
  272. python3 scripts/context_sizer.py .
  273. python3 scripts/resource_boundary_check.py .
  274. python3 scripts/governance_check.py . --require-manifest
  275. python3 scripts/compile_skill.py .
  276. python3 scripts/cross_packager.py . --platform openai --platform claude --platform generic --platform vscode --expectations evals/packaging_expectations.json --zip
  277. python3 scripts/probe_runtime_permissions.py . --package-dir dist
  278. python3 tests/verify_packager_failures.py
  279. ```
  280. Or run everything together:
  281. ```bash
  282. make test
  283. ```
  284. Unified authoring flow:
  285. ```bash
  286. python3 scripts/yao.py init my-skill --description "Describe what the skill does."
  287. python3 scripts/yao.py validate my-skill
  288. python3 scripts/yao.py workspace-flow --target root --label first-pass --self
  289. python3 scripts/yao.py review-viewer my-skill
  290. python3 scripts/yao.py review --target root --self
  291. python3 scripts/yao.py release-snapshot --target root --label release-candidate --self
  292. python3 scripts/yao.py skill-ir . --output-json skill-ir/examples/yao-meta-skill.json --self
  293. python3 scripts/yao.py compile-skill . --self
  294. python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip --self
  295. python3 scripts/yao.py runtime-permissions . --package-dir dist --self
  296. python3 scripts/yao.py package-verify . --package-dir dist --require-zip --self
  297. python3 scripts/yao.py test --self
  298. ```
  299. ## Results
  300. The homepage panel below is generated from the current eval suite so the family-level outcome is visible without opening raw JSON.
  301. <!-- BEGIN:EVAL_RESULTS -->
  302. - regression corpus: `66` prompts across `21` families
  303. - aggregate result: `0` false positives, `0` false negatives, average precision `1.0`, average recall `1.0`
  304. - suite status:
  305. | Suite | Cases | FP | FN | Precision | Recall |
  306. | --- | ---: | ---: | ---: | ---: | ---: |
  307. | train | 31 | 0 | 0 | 1.0 | 1.0 |
  308. | dev | 22 | 0 | 0 | 1.0 | 1.0 |
  309. | holdout | 13 | 0 | 0 | 1.0 | 1.0 |
  310. | Family | Cases | Pass Rate |
  311. | --- | ---: | ---: |
  312. | `brainstorm_only` | 2 | 1.0 |
  313. | `brainstorm_vs_build` | 1 | 1.0 |
  314. | `complex_multi_asset` | 3 | 1.0 |
  315. | `document_export_vs_agent_skill` | 4 | 1.0 |
  316. | `document_only` | 3 | 1.0 |
  317. | `explain_not_package` | 1 | 1.0 |
  318. | `explain_only` | 5 | 1.0 |
  319. | `future_outline_vs_build` | 4 | 1.0 |
  320. | `iterate_existing_skill` | 5 | 1.0 |
  321. | `long_context_document_only` | 3 | 1.0 |
  322. | `long_context_near_neighbor` | 3 | 1.0 |
  323. | `long_context_summary_only` | 2 | 1.0 |
  324. | `long_context_trigger` | 4 | 1.0 |
  325. | `meta_skill_creation` | 1 | 1.0 |
  326. | `one_off_vs_reusable` | 2 | 1.0 |
  327. | `package_for_team` | 2 | 1.0 |
  328. | `paraphrase_trigger` | 5 | 1.0 |
  329. | `partial_scaffold_not_full_skill` | 4 | 1.0 |
  330. | `summary_only` | 3 | 1.0 |
  331. | `translate_only` | 4 | 1.0 |
  332. | `workflow_to_skill` | 5 | 1.0 |
  333. Full reports: [reports/eval_suite.json](reports/eval_suite.json) and [reports/family_summary.md](reports/family_summary.md)
  334. <!-- END:EVAL_RESULTS -->
  335. - packaging validation: `openai`, `claude`, `generic`, and `vscode` targets pass contract checks and carry IR provenance, semantic parity metadata, and target-native behavior contracts
  336. - target compiler validation: `openai`, `claude`, `generic`, Agent Skills compatible, and VS Code / Copilot contracts are compiled from Skill IR with generated-file mappings, adapter modes, native surfaces, permission enforcement notes, and unsupported-feature notes
  337. - runtime permission probes: `openai`, `claude`, `generic`, and `vscode` adapters expose explicit permission contracts; current targets report `0` native-enforcement adapters and `4` metadata fallbacks with residual risks visible to reviewers
  338. - portability score: `100/100` with neutral activation, execution, trust, and degradation metadata preserved across all exported targets
  339. - description optimization suite: root, team frontend review, and governed incident command pass blind and adversarial holdout gates; governed incident command still carries one visible holdout miss, and adversarial calibration plus family drift are now tracked separately
  340. - judge-backed blind eval: root, team frontend review, and governed incident command now pass an independent rubric judge on blind holdout prompts
  341. - human blind A/B snapshot: a single reviewer selected `yao-meta-skill` over the bundled OpenAI `skill-creator` in `5/5` realistic skill-creation scenarios; evidence is published in [reports/blind-human-review-2026-06-29/adjudication.md](reports/blind-human-review-2026-06-29/adjudication.md)
  342. - packaging failure fixtures: invalid metadata, invalid YAML, and unsupported targets fail as expected
  343. - failure library regressions: anti-pattern families pass automated checks
  344. - governance and resource-boundary checks are part of the default test path
  345. - root governance maturity score: `90/100`; governed benchmark example: `95/100`
  346. - CJK-aware trigger matching is now covered by explicit Chinese build, packaging, eval, and near-neighbor cases
  347. - context budgets: root `944/1000`, complex benchmark `790/1000`, governed benchmark `760/1000`
  348. - quality density: root `137.7`, complex benchmark `164.6`, governed benchmark `171.1`
  349. - regression milestones are tracked in [reports/regression_history.md](reports/regression_history.md)
  350. - description drift history is tracked in [reports/description_drift_history.md](reports/description_drift_history.md)
  351. - route confusion is tracked in [reports/route_scorecard.md](reports/route_scorecard.md)
  352. - promotion evidence is summarized in [reports/iteration_ledger.md](reports/iteration_ledger.md)
  353. - promotion decisions are published in [reports/promotion_decisions.md](reports/promotion_decisions.md)
  354. - candidate lifecycle states are published in [reports/candidate_registry.md](reports/candidate_registry.md)
  355. - lightweight with-skill vs baseline comparison is published in [reports/baseline-compare.md](reports/baseline-compare.md)
  356. - Review Studio 2.0 gate evidence is published in [reports/review-studio.html](reports/review-studio.html)
  357. - Review Studio fix actions are embedded in [reports/review-studio.json](reports/review-studio.json)
  358. - Skill OS 2.0 blueprint coverage is published in [reports/skill_os2_coverage.md](reports/skill_os2_coverage.md)
  359. - reviewer waiver evidence is published in [reports/review_waivers.md](reports/review_waivers.md)
  360. - remaining world-class evidence tasks are published in [reports/world_class_evidence_plan.md](reports/world_class_evidence_plan.md)
  361. - current world-class evidence acceptance state is published in [reports/world_class_evidence_ledger.md](reports/world_class_evidence_ledger.md)
  362. - world-class evidence intake readiness is published in [reports/world_class_evidence_intake.md](reports/world_class_evidence_intake.md)
  363. - world-class submission review queue is published in [reports/world_class_submission_review.md](reports/world_class_submission_review.md)
  364. - world-class operator runbook is published in [reports/world_class_operator_runbook.md](reports/world_class_operator_runbook.md) and [reports/world_class_operator_runbook.html](reports/world_class_operator_runbook.html)
  365. - world-class public claim guard status is published in [reports/world_class_claim_guard.md](reports/world_class_claim_guard.md)
  366. - benchmark reproducibility evidence is published in [reports/benchmark_reproducibility.md](reports/benchmark_reproducibility.md)
  367. - cross-report evidence consistency is published in [reports/evidence_consistency.md](reports/evidence_consistency.md)
  368. - target compiler evidence is published in [reports/compiled_targets.md](reports/compiled_targets.md)
  369. - Python runtime compatibility evidence is published in [reports/python_compatibility.md](reports/python_compatibility.md)
  370. - registry package metadata and audit status are published in [reports/registry_audit.md](reports/registry_audit.md)
  371. - package archive verification is published in [reports/package_verification.md](reports/package_verification.md)
  372. - temporary local install simulation is published in [reports/install_simulation.md](reports/install_simulation.md)
  373. - upgrade diff, version-bump recommendation, and release-note evidence are published in [reports/upgrade_check.md](reports/upgrade_check.md)
  374. - local-first adoption and drift telemetry is summarized in [reports/adoption_drift_report.md](reports/adoption_drift_report.md)
  375. - context budget summaries are tracked in [reports/context_budget.md](reports/context_budget.md)
  376. - portability status is tracked in [reports/portability_score.md](reports/portability_score.md)
  377. ## Current Strengths
  378. The latest weighted review puts Yao at `91.5/100`. The strongest dimensions are the ones that matter most when skills become long-lived team assets:
  379. - **Method depth `9.5`**: formal skill engineering doctrine, archetypes, gate selection, non-skill decisions, lifecycle governance, and resource boundaries.
  380. - **Toolchain completeness `9.5`**: authoring, validation, benchmark scan, description optimization, report generation, promotion checks, packaging, CI, and portability checks are wired into one operational flow.
  381. - **Eval and test rigor `9.5`**: trigger quality is checked with train/dev/holdout, blind holdout, adversarial holdout, judge-backed blind eval, route confusion, drift history, and promotion gates.
  382. - **Governance and lifecycle `9.5`**: important skills can carry owner, lifecycle state, review cadence, maturity score, trust boundaries, promotion decisions, and regression history.
  383. - **Local execution reliability `9.5`**: the repository is executable locally through `make test`, `make ci-test`, and the unified `scripts/yao.py` authoring CLI.
  384. - **Portability and distribution `9.0`**: neutral source metadata, client adapters, degradation rules, packaging contracts, and portability scoring preserve reusable semantics across target environments.
  385. - **Systems stability**: generated skills now include a system model that turns boundary discipline, feedback loops, drift watch, and leverage-point analysis into reviewer-visible evidence.
  386. - **Context discipline `8.0`**: the entrypoint is still held under budget, but this is tracked as a live constraint because the system now carries more reports, examples, benchmark assets, and generated evidence.
  387. - **Onboarding and review experience `6.5`**: quickstart, HTML overview, side-by-side review viewer, and feedback logs have improved the first-run experience, but this remains the clearest UX improvement area.
  388. The current direction is deliberate: keep the entrypoint light, make evaluation hard to fake, make governance visible, and continue reducing the friction of first-time creation and review.
  389. ## Why Yao
  390. - **Lightweight**: the entrypoint stays compact, context budgets are explicit, and extra structure is added only when it pays for itself.
  391. - **Rigorous**: trigger quality is checked with family regressions, blind holdout, adversarial holdout, route confusion, judge-backed blind eval, and promotion gates.
  392. - **Governed**: important skills are treated as maintainable assets with lifecycle state, maturity expectations, ownership, and review cadence.
  393. - **Portable**: source metadata stays neutral while adapters, degradation rules, and packaging contracts preserve reusable semantics across environments.
  394. ## What It Does
  395. This project helps you create, refactor, evaluate, and package skills as durable capability bundles rather than one-off prompts.
  396. The design logic is simple:
  397. 1. Capture the real recurring job behind the user's request.
  398. 2. Set a clean skill boundary so one package does one coherent job.
  399. 3. Optimize the trigger description before over-writing the body.
  400. 4. Keep the main skill file small and move details into references or scripts.
  401. 5. Add quality gates only when they pay for themselves.
  402. 6. Export compatibility artifacts only for the clients you actually need.
  403. ## Method Doctrine
  404. The repository now treats method as a first-class asset instead of scattered guidance.
  405. - [Skill Engineering Method](references/skill-engineering-method.md)
  406. - [Intent Dialogue](references/intent-dialogue.md)
  407. - [Reference Scan Strategy](references/reference-scan.md)
  408. - [Pattern Extraction Doctrine](references/pattern-extraction-doctrine.md)
  409. - [Output Quality Risk](references/output-quality-risk.md)
  410. - [Authoring Discipline](references/authoring-discipline.md)
  411. - [Skill Archetypes](references/skill-archetypes.md)
  412. - [Gate Selection](references/gate-selection.md)
  413. - [Iteration Philosophy](references/iteration-philosophy.md)
  414. - [Non-Skill Decision Tree](references/non-skill-decision-tree.md)
  415. - [Regression Cause Taxonomy](references/regression-cause-taxonomy.md)
  416. - [Human Review Template](references/human-review-template.md)
  417. ## Why It Exists
  418. Most teams keep valuable operating knowledge scattered across chats, personal prompts, oral habits, and undocumented workflows. This project converts that hidden process knowledge into:
  419. - discoverable skill packages
  420. - repeatable execution flows
  421. - lower-context instructions
  422. - reusable team assets
  423. - compatibility-ready distributions
  424. ## Repository Structure
  425. ```text
  426. yao-meta-skill/
  427. ├── SKILL.md
  428. ├── README.md
  429. ├── VERSION
  430. ├── LICENSE
  431. ├── .gitignore
  432. ├── agents/
  433. │ └── interface.yaml
  434. ├── evals/
  435. ├── examples/
  436. ├── references/
  437. ├── scripts/
  438. └── templates/
  439. ```
  440. ## Core Components
  441. ### `SKILL.md`
  442. The main skill entrypoint. It defines the trigger surface, operating modes, compact workflow, and output contract.
  443. ### `agents/interface.yaml`
  444. The neutral metadata source of truth. It stores display and compatibility metadata without locking the source tree to one vendor-specific path.
  445. ### `references/`
  446. Long-form material that should not bloat the main skill file. This includes design rules, evaluation guidance, compatibility strategy, and quality rubrics.
  447. ### `scripts/`
  448. Utility scripts that make the meta-skill operational:
  449. - `trigger_eval.py`: evaluates trigger descriptions with semantic intent concepts, explicit exclusions, and near-neighbor prompts
  450. - `run_eval_suite.py`: runs train/dev/holdout trigger suites, reports family-level regressions, and fails if aggregate regressions appear
  451. - `optimize_description.py`: generates candidate descriptions, scores them on dev, visible holdout, blind holdout, and adversarial holdout suites, then reports calibration and family health
  452. - `judge_blind_eval.py`: applies an independent rubric judge to blind-holdout prompts so blind acceptance is not backed only by the main threshold scorer
  453. - `run_description_optimization_suite.py`: runs description optimization across the root skill and governed examples, then writes reusable reports and optional drift snapshots with calibration and family summaries
  454. - `promotion_checker.py`: applies promotion policy to current description candidates, writes promotion decisions, builds candidate registries, and emits iteration bundles with review stubs
  455. - `create_iteration_snapshot.py`: freezes the current promotion decision into a versioned release snapshot with review, route, and context evidence
  456. - `yao.py`: unified authoring CLI that exposes init, validate, optimize-description, promote-check, python-compat, review, release-snapshot, workspace-flow, report, skill-report, skill-interpretation, skill-ir, compile-skill, output-exec, output-review, skill-os2-audit, skill-os2-coverage, world-class-evidence, world-class-ledger, world-class-intake, world-class-preflight, world-class-submission-kit, world-class-submission-review, world-class-runbook, world-class-claim-guard, benchmark-reproducibility, evidence-consistency, adapt-scan, adapt-propose, adapt-apply, daily-skillops, weekly-curator, telemetry-emit, telemetry-hooks, telemetry-import, package, registry-audit, package-verify, install-simulate, upgrade-check, review-waivers, and test as one entrypoint
  457. - `render_description_drift_history.py`: turns description-optimization snapshots into a readable drift-history report
  458. - `build_confusion_matrix.py`: scores route confusion across tracked sibling skills and `no_route` cases, then writes a route scorecard and optional milestone snapshot
  459. - `render_iteration_ledger.py`: compresses regression milestones, description optimization drift, and route scorecards into one iteration-facing ledger
  460. - `context_sizer.py`: estimates context weight and warns when the initial load gets too large
  461. - `resource_boundary_check.py`: audits whether detail is split across `SKILL.md`, `references/`, `scripts/`, `assets/`, and `evals/` appropriately
  462. - `governance_check.py`: validates owner, review cadence, lifecycle stage, and maturity metadata
  463. - `render_context_reports.py`: generates root and example context-budget reports plus a shared context summary
  464. - `render_regression_history.py`: turns milestone snapshots into a readable regression history report
  465. - `render_skill_os2_audit.py`: renders a requirement-by-requirement Skill OS 2.0 audit that separates landed local evidence from human-required and external-required gaps
  466. - `render_skill_os2_coverage.py`: maps the Skill OS 2.0 upgrade blueprint to local artifacts, commands, tests, and remaining evidence boundaries
  467. - `render_daily_skillops_report.py`: renders an explicit-source Daily SkillOps operations report that summarizes redacted user patterns, proposal-only adaptations, approval state, release evidence, and world-class evidence gaps without scanning private logs or applying patches
  468. - `render_weekly_curator_report.py`: renders a weekly SkillOps curator report from generated daily reports, Skill Atlas, benchmark lock, evidence consistency, and world-class ledger state without scanning private logs or applying patches
  469. - `skillops_opportunity.py`: scores redacted SkillOps opportunities and maps them to approval-gated action types such as report-only, AGENTS update, existing-skill patch, or eval addition
  470. - `render_world_class_evidence_plan.py`: renders executable evidence tasks for remaining world-class gaps without treating planned external work as completed evidence
  471. - `render_world_class_evidence_ledger.py`: renders a machine-checkable ledger for current world-class evidence acceptance, anti-overclaim guards, provenance requirements, and privacy contracts
  472. - `render_world_class_evidence_intake.py`: validates world-class external and human evidence packets against provenance, privacy, artifact, and anti-overclaim requirements before ledger review
  473. - `render_world_class_preflight.py`: renders redacted collection preflight checks for pending provider, human, native-permission, and native-client evidence without accepting evidence
  474. - `render_world_class_submission_review.py`: renders a read-only queue that compares submissions, intake validation, source evidence, and ledger state without accepting evidence
  475. - `render_world_class_operator_runbook.py`: renders an operator-facing checklist and command map for collecting pending world-class evidence without accepting evidence
  476. - `render_world_class_claim_guard.py`: scans README, docs, and reports for premature world-class completion claims while accepted evidence is still pending
  477. - `render_benchmark_reproducibility.py`: renders methodology, artifact, failure-disclosure, and reproduction-command evidence for public benchmark claims
  478. - `render_evidence_consistency.py`: compares generated report facts across benchmark reproducibility, overview, interpretation, adoption drift, world-class ledger, coverage, and Review Studio artifacts
  479. - `python_compat_check.py`: checks Python source for supported-runtime compatibility hazards such as Python 3.11 f-string expression backslashes
  480. - `cross_packager.py`: builds client-specific export artifacts from Skill IR plus neutral metadata, with explicit platform contracts and validation
  481. - `render_portability_report.py`: scores cross-environment portability from neutral metadata, degradation rules, and consumer validation coverage
  482. - `render_skill_overview.py`: generates the white-background bilingual HTML skill audit report with sticky four-character Chinese navigation, top-right language switch, v2 scorecard, inline SVG charts, contract boundary, quality review, risk governance, assets, and iteration roadmap
  483. - `render_skill_interpretation.py`: renders `reports/skill-interpretation.html/json` as the first-class post-creation interpretation report while reusing the Skill Overview v2 model and Kami white layout
  484. - `export_skill_ir.py`: exports the 2.0 platform-neutral Skill IR contract from `SKILL.md`, manifest, interface metadata, evals, resources, and reports
  485. - `compile_skill.py`: compiles Skill IR into target-specific semantic contracts, generated-file maps, adapter modes, target-native behavior contracts, preserved semantics, warnings, and unsupported-feature notes
  486. - `run_output_eval.py`: runs the Output Eval Lab v0 with static with-skill vs baseline assertion grading, blind A/B review pack generation, and separate answer key artifacts
  487. - `run_output_execution.py`: records output-eval execution evidence, distinguishing recorded fixtures, command runners, and provider-backed model runs with timing and token metadata
  488. - `local_output_eval_runner.py`: deterministic local runner for command-executed output-eval smoke evidence without claiming provider-backed model generation
  489. - `adjudicate_output_review.py`: records reviewer choices for blind A/B output evals, compares them with the answer key, and renders pending, match, disagreement, and invalid-decision audit reports
  490. - `render_review_annotations.py`: records reviewer annotations tied to Review Studio gates, source/report paths, and optional line numbers, with open blocker annotations reflected in Review Studio decisions
  491. - `run_conformance_suite.py`: verifies runtime conformance for OpenAI, Claude, Agent Skills, VS Code/Copilot-style, and generic targets
  492. - `trust_check.py`: generates the trust/security report for scripts, dependencies, secret risk, bounded network host policy, execution-level `--help` smoke checks, permission inputs, trust metadata, and stable source-contract integrity
  493. - `build_skill_atlas.py`: builds the Skill Atlas catalog, route-overlap matrix, dependency graph, stale report, owner gaps, aggregate drift signals, and HTML overview for a multi-skill workspace
  494. - `registry_audit.py`: builds registry package metadata and audits version, owner, license, checksum, Skill IR source, and compatibility matrix
  495. - `verify_package.py`: verifies generated package manifests, target adapters, zip archive safety, archive checksum, and registry parity
  496. - `simulate_install.py`: extracts a generated zip into a temporary skill root and verifies entrypoint, manifest, interface, reports, and adapters can be loaded
  497. - `upgrade_check.py`: compares current and previous registry package metadata, recommends a version bump, and blocks incompatible upgrade claims
  498. - `render_adoption_drift_report.py`: records metadata-only local telemetry and renders adoption, missed-trigger, bad-output, script-error, and review-drift signals without packaging raw event logs
  499. - `import_telemetry_events.py`: imports external metadata-only telemetry JSONL after whole-file privacy validation, then refreshes the aggregate adoption drift report
  500. - `emit_telemetry_event.py`: emits one metadata-only external client event into a local spool for later `telemetry-import`, with dry-run validation and raw-content field blocking
  501. - `render_telemetry_hook_recipes.py`: renders Browser, Chrome, VS Code, CLI wrapper, and provider-adapter telemetry hook recipes with dry-run commands and explicit native-integration caveats
  502. - `telemetry_native_host.py`: receives Browser/Chrome Native Messaging length-prefixed JSON events, rejects raw-content fields, appends metadata-only events, and writes local launcher/manifest files for operator installation
  503. - `yao_cli_telemetry.py`: opt-in metadata-only `yao.py` run capture for command name, source, outcome, and failure class without command arguments or raw content
  504. - `render_review_waivers.py`: validates human reviewer risk approvals with gate keys, reasons, expiry dates, and blocker-safe waiver policy
  505. - `init_skill.py`, `lint_skill.py`, `validate_skill.py`, `diff_eval.py`: minimal authoring toolchain
  506. - `check_update.py`: checks GitHub for a newer `VERSION` or remote manifest version, stores its cache in the user cache directory, and leaves Skill source/install directories unchanged
  507. - `render_output_risk_profile.py`: predicts output-specific failure modes such as generic headings, citation clutter, screenshot mistakes, weak Markdown tables, and missing execution assumptions
  508. ### `evals/`
  509. Reusable trigger and packaging checks, including baseline and improved descriptions for comparison plus the root semantic configuration that drives description optimization.
  510. This directory also contains route confusion fixtures and promotion policy rules for deciding when a route is promotable.
  511. ### `examples/`
  512. End-to-end examples showing raw workflow input, design summary, final generated skill shape, and targeted description-optimization packs where route wording is tuned against example-specific dev and holdout cases.
  513. ### `.github/workflows/test.yml`
  514. Continuous integration entrypoint that runs the full local regression suite on push and pull request.
  515. ## Validation Notes
  516. - Trigger evaluation now uses a local semantic-intent model with explicit positive concepts, exclusion concepts, and boundary-case reporting.
  517. - The sample trigger report now covers a larger positive, negative, and near-neighbor set rather than a tiny demo set.
  518. - Train/dev/holdout trigger suites now separate iterative tuning from final verification.
  519. - Description optimization now uses dev for ranking, visible holdout for non-regression, blind holdout for acceptance, and adversarial holdout for harder route-collision checks without feeding the ranking loop.
  520. - Judge-backed blind eval now adds a rubric-based second opinion for blind prompts, so blind acceptance is not decided by one scorer alone.
  521. - Description drift history now records adversarial calibration gaps and family coverage, so routing changes can be judged on confidence and family stability rather than raw error counts alone.
  522. - Route confusion is now tracked explicitly across the root meta-skill, frontend review skill, governed incident skill, and `no_route` cases, so route theft is visible instead of implicit.
  523. - Promotion policy now requires visible holdout, blind holdout, adversarial holdout, and route confusion to stay clean before a description should be considered promotable.
  524. - Promotion checking now emits explicit decisions, candidate lifecycle states, iteration bundles, and human-review stubs rather than leaving promotion as a prose-only step.
  525. - Promotion decisions now distinguish “no candidate beat current” from “current still has residual route risk,” so iteration can be audited without forcing every issue into a false block.
  526. - Packaging validation now uses explicit contracts and YAML parsing, but it is still a lightweight local validation layer rather than a full platform integration suite.
  527. - `evals/failure-cases.md` captures known weak spots that should remain part of regression checks.
  528. - `failures/` captures reusable anti-pattern writeups and machine-runnable failure cases for routing, packaging, and authoring failures.
  529. - `tests/verify_packager_failures.py` checks that invalid metadata, invalid YAML, and unsupported targets fail clearly.
  530. - Governance metadata and resource-boundary rules now have runnable checks instead of staying as prose only.
  531. - Governance checks now emit a maturity score so governed assets can be compared instead of only pass/fail checked.
  532. - Description optimization drift history is now versioned separately from the main trigger regression history so routing improvements are visible over time.
  533. - Iteration evidence now records why a candidate was kept, blocked, or promotable via a shared regression-cause taxonomy and bundle artifacts.
  534. - Declared maturity tiers are checked against recommended minimum governance scores, so `production`, `library`, and `governed` assets can be compared without forcing every strong example into the same label.
  535. - Context budgets are now tiered and explicit, so a governed skill can still choose a stricter `production`-sized initial-load budget.
  536. - Resource-boundary checks now detect decorative directories and compute a local quality-density signal instead of only checking raw token counts.
  537. ### `templates/`
  538. Starter templates for simple and more advanced skill packages.
  539. ## How To Use
  540. ### 1. Use the skill directly
  541. Invoke `yao-meta-skill` when you want to:
  542. - create a new skill
  543. - improve an existing skill
  544. - add evals to a skill
  545. - convert a workflow into a reusable package
  546. - prepare a skill for wider team adoption
  547. ### 2. Generate a new skill package
  548. The typical flow is:
  549. 1. describe the workflow or capability
  550. 2. identify trigger phrases and outputs
  551. 3. choose scaffold, production, or library mode
  552. 4. generate the package
  553. 5. run the sizing and trigger checks if needed
  554. 6. export target-specific compatibility artifacts from the Skill IR contract
  555. ### 3. Export compatibility artifacts
  556. Examples:
  557. ```bash
  558. python3 scripts/export_skill_ir.py ./yao-meta-skill --output-json ./yao-meta-skill/reports/skill-ir.json
  559. python3 scripts/compile_skill.py ./yao-meta-skill --target openai --target claude --target generic
  560. python3 scripts/cross_packager.py ./yao-meta-skill --platform openai --platform claude --expectations evals/packaging_expectations.json --zip
  561. python3 scripts/context_sizer.py ./yao-meta-skill
  562. python3 scripts/resource_boundary_check.py ./yao-meta-skill
  563. python3 scripts/governance_check.py ./yao-meta-skill --require-manifest
  564. python3 scripts/trigger_eval.py --description-file evals/improved_description.txt --cases evals/trigger_cases.json --baseline-description-file evals/baseline_description.txt
  565. ```
  566. ## Advantages
  567. - **Method-first, not prompt-first**: skill creation is treated as a formal engineering workflow with archetypes, gate selection, and non-skill decisions.
  568. - **Trigger-aware by design**: descriptions are optimized with route confusion, blind holdout, adversarial families, and promotion policy instead of one-shot intuition.
  569. - **Lightweight at the entrypoint**: `SKILL.md` stays compact while references, scripts, and evals are only added when they pay for themselves.
  570. - **Toolchain-backed**: initialization, validation, optimization, reporting, packaging, and testing are available through one unified CLI and CI path.
  571. - **Governed as an asset**: important skills can carry ownership, lifecycle state, maturity expectations, and review cadence.
  572. - **Portable by default**: source metadata stays neutral while adapters and degradation rules preserve compatibility across target environments.
  573. - **Evidence-rich**: route scorecards, regression history, context budgets, portability scores, and promotion decisions are published as artifacts instead of hidden implementation detail.
  574. ## Best Fit
  575. This project is best for:
  576. - agent builders
  577. - internal tooling teams
  578. - prompt engineers moving toward structured skills
  579. - organizations building reusable skill libraries
  580. ## Documentation
  581. | Language | Entry |
  582. | --- | --- |
  583. | English | [README.md](README.md) |
  584. | 中文 | [docs/README.zh-CN.md](docs/README.zh-CN.md) |
  585. | 日本語 | [docs/README.ja-JP.md](docs/README.ja-JP.md) |
  586. | Français | [docs/README.fr-FR.md](docs/README.fr-FR.md) |
  587. | Русский | [docs/README.ru-RU.md](docs/README.ru-RU.md) |
  588. ## Examples And Evals
  589. - Examples: [examples/README.md](examples/README.md)
  590. - Evals: [evals/README.md](evals/README.md)
  591. - Failure library: [failures/README.md](failures/README.md)
  592. - Failure regression check: [verify_failure_regressions.py](tests/verify_failure_regressions.py)
  593. - Regression history: [reports/regression_history.md](reports/regression_history.md)
  594. - Root governance score: [reports/governance_score.json](reports/governance_score.json)
  595. - Packaging contracts: [references/packaging-contracts.md](references/packaging-contracts.md)
  596. - Governance model: [references/governance.md](references/governance.md)
  597. - Resource boundary spec: [references/resource-boundaries.md](references/resource-boundaries.md)
  598. - Platform capability matrix: [references/platform-capability-matrix.md](references/platform-capability-matrix.md)
  599. - Failure fixtures: [tests/fixtures](tests/fixtures)
  600. - Adapter snapshots: [tests/snapshots](tests/snapshots)
  601. - Evolution example: [examples/evolution-frontend-review/README.md](examples/evolution-frontend-review/README.md)
  602. - Governed example: [examples/governed-incident-command/design-summary.md](examples/governed-incident-command/design-summary.md)
  603. - Governed example score: [examples/governed-incident-command/generated-skill/reports/governance_score.json](examples/governed-incident-command/generated-skill/reports/governance_score.json)
  604. ## License
  605. MIT. See [LICENSE](LICENSE).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.