Skip to content
Back to search

agent-native-apps

Agent-Native app-backed skills that bundle instructions, MCP connectors, and UI surfaces.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/BuilderIO/agent-native with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/BuilderIO/agent-native'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/builderio--agent-native/badge.svg)](/listings/builderio--agent-native)

HTML

<a href="/listings/builderio--agent-native"><img src="/api/listings/builderio--agent-native/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

npx --yes @agent-native/core@latest create my-agent --standalone --template chat

Follow the [getting started guide](https://agent-native.com/docs/getting-started?utm_source=github&utm_medium=referral&utm_content=readme) for a full intro to the framework.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 67 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "agent-native-apps",
  3. "description": "Agent-Native app-backed skills that bundle instructions, MCP connectors, and UI surfaces.",
  4. "owner": { "name": "Agent-Native" },
  5. "plugins": [
  6. {
  7. "name": "agent-native",
  8. "displayName": "Agent-Native",
  9. "description": "Open and operate granted Agent-Native apps through Dispatch MCP, with inline app surfaces and current browser screen state.",
  10. "source": "./.agents/plugins/agent-native",
  11. "autoUpdate": true,
  12. "homepage": "https://dispatch.agent-native.com/?utm_source=claude-plugins&utm_medium=referral&utm_content=marketplace-homepage",
  13. "keywords": [
  14. "agent-native",
  15. "agent-native",
  16. "mcp",
  17. "skills",
  18. "app-backed-skill"
  19. ]
  20. },
  21. {
  22. "name": "agent-native-visual-plans",
  23. "displayName": "Agent-Native Plan",
  24. "description": "Create rich interactive visual plans, recaps, and repo-native visual docs with diagrams, file maps, annotated code and diffs, API/schema summaries, feedback, and HTML export.",
  25. "source": "./.agents/plugins/agent-native-visual-plans",
  26. "autoUpdate": true,
  27. "homepage": "https://plan.agent-native.com/?utm_source=claude-plugins&utm_medium=referral&utm_content=marketplace-homepage",
  28. "keywords": [
  29. "agent-native",
  30. "visual-plans",
  31. "mcp",
  32. "skills",
  33. "app-backed-skill"
  34. ]
  35. },
  36. {
  37. "name": "agent-native-design",
  38. "displayName": "Design",
  39. "description": "Explore, compare, iterate, and export interactive UI design prototypes from the Design app.",
  40. "source": "./.agents/plugins/agent-native-design",
  41. "autoUpdate": true,
  42. "homepage": "https://design.agent-native.com/?utm_source=claude-plugins&utm_medium=referral&utm_content=marketplace-homepage",
  43. "keywords": [
  44. "agent-native",
  45. "design",
  46. "mcp",
  47. "skills",
  48. "app-backed-skill"
  49. ]
  50. },
  51. {
  52. "name": "agent-native-turn-into-app",
  53. "displayName": "Turn Into App",
  54. "description": "Turn a thread, skill, spreadsheet, or Claude/ChatGPT project into a visual Agent-Native app. Local code agents build, run, and screenshot-review it; Claude and ChatGPT on the web hand a bounded source brief to Builder through Dispatch.",
  55. "source": "./.agents/plugins/agent-native-turn-into-app",
  56. "autoUpdate": true,
  57. "homepage": "https://dispatch.agent-native.com/?utm_source=claude-plugins&utm_medium=referral&utm_content=marketplace-homepage",
  58. "keywords": [
  59. "agent-native",
  60. "turn-into-app",
  61. "mcp",
  62. "skills",
  63. "app-backed-skill"
  64. ]
  65. }
  66. ]
  67. }
README.md · README · 299 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <picture>
  2. <source
  3. media="(prefers-color-scheme: dark)"
  4. srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F5476444e54ee4a958966c90caca468e3"
  5. />
  6. <source
  7. media="(prefers-color-scheme: light)"
  8. srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F7628600bc10a4940b78f42c5df7628b0"
  9. />
  10. <img
  11. alt="Agent-Native: The agentic application framework"
  12. src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F7628600bc10a4940b78f42c5df7628b0"
  13. />
  14. </picture>
  15. # Agent-Native
  16. Agent-Native is an open-source TypeScript framework for building agents that pair autonomous work with a purpose-built UI. Define each capability once as an [action](https://agent-native.com/docs/actions-overview?utm_source=github&utm_medium=referral&utm_content=readme): the agent uses it as a tool, and the UI calls it from code.
  17. ## Quick start
  18. ```bash
  19. npx --yes @agent-native/core@latest create my-agent --standalone --template chat
  20. ```
  21. Follow the [getting started guide](https://agent-native.com/docs/getting-started?utm_source=github&utm_medium=referral&utm_content=readme) for a full intro to the framework.
  22. ## Why build agents with UIs?
  23. Coding agents work with more than a text box. Their environment provides context, tools, files, tests, and previews that make their capabilities and results visible.
  24. Knowledge work needs that same kind of environment. A UI shows what an agent can do and gives people familiar ways to inspect, edit, approve, and share its work.
  25. ## How Agent-Native works
  26. - **[Shared actions](https://agent-native.com/docs/actions-overview?utm_source=github&utm_medium=referral&utm_content=readme).** The agent calls each capability as a tool, and the UI calls it from code. Both paths use the same validation, permissions, and implementation.
  27. - **[Shared data](https://agent-native.com/docs/server-database?utm_source=github&utm_medium=referral&utm_content=readme).** Work done by the agent appears in the UI, and work done in the UI is available to the agent.
  28. - **[Shared application state](https://agent-native.com/docs/context-awareness?utm_source=github&utm_medium=referral&utm_content=readme).** The agent receives relevant UI state, such as the current page, selected record, or active view.
  29. The agent does not click through the UI. It works through the same action layer as the UI.
  30. ### Example: a shared action
  31. Create `actions/hello.ts`:
  32. ```ts
  33. import { defineAction } from "@agent-native/core/action";
  34. import { z } from "zod";
  35. // One action powers every app surface: UI, agent, HTTP, MCP, A2A, and CLI.
  36. export default defineAction({
  37. description: "Return a friendly greeting.",
  38. schema: z.object({
  39. name: z.string().default("world").describe("Name to greet"),
  40. }),
  41. http: { method: "GET" },
  42. run: async ({ name }) => {
  43. return { message: `Hello, ${name}!` };
  44. },
  45. });
  46. ```
  47. The agent receives `hello` as a tool. React calls the same function with `useActionQuery("hello", { name: "Alex" })`. Agent-Native also exposes it through HTTP, MCP, A2A, and the CLI.
  48. Actions may set `persistInChatHistory: false` for sensitive results. Live tool calls and the current UI retain their values, while durable chat, trace and audit payloads omit the private content and its derived assistant output. Later requests use the server's sanitized thread history; unavailable projections return a retryable error. Browser caching is controlled separately with `persistInBrowser: false`. The current model invocation still receives the tool result. Private turns skip Context Xray transformation and manifest persistence. This policy does not erase historical records.
  49. Private approvals retain an opaque server approval ID, with server-keyed argument checks. Pending approvals created before this format or before a server encryption-key rotation require a new approval. An interrupted turn that already dispatched a private tool cannot safely reconstruct its omitted input and result, so automatic recovery stops and requires a new turn. A verified approval-only pause that did not execute remains resumable.
  50. ## Included
  51. - **[Agent chat](https://agent-native.com/docs/agent-surfaces?utm_source=github&utm_medium=referral&utm_content=readme):** Let people delegate work, ask questions, and review results in the same UI.
  52. - **[Authentication and permissions](https://agent-native.com/docs/authentication?utm_source=github&utm_medium=referral&utm_content=readme):** Control who can access and change shared work.
  53. - **[Skills and memory](https://agent-native.com/docs/agent-resources?utm_source=github&utm_medium=referral&utm_content=readme):** Give agents reusable expertise and persistent context.
  54. - **[Automations](https://agent-native.com/docs/automations?utm_source=github&utm_medium=referral&utm_content=readme):** Run agent work on schedules or events.
  55. - **[Agent teams](https://agent-native.com/docs/agent-teams?utm_source=github&utm_medium=referral&utm_content=readme):** Delegate work to specialist agents in the same workspace or across connected agents.
  56. - **[PostgreSQL backend](https://agent-native.com/docs/server-database?utm_source=github&utm_medium=referral&utm_content=readme):** Use PostgreSQL in production and PGlite for local development on any Nitro-compatible host.
  57. ### Example: edit a running app visually
  58. Use `/visual-edit` in Claude Code to edit a running app on a canvas instead of
  59. prompting for each small UI nudge. Compare live pages together, then ask Claude
  60. to apply the chosen edits to your source. For example:
  61. ```text
  62. /visual-edit the onboarding flow, plus home at every breakpoint
  63. ```
  64. Review the onboarding screens alongside Home at desktop, tablet, and mobile
  65. sizes, adjust a button's color or spacing directly, then ask Claude to apply the
  66. pending edits to your source. See the [Visual Edit guide](./skills/visual-edit/README.md)
  67. for the full walkthrough and sharing details.
  68. To bring an effect or section from another site into yours, use `/remix` in your
  69. repo: `/remix add the stripe hero animation to ours but make it blue`. It
  70. rebuilds the behavior in original code with typed props or CSS knobs. To pick
  71. visually, ask `/remix in Design, the stripe hero into my homepage` and select
  72. source and destination on the canvas. See the [Remix guide](./skills/remix/README.md).
  73. Bring your LLM, SQL database, tools, and infrastructure. Everything you build stays yours.
  74. See Agent-Native in action:
  75. https://github.com/user-attachments/assets/ef51644b-6506-46d8-8083-0af7b7e5b65c
  76. <br />
  77. ## Open-source agents
  78. Start from one of these agents or use it as an example for your own.
  79. For a source-to-answer architecture example, see [Brain’s visual overview](templates/brain/README.md#brain-at-a-glance).
  80. <br />
  81. <table>
  82. <tr>
  83. <td width="33%" align="center" valign="top">
  84. <br />
  85. **Clips**
  86. <a href="https://agent-native.com/apps/clips/?utm_source=github&utm_medium=referral&utm_content=readme">
  87. <picture>
  88. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F4dc9c87b9a224132855e1cb68cd89f9e?format=webp&width=800">
  89. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F462fdb643d5d403a8c54540c5c3292f6?format=webp&width=800" alt="Clips app screenshot" width="100%">
  90. </picture>
  91. </a>
  92. Record and understand meetings, screens, and voice notes.
  93. <br />
  94. </td>
  95. <td width="33%" align="center" valign="top">
  96. <br />
  97. **Design**
  98. <a href="https://agent-native.com/apps/design/?utm_source=github&utm_medium=referral&utm_content=readme">
  99. <picture>
  100. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F072f66a3e360464fb48617670ceee46f?format=webp&width=800">
  101. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2Fc9f693f170294bde8ebc733cff368af9?format=webp&width=800" alt="Design app screenshot" width="100%">
  102. </picture>
  103. </a>
  104. Generate and refine interactive designs.
  105. <br />
  106. </td>
  107. <td width="33%" align="center" valign="top">
  108. <br />
  109. **Slides**
  110. <a href="https://agent-native.com/apps/slides/?utm_source=github&utm_medium=referral&utm_content=readme">
  111. <picture>
  112. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F710f5b7586cc41deaa0e6f8de658a499?format=webp&width=800">
  113. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F6223a64717a04931a6f509696019f48b?format=webp&width=800" alt="Slides app screenshot" width="100%">
  114. </picture>
  115. </a>
  116. Create and edit on-brand presentations.
  117. <br />
  118. </td>
  119. </tr>
  120. <tr>
  121. <td width="33%" align="center" valign="top">
  122. <br />
  123. **Analytics**
  124. <a href="https://agent-native.com/apps/analytics/?utm_source=github&utm_medium=referral&utm_content=readme">
  125. <picture>
  126. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2Fba632b39758d448594f7e5d2403e5d0f?format=webp&width=800">
  127. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F976ea93771b64b9e8c1b3eb12e606e07?format=webp&width=800" alt="Analytics app screenshot" width="100%">
  128. </picture>
  129. </a>
  130. Ask questions of your data and build dashboards.
  131. <br />
  132. </td>
  133. <td width="33%" align="center" valign="top">
  134. <br />
  135. **Calendar**
  136. <a href="https://agent-native.com/apps/calendar/?utm_source=github&utm_medium=referral&utm_content=readme">
  137. <picture>
  138. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F91d4c18d256b49e6bca24c23ce90a4f2?format=webp&width=800">
  139. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F8d9d48bb2f1d498f9601ac28c64edd4c?format=webp&width=800" alt="Calendar app screenshot" width="100%">
  140. </picture>
  141. </a>
  142. Find time, schedule events, and manage bookings.
  143. <br />
  144. </td>
  145. <td width="33%" align="center" valign="top">
  146. <br />
  147. **Mail**
  148. <a href="https://agent-native.com/apps/mail/?utm_source=github&utm_medium=referral&utm_content=readme">
  149. <picture>
  150. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F216aaef9859144ffa3eb9498fa1132da?format=webp&width=800">
  151. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F064aee3ec5cd44879b9c186f63a6d6f4?format=webp&width=800" alt="Mail app screenshot" width="100%">
  152. </picture>
  153. </a>
  154. Prioritize email, draft replies, and follow up.
  155. <br />
  156. </td>
  157. </tr>
  158. <tr>
  159. <td width="33%" align="center" valign="top">
  160. <br />
  161. **Assets**
  162. <a href="https://agent-native.com/apps/assets/?utm_source=github&utm_medium=referral&utm_content=readme">
  163. <picture>
  164. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2Fde445dd867744e9bac5d3e5d04c903b2?format=webp&width=800">
  165. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2Fac5701f0871f4edbb06fa9cdb12b166e?format=webp&width=800" alt="Assets app screenshot" width="100%">
  166. </picture>
  167. </a>
  168. Create and organize on-brand media.
  169. <br />
  170. </td>
  171. <td width="33%" align="center" valign="top">
  172. <br />
  173. **Content**
  174. <a href="https://agent-native.com/apps/content/?utm_source=github&utm_medium=referral&utm_content=readme">
  175. <picture>
  176. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2Fa9f30380b03b4fd1a7d2ab371ddfd798?format=webp&width=800">
  177. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F979f792c79834470a513a9d9b733dd84?format=webp&width=800" alt="Content app screenshot" width="100%">
  178. </picture>
  179. </a>
  180. Draft, organize, and publish content.
  181. <br />
  182. </td>
  183. <td width="33%" align="center" valign="top">
  184. <br />
  185. **Plans**
  186. <a href="https://agent-native.com/apps/plan/?utm_source=github&utm_medium=referral&utm_content=readme">
  187. <picture>
  188. <source media="(prefers-color-scheme: dark)" srcset="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2Fe89439e917044fc9ac9663737e35bf1f?format=webp&width=800">
  189. <img src="https://cdn.builder.io/api/v1/image/assets%2FYJIGb4i01jvw0SRdL5Bt%2F98427229c8c84c30afee56172503c294?format=webp&width=800" alt="Plans app screenshot" width="100%">
  190. </picture>
  191. </a>
  192. Create and review visual plans with diagrams, wireframes, and prototypes.
  193. <br />
  194. </td>
  195. </tr>
  196. </table>
  197. Explore the [full app gallery](https://agent-native.com/apps?utm_source=github&utm_medium=referral&utm_content=readme), or start with the [framework guide](https://agent-native.com/docs/getting-started?utm_source=github&utm_medium=referral&utm_content=readme).
  198. ## Documentation and community
  199. - Read the [documentation](https://agent-native.com/docs?utm_source=github&utm_medium=referral&utm_content=readme).
  200. - Join [Discord](https://discord.gg/qm82StQ2NC) to ask questions, share what you're building, and get help.
  201. - The CLI collects anonymous usage data. [Telemetry](https://agent-native.com/docs/telemetry?utm_source=github&utm_medium=referral&utm_content=readme) lists what it sends and how to turn it off.
  202. ## Contributing
  203. Want to send a change back? Start with [CONTRIBUTING.md](./CONTRIBUTING.md). [DEVELOPMENT.md](./DEVELOPMENT.md) covers local setup, workspace structure, and guard scripts.
  204. ## License
  205. MIT
.agents/skills/fragility-common/scripts/findings-upsert.ts · Bundled script · 526 lines · 1 finding marked
  1. // The only script that writes refactor findings to the configured tracker.
  2. // Two inputs, one path:
  3. // --plan <file> a finished plan
  4. // --key KEY --system <system> a hot system already covered by KEY (no plan)
    • Note: Usual in code (tag names in paths and patterns): Contains a <system> block, a common wrapper for instructions hidden in a tool description
    • Line 4 on GitHub
  5. // Outcomes, idempotent on re-run:
  6. // no ticket → create (label, plan, run link)
  7. // open ticket → sighting; comment (+ updated plan) at most once per cooldown
  8. // declined ticket → sighting; comment at most once per four cooldowns
  9. // fixed ticket → plan: create a linked recurrence ticket; system: refuse
  10. // A sighting with a bug source always comments once per run: a real bug traced
  11. // to the pattern is new evidence, not nightly noise. Dry-run unless --apply.
  12. import path from "node:path";
  13. import {
  14. appendFindingResult,
  15. type FindingResult,
  16. readFindingResults,
  17. readTargets,
  18. } from "../lib/artifacts.ts";
  19. import {
  20. type Args,
  21. argList,
  22. argString,
  23. main,
  24. rel,
  25. runId,
  26. ScriptError,
  27. } from "../lib/cli.ts";
  28. import {
  29. code,
  30. commentDue,
  31. type Finding,
  32. type Inline,
  33. loadTracker,
  34. mergeSighting,
  35. overlap,
  36. type Paragraph,
  37. planNamePattern,
  38. runLink,
  39. runMarker,
  40. type Sighting,
  41. text,
  42. type Tracker,
  43. } from "../lib/findings.ts";
  44. import {
  45. type PlanMeta,
  46. readPlan,
  47. TODO,
  48. writePlan,
  49. } from "../lib/plan-format.ts";
  50. const USAGE = `findings-upsert --plan <file> [--key KEY | --distinct-from KEY,KEY] [--apply] [--run-url <url> | --no-run-url] [--force-comment] [--over-cap]
  51. findings-upsert --run <id> --key KEY --system <system in targets.json> [--source <bug url or ref> --source-title "<title>" --note "<how>"] [--apply]
  52. KEY is ENG-123 for Jira, or an issue number or URL for GitHub.
  53. Plan mode matches by fingerprint, then by overlap (fingerprint anchor, same or nested system, shared files).
  54. Overlapping tickets must be judged: --key if the plan is that ticket, --distinct-from if it is a different root problem.`;
  55. interface Subject {
  56. runId: string;
  57. sighting: Sighting;
  58. base: { fingerprint: string; systems: string[]; paths: string[] };
  59. /** One line for the comment: what this run saw. */
  60. lead: Inline[];
  61. bugSource: string | null;
  62. }
  63. main(async (args) => {
  64. const planFile = argString(args, "plan");
  65. const rawKey = argString(args, "key");
  66. const system = argString(args, "system");
  67. if (args.help || (!planFile && !(rawKey && system))) {
  68. console.log(USAGE);
  69. if (!args.help) process.exitCode = 1;
  70. return;
  71. }
  72. const link = argString(args, "run-url") ?? runLink().url;
  73. if (!link && !args["no-run-url"]) {
  74. throw new ScriptError(
  75. "no run link: set FRAGILITY_RUN_URL or pass --run-url; pass --no-run-url only for local runs with no shareable link",
  76. 2,
  77. );
  78. }
  79. const tracker = (await loadTracker()).createTracker();
  80. const key = rawKey ? tracker.parseKey(rawKey) : undefined;
  81. const findings = await tracker.findings();
  82. if (planFile) await upsertPlan(tracker, findings, args, planFile, key, link);
  83. else await sightSystem(tracker, findings, args, key!, system!, link);
  84. });
  85. async function upsertPlan(
  86. tracker: Tracker,
  87. findings: Finding[],
  88. args: Args,
  89. planFile: string,
  90. key: string | undefined,
  91. link: string | null,
  92. ): Promise<void> {
  93. const file = path.resolve(planFile);
  94. const { meta, body } = readPlan(file);
  95. assertFinished(meta, body, file);
  96. const match = matchPlan(
  97. tracker,
  98. meta,
  99. findings,
  100. key,
  101. argList(args, "distinct-from").map((k) => tracker.parseKey(k)),
  102. );
  103. const subject: Subject = {
  104. runId: meta.runId,
  105. sighting: sightingFrom(meta),
  106. base: {
  107. fingerprint: meta.fingerprint,
  108. systems: meta.systems,
  109. paths: meta.paths,
  110. },
  111. lead: planLead(meta),
  112. bugSource: meta.trigger === "bug" ? meta.source : null,
  113. };
  114. // A ticket this run already created (then failed partway) is finished, not
  115. // sighted; once that ticket is resolved, the plan is a recurrence instead.
  116. const resuming =
  117. match !== null &&
  118. match.open &&
  119. readFindingResults(meta.runId).some(
  120. (r) =>
  121. r.key === match.key &&
  122. r.plan === rel(file) &&
  123. (r.action === "create" || r.action === "recurrence"),
  124. );
  125. const recurrence =
  126. !resuming && match !== null && !match.open && !match.declined;
  127. const action = resuming
  128. ? "resume-create"
  129. : match && !recurrence
  130. ? "sighting"
  131. : recurrence
  132. ? "recurrence"
  133. : "create";
  134. const planName = `${path.basename(file, ".md")}-${meta.runId}.md`;
  135. const creating = action === "create" || action === "recurrence";
  136. if (creating && meta.confidence.toLowerCase() === "low") {
  137. throw new ScriptError(
  138. "low-confidence plans are not filed; record the system in decisions.json instead",
  139. );
  140. }
  141. if (!args.apply) {
  142. console.log(
  143. JSON.stringify(
  144. {
  145. dryRun: true,
  146. tracker: tracker.name,
  147. action,
  148. key: match?.key ?? null,
  149. fingerprint: meta.fingerprint,
  150. planName,
  151. runLink: link,
  152. },
  153. null,
  154. 2,
  155. ),
  156. );
  157. return;
  158. }
  159. const planVersions = (match?.plans ?? []).filter((a) =>
  160. planNamePattern(path.basename(file, ".md")).test(a),
  161. );
  162. const resultFor = (action: string, key: string): FindingResult => ({
  163. action,
  164. key,
  165. url: tracker.ticketUrl(key),
  166. plan: rel(file),
  167. system: meta.systems[0],
  168. source: subject.bugSource,
  169. at: new Date().toISOString(),
  170. });
  171. let result: FindingResult;
  172. if (action === "sighting") {
  173. const key = match!.key;
  174. const { commented, commentedThisRun } = await recordSighting(
  175. tracker,
  176. match!,
  177. subject,
  178. link,
  179. Boolean(args["force-comment"]),
  180. );
  181. // Reconciled on every run, so a failed upload after a comment is retried.
  182. const wantsThisRunsPlan = commentedThisRun
  183. ? !planVersions.includes(planName)
  184. : planVersions.length === 0;
  185. if (wantsThisRunsPlan) await tracker.attachPlan(key, file, planName);
  186. result = resultFor(commented ? "sighting-comment" : "sighting-silent", key);
  187. appendFindingResult(meta.runId, result);
  188. } else {
  189. let key: string;
  190. let recurrenceOf: string | undefined;
  191. if (action === "resume-create") {
  192. key = match!.key;
  193. recurrenceOf = match!.recurrenceOf ?? undefined;
  194. result = resultFor(action, key);
  195. } else {
  196. const created = readFindingResults(meta.runId).filter(
  197. (r) => r.action === "create" || r.action === "recurrence",
  198. ).length;
  199. if (created >= tracker.maxNewTicketsPerRun && !args["over-cap"]) {
  200. throw new ScriptError(
  201. `run ${meta.runId} already created ${created} tickets (cap ${tracker.maxNewTicketsPerRun}); keep the plan file and report it, or pass --over-cap if a human asked for more`,
  202. );
  203. }
  204. const previous = recurrence ? match : null;
  205. key = await tracker.create({
  206. meta,
  207. facts: createFacts(meta, link, previous),
  208. previous,
  209. planFile: file,
  210. planName,
  211. });
  212. recurrenceOf = previous?.key;
  213. // Recorded before the fallible follow-ups, so the cap counts it and a
  214. // re-run resumes this ticket instead of treating it as a sighting.
  215. result = resultFor(action, key);
  216. appendFindingResult(meta.runId, result);
  217. }
  218. if (!(await tracker.currentProperty(key))) {
  219. await tracker.setProperty(
  220. key,
  221. mergeSighting(null, subject.base, subject.sighting, {
  222. commentedOn: subject.sighting.date,
  223. recurrenceOf,
  224. }),
  225. );
  226. }
  227. if (!planVersions.includes(planName))
  228. await tracker.attachPlan(key, file, planName);
  229. if (recurrenceOf) await tracker.ensureRecurrenceLink(key, recurrenceOf);
  230. }
  231. writePlan(file, { ...meta, ticket: result.key }, body);
  232. console.log(JSON.stringify(result, null, 2));
  233. }
  234. async function sightSystem(
  235. tracker: Tracker,
  236. findings: Finding[],
  237. args: Args,
  238. key: string,
  239. system: string,
  240. link: string | null,
  241. ): Promise<void> {
  242. const id = runId(args);
  243. const note = argString(args, "note");
  244. const source = argString(args, "source") ?? null;
  245. const sourceTitle = argString(args, "source-title") ?? null;
  246. if (source && !note) {
  247. throw new ScriptError(
  248. "--source needs --note with one sentence on how this bug is an instance of the ticket's pattern",
  249. );
  250. }
  251. const target = readTargets(id).find((t) => t.system === system);
  252. if (!target)
  253. throw new ScriptError(`${system} is not in run ${id}'s targets.json`);
  254. const finding = findings.find((f) => f.key === key);
  255. if (!finding)
  256. throw new ScriptError(
  257. `${key} is not a ${tracker.label} ticket in ${tracker.name}`,
  258. );
  259. if (!finding.open && !finding.declined) {
  260. throw new ScriptError(
  261. `${key} was resolved (${finding.resolution ?? finding.status}); a hot system after a fix is a recurrence. Write a plan and upsert it, which opens a linked recurrence ticket.`,
  262. );
  263. }
  264. const summary = `${target.verdict}, score ${target.score}, ${target.windowCommits ?? "n/a"} window commits, ${target.lookbackFixes} lookback fixes; weekly fixes ${target.weeklyFixes.join(" → ")}`;
  265. const lead: Inline[] = source
  266. ? [
  267. text("Bug report "),
  268. /^https?:\/\//.test(source)
  269. ? text(sourceTitle ?? source, source)
  270. : text(sourceTitle ? `${sourceTitle} (${source})` : source),
  271. text(` traced to this pattern in ${runMarker(id)} (${system}). `),
  272. text(`${note} `),
  273. ]
  274. : [text(`Seen again in ${runMarker(id)} (${system}): ${summary}. `)];
  275. const subject: Subject = {
  276. runId: id,
  277. sighting: {
  278. runId: id,
  279. date: today(),
  280. score: target.score,
  281. verdict: target.verdict,
  282. windowCommits: target.windowCommits,
  283. lookbackFixes: target.lookbackFixes,
  284. source,
  285. },
  286. base: {
  287. fingerprint: finding.fingerprint ?? `fsys:${system}:external`,
  288. systems: [system],
  289. paths: target.files,
  290. },
  291. lead,
  292. bugSource: source,
  293. };
  294. if (!args.apply) {
  295. const comment =
  296. Boolean(source) || commentDue(finding, tracker.sightingCooldownDays);
  297. console.log(
  298. JSON.stringify(
  299. { dryRun: true, tracker: tracker.name, key, system, comment, summary },
  300. null,
  301. 2,
  302. ),
  303. );
  304. return;
  305. }
  306. const { commented } = await recordSighting(
  307. tracker,
  308. finding,
  309. subject,
  310. link,
  311. false,
  312. );
  313. const result: FindingResult = {
  314. action: commented ? "sighting-comment" : "sighting-silent",
  315. key,
  316. url: tracker.ticketUrl(key),
  317. system,
  318. source,
  319. at: new Date().toISOString(),
  320. };
  321. appendFindingResult(id, result);
  322. console.log(JSON.stringify({ ...result, summary }, null, 2));
  323. }
  324. // Each write checks what a previous partial run already did, so a re-run
  325. // after a failure finishes the job instead of repeating it.
  326. async function recordSighting(
  327. tracker: Tracker,
  328. finding: Finding,
  329. subject: Subject,
  330. link: string | null,
  331. force: boolean,
  332. ): Promise<{ commented: boolean; commentedThisRun: boolean }> {
  333. const existing = await tracker.currentProperty(finding.key);
  334. const alreadyCommented = await tracker.hasRunComment(
  335. finding.key,
  336. subject.runId,
  337. );
  338. const due =
  339. force ||
  340. Boolean(subject.bugSource) ||
  341. commentDue(finding, tracker.sightingCooldownDays);
  342. const comment = due && !alreadyCommented;
  343. if (comment)
  344. await tracker.comment(finding.key, sightingComment(finding, subject, link));
  345. // A run's comment counts from the day the run first sighted the ticket, so a
  346. // replay never pushes lastCommented later, and a retry after a failed
  347. // property write still records the comment it posted.
  348. const runDate =
  349. existing?.sightings.find((s) => s.runId === subject.runId)?.date ??
  350. subject.sighting.date;
  351. const stamped = (existing?.lastCommented ?? "") >= runDate;
  352. await tracker.setProperty(
  353. finding.key,
  354. mergeSighting(existing, subject.base, subject.sighting, {
  355. commentedOn: comment
  356. ? subject.sighting.date
  357. : alreadyCommented && !stamped
  358. ? runDate
  359. : undefined,
  360. }),
  361. );
  362. if (finding.recurrenceOf)
  363. await tracker.ensureRecurrenceLink(finding.key, finding.recurrenceOf);
  364. return { commented: comment, commentedThisRun: comment || alreadyCommented };
  365. }
  366. function matchPlan(
  367. tracker: Tracker,
  368. meta: PlanMeta,
  369. findings: Finding[],
  370. key: string | undefined,
  371. distinctFrom: string[],
  372. ): Finding | null {
  373. if (key) {
  374. const chosen = findings.find((f) => f.key === key);
  375. if (!chosen)
  376. throw new ScriptError(
  377. `--key ${key} is not a ${tracker.label} ticket in ${tracker.name}`,
  378. );
  379. return chosen;
  380. }
  381. const exact = findings.find((f) => f.fingerprint === meta.fingerprint);
  382. if (exact) return exact;
  383. const overlapping = findings
  384. .map((f) => ({
  385. f,
  386. why: meta.systems.flatMap((s) => overlap(s, meta.paths, f)),
  387. }))
  388. .filter(({ f, why }) => why.length > 0 && !distinctFrom.includes(f.key));
  389. if (overlapping.length) {
  390. const list = overlapping
  391. .map(
  392. ({ f, why }) =>
  393. `${f.key} [${f.status}] "${f.summary}" (${[...new Set(why)].join("; ")})`,
  394. )
  395. .join("\n ");
  396. throw new ScriptError(
  397. `this plan overlaps existing tickets:\n ${list}\nPass --key <KEY> if the plan is that ticket's root problem, or --distinct-from ${overlapping.map(({ f }) => f.key).join(",")} if it is a different one.`,
  398. );
  399. }
  400. return null;
  401. }
  402. function assertFinished(meta: PlanMeta, body: string, file: string): void {
  403. const problems: string[] = [];
  404. if (body.includes(TODO) || meta.summary.includes(TODO))
  405. problems.push(`unfilled ${TODO} markers`);
  406. if (!["high", "medium", "low"].includes(meta.confidence.toLowerCase()))
  407. problems.push("confidence must be high, medium, or low");
  408. if (meta.systems.length === 0) problems.push("no systems listed");
  409. if (problems.length)
  410. throw new ScriptError(`${rel(file)} is not ready: ${problems.join("; ")}`);
  411. }
  412. function today(): string {
  413. return new Date().toISOString().slice(0, 10);
  414. }
  415. function sightingFrom(meta: PlanMeta): Sighting {
  416. return {
  417. runId: meta.runId,
  418. date: today(),
  419. score: meta.score,
  420. verdict: meta.verdict,
  421. windowCommits: meta.windowCommits,
  422. lookbackFixes: meta.lookbackFixes,
  423. source: meta.source,
  424. };
  425. }
  426. function planLead(meta: PlanMeta): Inline[] {
  427. return meta.trigger === "bug"
  428. ? [
  429. text("Bug report "),
  430. sourceText(meta.source),
  431. text(
  432. ` traced to this pattern in ${runMarker(meta.runId)}: ${meta.summary} `,
  433. ),
  434. ]
  435. : [
  436. text(
  437. `Seen again in ${runMarker(meta.runId)}: ${meta.verdict}, score ${meta.score ?? "n/a"}, ${meta.windowCommits ?? "?"} window commits, ${meta.lookbackFixes ?? "?"} lookback fixes. Updated plan attached. `,
  438. ),
  439. ];
  440. }
  441. function sightingComment(
  442. finding: Finding,
  443. subject: Subject,
  444. link: string | null,
  445. ): Paragraph[] {
  446. const since = finding.firstSeen ?? finding.sightings[0]?.date;
  447. const declined = finding.declined
  448. ? [
  449. text(
  450. "This ticket was declined, so no new ticket was opened; consider reopening. ",
  451. ),
  452. ]
  453. : [];
  454. return [
  455. [...subject.lead, ...declined],
  456. [
  457. text(
  458. `${finding.sightingCount + 1} sightings${since ? ` since ${since}` : ""}. `,
  459. ),
  460. ...(link ? [text("Run", link)] : []),
  461. ],
  462. ];
  463. }
  464. function sourceText(source: string | null): Inline {
  465. if (!source) return text("unlinked report");
  466. return /^https?:\/\//.test(source) ? text(source, source) : text(source);
  467. }
  468. function createFacts(
  469. meta: PlanMeta,
  470. link: string | null,
  471. previous: Finding | null,
  472. ): Paragraph[] {
  473. const facts: Paragraph[] = [
  474. [
  475. text(
  476. `Verdict: ${meta.verdict}, confidence ${meta.confidence}, score ${meta.score ?? "n/a"}.`,
  477. ),
  478. ],
  479. meta.trigger === "bug"
  480. ? [
  481. text("Triggered by bug report: "),
  482. sourceText(meta.source),
  483. text(
  484. `. ${meta.lookbackFixes ?? "?"} fixes in the lookback on these systems.`,
  485. ),
  486. ]
  487. : [
  488. text(
  489. `Signal: ${meta.windowCommits ?? "?"} commits in the review window, ${meta.lookbackFixes ?? "?"} fixes in the lookback.`,
  490. ),
  491. ],
  492. [text("Systems: "), code(meta.systems.join(", "))],
  493. link
  494. ? [text("Analysis run: "), text(link, link)]
  495. : [text("Analysis run: local run, no shareable link")],
  496. [text("Fingerprint: "), code(meta.fingerprint)],
  497. ];
  498. if (previous)
  499. facts.unshift([
  500. text("Recurrence of "),
  501. text(previous.key, previous.url),
  502. text(
  503. `, which was resolved as ${previous.resolution ?? previous.status}.`,
  504. ),
  505. ]);
  506. return facts;
  507. }
.agents/skills/investigate-bug/scripts/bug-verdict.ts · Bundled script · 152 lines · 1 finding marked
  1. // Records the outcome of a bug-report run, one entry per symptom:
  2. // one-off → a local defect; the root cause and the suggested local fix
  3. // pattern → an instance of a systemic problem; a plan (filed or not)
  4. // known → an existing refactor-findings ticket already covers it
  5. // needs-info → cannot be decided from code and history; what to ask for
  6. // Re-running with the same --symptom replaces that entry.
  7. import path from "node:path";
  8. import {
  9. artifact,
  10. readBug,
  11. readFindingResults,
  12. } from "../../fragility-common/lib/artifacts.ts";
  13. import {
  14. argString,
  15. main,
  16. readJsonOr,
  17. rel,
  18. runId,
  19. ScriptError,
  20. writeJson,
  21. } from "../../fragility-common/lib/cli.ts";
  22. import { loadTracker } from "../../fragility-common/lib/findings.ts";
  23. import { readPlan } from "../../fragility-common/lib/plan-format.ts";
  24. export interface BugVerdict {
  25. symptom: string;
  26. verdict: "one-off" | "pattern" | "known" | "needs-info";
  27. rootCause: string;
  28. trigger: string | null;
  29. ruledOut: string | null;
  30. evidence: "reproduced" | "traced" | "inferred";
  31. reason: string;
  32. fix: string | null;
  33. plan: string | null;
  34. ticket: string | null;
  35. ticketUrl: string | null;
  36. unfiled: string | null;
  37. ask: string | null;
  38. at: string;
  39. }
  40. const VERDICTS = ["one-off", "pattern", "known", "needs-info"] as const;
  41. const EVIDENCE = ["reproduced", "traced", "inferred"] as const;
  42. main(async (args) => {
  43. const verdict = argString(args, "verdict");
  44. const rootCause = argString(args, "root-cause");
  45. const reason = argString(args, "reason");
  46. if (args.help || !verdict || !rootCause || !reason) {
  47. console.log(
  48. 'bug-verdict --run <id> --verdict one-off|pattern|known|needs-info --root-cause "<file:line, what goes wrong>" --reason "<why>" [--symptom "<label>"] [--fix "<local fix>"] [--plan <file>] [--unfiled "<why no ticket>"] [--ticket <ENG-123 or #123>] [--ask "<what to get from the reporter>"] --evidence reproduced|traced|inferred --trigger "<what changed to break it now, or none-found: what you checked>" --ruled-out "<other explanations and the check that ruled each out>"\n one-off needs --fix; pattern needs --plan; known needs --ticket with a sighting recorded in this run; needs-info needs --ask.\n --trigger and --ruled-out are required for one-off and pattern.',
  49. );
  50. if (!args.help) process.exitCode = 1;
  51. return;
  52. }
  53. if (!(VERDICTS as readonly string[]).includes(verdict))
  54. throw new ScriptError(`--verdict must be one of ${VERDICTS.join(", ")}`);
  55. const evidence = argString(args, "evidence");
  56. if (!evidence || !(EVIDENCE as readonly string[]).includes(evidence))
  57. throw new ScriptError(`--evidence must be one of ${EVIDENCE.join(", ")}`);
  58. const trigger = argString(args, "trigger") ?? null;
  59. const ruledOut = argString(args, "ruled-out") ?? null;
  60. // A decided verdict must explain "why now" and survive at least one rival
  61. // explanation; a single plausible story is how the real cause gets missed.
  62. if (
  63. (verdict === "one-off" || verdict === "pattern") &&
  64. (!trigger || !ruledOut)
  65. ) {
  66. throw new ScriptError(
  67. `${verdict} needs --trigger (the recent change that broke it, or "none-found: <what you checked>") and --ruled-out (rival explanations and the check that ruled each out)`,
  68. );
  69. }
  70. const id = runId(args);
  71. if (!readBug(id))
  72. throw new ScriptError(
  73. `run ${id} has no bug.json; start it with bug-intake`,
  74. );
  75. const fix = argString(args, "fix") ?? null;
  76. const planArg = argString(args, "plan");
  77. const unfiled = argString(args, "unfiled") ?? null;
  78. const ask = argString(args, "ask") ?? null;
  79. const tracker = await loadTracker();
  80. const ticketArg = argString(args, "ticket");
  81. let ticket = ticketArg ? tracker.parseKey(ticketArg) : null;
  82. let plan: string | null = null;
  83. if (verdict === "one-off") {
  84. if (!fix) throw new ScriptError("one-off needs --fix with the local fix");
  85. if (planArg || ticket)
  86. throw new ScriptError("one-off takes no --plan or --ticket");
  87. }
  88. if (verdict === "pattern") {
  89. if (!planArg) throw new ScriptError("pattern needs --plan <file>");
  90. const file = path.resolve(planArg);
  91. const { meta } = readPlan(file);
  92. if (meta.runId !== id)
  93. throw new ScriptError(
  94. `${planArg} belongs to run ${meta.runId}, not ${id}`,
  95. );
  96. plan = rel(file);
  97. ticket = meta.ticket;
  98. if (!ticket && !unfiled) {
  99. throw new ScriptError(
  100. `${planArg} has no ticket. File it with findings-upsert --apply, or pass --unfiled "<reason>" (low confidence, ticket cap)`,
  101. );
  102. }
  103. }
  104. if (verdict === "needs-info") {
  105. if (!ask)
  106. throw new ScriptError(
  107. "needs-info needs --ask with the log, repro, or detail that would decide it",
  108. );
  109. if (planArg || ticket)
  110. throw new ScriptError("needs-info takes no --plan or --ticket");
  111. }
  112. if (verdict === "known") {
  113. if (!ticket) throw new ScriptError("known needs --ticket");
  114. const results = readFindingResults(id);
  115. // A nightly or unrelated sighting is not evidence the bug is covered.
  116. if (!results.some((r) => r.key === ticket && r.source)) {
  117. throw new ScriptError(
  118. `no bug-linked sighting on ${ticket} in this run; run findings-upsert --run ${id} --key ${ticket} --system <system> --source <report> --note "..." --apply first`,
    • Note: Usual in code (tag names in paths and patterns): Contains a <system> block, a common wrapper for instructions hidden in a tool description
    • Line 124 on GitHub
  119. );
  120. }
  121. }
  122. const entry: BugVerdict = {
  123. symptom: argString(args, "symptom") ?? "main",
  124. verdict: verdict as BugVerdict["verdict"],
  125. rootCause,
  126. trigger,
  127. ruledOut,
  128. evidence: evidence as BugVerdict["evidence"],
  129. reason,
  130. fix,
  131. plan,
  132. ticket,
  133. ticketUrl: ticket ? tracker.ticketUrl(ticket) : null,
  134. unfiled,
  135. ask,
  136. at: new Date().toISOString(),
  137. };
  138. const file = artifact(id, "verdicts");
  139. const existing = readJsonOr<BugVerdict[]>(file, []);
  140. writeJson(file, [
  141. ...existing.filter((v) => v.symptom !== entry.symptom),
  142. entry,
  143. ]);
  144. console.log(`${rel(file)}: ${entry.symptom} → ${entry.verdict}`);
  145. });

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.