Skip to content
Back to search

prodgate

MarketplaceWorks with Claude CodeGitHub

QA that ends in safe-for-production: state-bug hunting with proof, in rounds, on frozen commits.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/prodashu--prodgate/badge.svg)](/listings/prodashu--prodgate)

HTML

<a href="/listings/prodashu--prodgate"><img src="/api/listings/prodashu--prodgate/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

**1️⃣ Install prodgate**, in one line:

npx skills add ashusnapx/prodgate

<details> <summary>Or as a Claude Code plugin</summary>

/plugin marketplace add ashusnapx/prodgate
/plugin install prodgate@prodgate

</details>

**2️⃣ Install the two plugins prodgate requires.** Every agent prodgate runs uses both at their maximum setting, and prodgate will not start without them:

claude plugin marketplace add JuliusBrussee/caveman && claude plugin install caveman@caveman
claude plugin marketplace add DietrichGebert/ponytail && claude plugin install ponytail@ponytail

**3️⃣ Run it.** In any repo, say `/prodgate`, *"run QA"*, *"set up a QA pipeline"*, *"make it prod ready"* or *"bug hunt until zero"*.

It works on any language Claude Code can read. The gate gets sharper with the code-review-graph MCP (blast radius), `gitleaks`, `npm audit` and `pip-audit`.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 14 lines
  1. {
  2. "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
  3. "name": "prodgate",
  4. "description": "QA that ends in safe-for-production: state-bug hunting with proof, in rounds, on frozen commits.",
  5. "owner": { "name": "ashusnapx", "url": "https://github.com/ashusnapx" },
  6. "plugins": [
  7. {
  8. "name": "prodgate",
  9. "description": "Find and fix state bugs until a round finds nothing. Every bug comes with a failing test.",
  10. "source": "./",
  11. "category": "development"
  12. }
  13. ]
  14. }
README.md · README · 218 lines
  1. <div align="center">
  2. <img src="assets/hero.svg" alt="prodgate: hunt state bugs until zero, prove every one with a failing test" width="100%">
  3. <h3>The QA pipeline for Claude Code that hunts state bugs until zero,<br>and proves every one with a failing test.</h3>
  4. [![CI](https://github.com/ashusnapx/prodgate/actions/workflows/ci.yml/badge.svg)](https://github.com/ashusnapx/prodgate/actions/workflows/ci.yml)
  5. [![skills.sh](https://img.shields.io/badge/skills.sh-npx%20skills%20add%20ashusnapx%2Fprodgate-0F172A?style=flat-square)](https://skills.sh/ashusnapx/prodgate)
  6. [![Claude Code plugin](https://img.shields.io/badge/Claude%20Code-plugin-D97757?style=flat-square&logo=claude&logoColor=white)](#-install)
  7. [![Benchmark](https://img.shields.io/badge/benchmark-102%20real%20bugs%20%C2%B7%20re--check%20%E2%88%9239%25%20tokens-3B82F6?style=flat-square)](#-benchmark)
  8. [![Proof](https://img.shields.io/badge/every%20bug-a%20failing%20test-8B5CF6?style=flat-square)](#-why-prodgate)
  9. [![Lean](https://img.shields.io/badge/caveman%20%2B%20ponytail-ultra-F59E0B?style=flat-square)](#-standing-on-the-shoulders-of-caveman-and-ponytail)
  10. [![License: MIT](https://img.shields.io/badge/license-MIT-22C55E?style=flat-square)](LICENSE)
  11. **[⚡ Install](#-install)** · **[🧩 How it works](#-how-it-works)** · **[📊 Benchmark](#-benchmark)** · **[🙏 Credits](#-standing-on-the-shoulders-of-caveman-and-ponytail)** · **[🔬 Research](docs/RESEARCH.md)**
  12. </div>
  13. ---
  14. Most AI code review reads a diff and gives an opinion. **prodgate runs a loop until the code is safe to ship.** It maps every implicit state machine in your app, hunts bugs on a frozen commit, dismisses a finding only by quoting the code that makes it impossible, proves the rest with a failing test, fixes them test-first, and repeats until a round finds nothing. Then a production gate runs.
  15. It is built for the bugs that pass review and code coverage and still break production: 🔀 two requests interleaving · 💥 a job that crashes halfway · 📨 a webhook delivered twice · 🗂️ a tab left open on an old version · 🔁 a screen that never stops polling · 🏢 one tenant reading another's data.
  16. ## 🎬 What a run looks like
  17. ```diff
  18. @@ ◆ prodgate · round 1 · shop-api@1a2b3c4 @@
  19. 🗺️ 23 state machines mapped · 🔴 9 high risk · 🟡 10 medium · 🟢 4 low
  20. 🔎 41 reported · ✨ 41 new
  21. - ⚖️ ❌ 12 refuted with quoted code (e.g. orders.py:88 checks rowcount)
  22. + ⚖️ ✅ 22 confirmed · 🟡 7 unproven, sent to a test
  23. + 🛠️ ✅ 27 fixed with full evidence · 🚫 2 not reproduced, left alone
  24. @@ ◆ prodgate · round 2 · scope: 14 changed files + 9 callers · ♻️ 312 files skipped @@
  25. + 🛠️ ✅ 6 fixed
  26. @@ ◆ prodgate · round 3 · scope: 4 changed files @@
  27. + 🟢 clean round
  28. @@ 🚦 production gate @@
  29. + ✅ suites ✅ types ✅ lint ✅ build ✅ review ✅ security ✅ secrets ✅ migrations
  30. + 🚀 prodgate: ready to ship
  31. ```
  32. <sub>Illustrative run on an imaginary repo, to show the shape. Measured numbers are in the <a href="#-benchmark">benchmark</a>.</sub>
  33. ## ✨ Why prodgate
  34. | | 🤖 Typical AI review | 🛡️ prodgate |
  35. |---|---|---|
  36. | **What it reads** | ❌ The diff alone, or the whole repo every time | ✅ What changed since the last check, never-reviewed files and their callers. **Unchanged code costs 0 tokens** |
  37. | **When a finding counts** | ❌ When the model sounds sure | ✅ When a test fails because of it |
  38. | **How a finding is dismissed** | ❌ The model is unsure | ✅ Only by quoting the guard, lock or check that makes it impossible |
  39. | **When a fix counts** | ❌ The tests pass | ✅ Its test failed before, passes 3×, and fails again with only the fix undone |
  40. | **When it stops** | ❌ After one pass | ✅ A clean round on a frozen commit, then a production gate |
  41. | **Repeats** | ❌ The same issues every run | ✅ Every finding is keyed and never argued about twice |
  42. ## 🧠 The prompt at the heart of it
  43. ```text
  44. Find all of the implicit state machines in my app.
  45. ```
  46. Status columns, boolean flags, nullable timestamps, job queues, sessions, wizard steps, polling loops and cached screens are all state machines that nobody drew. Most production bugs live in the moves between their states. prodgate turns that one prompt into a pipeline.
  47. ## 🧩 How it works
  48. ```mermaid
  49. %%{init: {'theme':'base','themeVariables':{'fontFamily':'-apple-system, Segoe UI, Helvetica, sans-serif','fontSize':'15px','lineColor':'#94A3B8','primaryTextColor':'#0F172A','edgeLabelBackground':'#F1F5F9'}}}%%
  50. flowchart TD
  51. A([📌 Freeze a commit]):::freeze --> B{📒 Ledger<br/>what changed?}:::ledger
  52. B -- nothing --> Z([💤 0 agents · 0 tokens]):::zero
  53. B -- diff + callers --> C[🗺️ Map<br/>state machines]:::map --> D[🔎 Hunt<br/>per risky machine]:::hunt --> E[⚖️ Check<br/>per file group]:::check
  54. E -- refuted with<br/>quoted code --> Y([🗑️ never reported again]):::drop
  55. E -- confirmed<br/>or unproven --> F[🧪 Failing test first]:::prove
  56. F -- cannot fail --> X([🗑️ dropped]):::drop
  57. F -- fails --> G[🛠️ Smallest fix<br/>passes 3× · fails without it]:::fix
  58. G --> H[🔁 Commit, next round]:::loop --> B
  59. H -. clean round .-> I([🚀 Production gate]):::gate
  60. classDef freeze fill:#E2E8F0,stroke:#64748B,color:#0F172A
  61. classDef ledger fill:#FDE68A,stroke:#D97706,color:#451A03
  62. classDef zero fill:#F1F5F9,stroke:#94A3B8,color:#334155
  63. classDef map fill:#8B5CF6,stroke:#6D28D9,color:#FFFFFF
  64. classDef hunt fill:#3B82F6,stroke:#1D4ED8,color:#FFFFFF
  65. classDef check fill:#06B6D4,stroke:#0E7490,color:#FFFFFF
  66. classDef prove fill:#14B8A6,stroke:#0F766E,color:#FFFFFF
  67. classDef fix fill:#F59E0B,stroke:#B45309,color:#FFFFFF
  68. classDef loop fill:#64748B,stroke:#334155,color:#FFFFFF
  69. classDef gate fill:#22C55E,stroke:#15803D,color:#FFFFFF
  70. classDef drop fill:#FEE2E2,stroke:#DC2626,color:#7F1D1D
  71. ```
  72. | Stage | What happens |
  73. |---|---|
  74. | 🗺️ **Map** | Every implicit state machine, with states, transitions, guards, actors and a 🔴 🟡 🟢 risk level |
  75. | 🔎 **Hunt** | Races, stuck states, stale overwrites, illegal transitions, tenant leaks, double submits, lost UI state |
  76. | ⚖️ **Check** | One checker per file group. "Refuted" needs a quoted line; anything unsure goes to a test |
  77. | 🧪 **Prove** | A failing test, with a hook between the read and the write for races (never sleeps) |
  78. | 🛠️ **Fix** | The smallest change, accepted only with evidence. One owner per file, so fixes never collide |
  79. | 🚀 **Gate** | Suites, types, lint, build, fresh-eyes review, security, secrets, migrations, a browser smoke test |
  80. More in [how it works](docs/HOW-IT-WORKS.md).
  81. ## ⚡ Install
  82. **1️⃣ Install prodgate**, in one line:
  83. ```bash
  84. npx skills add ashusnapx/prodgate
  85. ```
  86. <details>
  87. <summary>Or as a Claude Code plugin</summary>
  88. ```text
  89. /plugin marketplace add ashusnapx/prodgate
  90. /plugin install prodgate@prodgate
  91. ```
  92. </details>
  93. **2️⃣ Install the two plugins prodgate requires.** Every agent prodgate runs uses both at their maximum setting, and prodgate will not start without them:
  94. ```bash
  95. claude plugin marketplace add JuliusBrussee/caveman && claude plugin install caveman@caveman
  96. claude plugin marketplace add DietrichGebert/ponytail && claude plugin install ponytail@ponytail
  97. ```
  98. **3️⃣ Run it.** In any repo, say `/prodgate`, *"run QA"*, *"set up a QA pipeline"*, *"make it prod ready"* or *"bug hunt until zero"*.
  99. It works on any language Claude Code can read. The gate gets sharper with the code-review-graph MCP (blast radius), `gitleaks`, `npm audit` and `pip-audit`.
  100. ## 💸 Built to waste nothing
  101. Most of an agent run's cost is the input it reads, re-sent on every turn. prodgate cuts input first:
  102. | | Saving | How |
  103. |---|---|---|
  104. | 📒 | **Ledger of coverage** | Each file is recorded with the commit it was checked at. A round reads only the diff, never-reviewed files and their callers |
  105. | 📌 | **Frozen commits** | Nobody hunts code that is still changing, so no finding goes stale before it is read |
  106. | 🧺 | **Batched checks** | One checker per file group settles all of that group's findings: each file is read once |
  107. | 🤐 | **Compact test output** | One line on a pass, only the failing lines on a fail |
  108. | 🪨 | **caveman + ponytail at ultra** | Required in every agent: terse output, the smallest complete change |
  109. | 🎚️ | **Risk tiers** | Low-risk machines are packed together and can run on a lighter model |
  110. ## 📊 Benchmark
  111. [`bench/`](bench/) is a public benchmark for state-bug finders: **8 realistic apps** (FastAPI + SQLAlchemy, plus TypeScript UI state), with **59 seeded bugs** across 8 bug classes and **24 decoys**, code that looks wrong but is correct. Every seeded bug was confirmed with a failing test, every decoy with a passing one, and every app's own test suite passes with the bugs in place.
  112. | Mode | What runs |
  113. |---|---|
  114. | 🤖 `oneshot` | A plain "review this codebase for bugs" agent per app |
  115. | 🧠 `prompt` | The state-machine prompt alone: one pass, nothing checked |
  116. | 🛡️ `prodgate` | The full pipeline, report-only |
  117. | 🔁 **re-check** | A small change lands in two apps, and each approach checks again |
  118. <img src="assets/results.svg" alt="Benchmark results: prodgate found 102 real bugs against 85 and 84; a re-check used 39% fewer tokens and gave 5 findings to read instead of 94" width="100%">
  119. | | 🤖 Plain review | 🧠 Prompt alone | 🛡️ prodgate |
  120. |---|---|---|---|
  121. | **Real bugs found** (distinct, each settled by a test) | 85 | 84 | **102** |
  122. | Seeded bugs found | 52/59 | **56/59** | 54/59 |
  123. | Every report ships with a failing test | ❌ | ❌ | ✅ |
  124. | 🔁 **Re-check after a 2-file change: tokens** | re-reads all 8 apps | 4.37M | **2.67M (−39%)** |
  125. | 🔁 **Re-check: findings to read** | everything again | 94 | **5 (−95%)** |
  126. | 💤 **Re-run on unchanged code** | full price again | 4.37M again | **0 tokens** |
  127. | First full run: tokens | **3.35M** | 3.87M | 35.0M |
  128. **The honest trade:** prodgate's first run costs about 10× a plain review, because it proves every finding with a test. It finds 20% more real bugs for that. After the first run, every re-check reads only what changed. Full method, per-class recall, raw data and limits are in [bench/RESULTS.md](bench/RESULTS.md).
  129. ## 🙏 Standing on the shoulders of caveman and ponytail
  130. prodgate's token discipline comes from two open-source projects. Every agent prodgate starts loads both at `ultra`, with no opt-out.
  131. <table>
  132. <tr>
  133. <td width="50%" align="center" valign="top">
  134. <a href="https://github.com/JuliusBrussee/caveman"><img src="https://raw.githubusercontent.com/JuliusBrussee/caveman/main/docs/assets/caveman-logo-banner.png" alt="caveman" width="320"></a>
  135. <br><br>
  136. <b><a href="https://github.com/JuliusBrussee/caveman">caveman</a></b> by <a href="https://github.com/JuliusBrussee">Julius Brussee</a>
  137. <br><br>
  138. Makes agents say less and read less. prodgate uses <code>ultra</code>, the strongest measured voice: <b>35% fewer output tokens than "be concise"</b>, by caveman's own measurement. When the caveman CLI is installed, prodgate's test runner also uses <code>caveman shrink</code>.
  139. </td>
  140. <td width="50%" align="center" valign="top">
  141. <a href="https://github.com/DietrichGebert/ponytail"><picture><source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/DietrichGebert/ponytail/main/assets/logo-dark.png"><img src="https://raw.githubusercontent.com/DietrichGebert/ponytail/main/assets/logo.png" alt="ponytail" width="220"></picture></a>
  142. <br><br>
  143. <b><a href="https://github.com/DietrichGebert/ponytail">ponytail</a></b> by <a href="https://github.com/DietrichGebert">Dietrich Gebert</a>
  144. <br><br>
  145. Makes agents write the smallest complete change. prodgate uses <code>ultra</code>, so fixers question the request first. By ponytail's own benchmark: <b>45% fewer tokens and 53% less code</b>, with 98% of risky logic shipping with a test.
  146. </td>
  147. </tr>
  148. </table>
  149. <sub>Logos belong to their projects and are shown here only to credit them. prodgate works with caveman and ponytail; it is not affiliated with or endorsed by either. caveman is Apache-2.0, ponytail is MIT.</sub>
  150. ## 🔬 Research
  151. prodgate's rules come from how Google, Meta, Amazon, Microsoft, the AIxCC winners, Jepsen and FoundationDB decide what counts as a bug. The short version: **executable proof, not LLM opinion, decides**. Read [the evidence and what was taken from it](docs/RESEARCH.md).
  152. ## 🗂️ Repo layout
  153. ```text
  154. skills/prodgate/ the skill: SKILL.md, prodgate.workflow.js, bin/ (ledger, qa-run, require-plugins)
  155. bench/ fixtures, answer keys, scorer, token counter, benchmark workflow
  156. docs/ how it works, research
  157. tests/ workflow control flow (fake agents), ledger and scorer tests
  158. cli/ the npm installer
  159. ```
  160. ## 🤝 Contributing
  161. New fixture apps with seeded state bugs are the most useful contribution: see [bench/SPEC.md](bench/SPEC.md). Bug reports that come with a failing test are welcome too.
  162. <div align="center">
  163. ---
  164. **⭐ If prodgate catches a bug before your users do, a star helps others find it.**
  165. MIT © ashusnapx
  166. </div>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.