Skip to content
Back to search

autoresearch

MarketplaceWorks with Claude CodeGitHub

Claude Autoresearch — autonomous goal-directed iteration for Claude Code. 14 commands: iterate, plan, debug, fix, security, ship, scenario, predict, learn, reason, probe, evals, improve, regression.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/uditgoenka-autoresearch/badge.svg)](/listings/uditgoenka-autoresearch)

HTML

<a href="/listings/uditgoenka-autoresearch"><img src="/api/listings/uditgoenka-autoresearch/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Claude Code

**Option A — npx install (recommended):**

npx skills add uditgoenka/autoresearch

All 14 commands are available after restarting Claude Code.

**Option B — Plugin install:**

/plugin marketplace add uditgoenka/autoresearch
/plugin install autoresearch@autoresearch

> **Note:** Start a new Claude Code session after installing. Reference files aren't resolvable in the same session where installation happened — this is a Claude Code platform limitation.

**Updating (no reinstall needed):**

/plugin update autoresearch

Run `/reload-plugins` to activate. No need to uninstall or re-clone.

**Option C — Manual copy:**

git clone https://github.com/uditgoenka/autoresearch.git

# Copy skill + subcommands to your project
cp -r autoresearch/.claude/skills/autoresearch .claude/skills/autoresearch
cp -r autoresearch/.claude/commands/autoresearch .claude/commands/autoresearch
cp autoresearch/.claude/commands/autoresearch.md .claude/commands/autoresearch.md

Or install globally:

cp -r autoresearch/.claude/skills/autoresearch ~/.claude/skills/autoresearch
cp -r autoresearch/.claude/commands/autoresearch ~/.claude/commands/autoresearch
cp autoresearch/.claude/commands/autoresearch.md ~/.claude/commands/autoresearch.md

**Option D — Guided installer:**

git clone https://github.com/uditgoenka/autoresearch.git
cd autoresearch
./scripts/install.sh --claude --global

OpenCode Quick Start

**Option A — Guided installer (recommended):**

git clone https://github.com/uditgoenka/autoresearch.git
cd autoresearch
./scripts/install.sh --opencode --global

**Option B — Manual copy:**

git clone https://github.com/uditgoenka/autoresearch.git

cp -r autoresearch/.opencode/skills/autoresearch .opencode/skills/autoresearch
cp autoresearch/.opencode/commands/autoresearch*.md .opencode/commands/

Or globally:

cp -r autoresearch/.opencode/skills/autoresearch ~/.config/opencode/skills/autoresearch
cp autoresearch/.opencode/commands/autoresearch*.md ~/.config/opencode/commands/

> All 14 commands available as `/autoresearch_debug`, `/autoresearch_fix`, `/autoresearch_improve`, etc.

Codex Quick Start

**Option A — Guided installer (recommended):**

git clone https://github.com/uditgoenka/autoresearch.git
cd autoresearch
./scripts/install.sh --codex --global

**Option B — Manual copy:**

git clone https://github.com/uditgoenka/autoresearch.git
cp -r autoresearch/.agents/skills/autoresearch ~/.codex/skills/autoresearch

> Invoke via `$autoresearch` mention syntax. Subcommands are keywords: `$autoresearch plan`, `$autoresearch debug`, `$autoresearch evals`, etc. > The installed Codex package includes the bundled orchestrator and regression helpers under `plugins/autoresearch/skills/autoresearch/` and `.agents/skills/autoresearch/`.

pi Quick Start

**Option A — Guided installer (recommended):**

git clone https://github.com/uditgoenka/autoresearch.git
cd autoresearch
./scripts/install.sh --pi --global

**Option B — pi package install:**

pi install git:github.com/uditgoenka/autoresearch

**Option C — Try without installing:**

pi -e ./pi-extension

> Invoke commands as pi prompt templates: `/autoresearch`, `/autoresearch_debug`, `/autoresearch_fix`, … (underscore names). The dispatcher is a skill: `/skill:autoresearch`. > The pi extension ports the Claude hook guardrails (scout/privacy/dangerous-cmd block, iteration context, simplify gate, stop-notify) to pi extension events. See [`pi-extension/README.md`](pi-extension/README.md) for the full hook→event mapping and `AR_DISABLE_*` flags.

Muse Quick Start

**Option A — Guided installer (recommended):**

git clone https://github.com/uditgoenka/autoresearch.git
cd autoresearch
./scripts/install.sh --muse --global

**Option B — Manual install with the Muse CLI:**

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 23 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
  3. "name": "autoresearch",
  4. "version": "2.3.0",
  5. "description": "Claude Autoresearch — autonomous goal-directed iteration for Claude Code. 14 commands: iterate, plan, debug, fix, security, ship, scenario, predict, learn, reason, probe, evals, improve, regression.",
  6. "owner": {
  7. "name": "Udit Goenka",
  8. "url": "https://github.com/uditgoenka"
  9. },
  10. "plugins": [
  11. {
  12. "name": "autoresearch",
  13. "description": "Autonomous improvement engine. 14 commands with bounded defaults, chain handoff, and adaptive evals checkpoints. 95% token reduction via modular architecture.",
  14. "version": "2.3.0",
  15. "author": {
  16. "name": "Udit Goenka",
  17. "url": "https://github.com/uditgoenka"
  18. },
  19. "source": "./claude-plugin",
  20. "category": "productivity"
  21. }
  22. ]
  23. }
README.md · README · 862 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <div align="center">
  2. # Autoresearch
  3. **Turn [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [OpenCode](https://opencode.ai), [OpenAI Codex](https://developers.openai.com/codex), [pi](https://github.com/earendil-works/pi-coding-agent), or Muse into a relentless improvement engine.**
  4. Based on [Karpathy's autoresearch](https://github.com/karpathy/autoresearch) — constraint + mechanical metric + autonomous iteration = compounding gains.
  5. [![Claude Code Skill](https://img.shields.io/badge/Claude_Code-Skill-blue?logo=anthropic&logoColor=white)](https://docs.anthropic.com/en/docs/claude-code)
  6. [![OpenCode](https://img.shields.io/badge/OpenCode-Skill-purple)](https://opencode.ai)
  7. [![Codex](https://img.shields.io/badge/Codex-Skill-green?logo=openai&logoColor=white)](https://developers.openai.com/codex)
  8. [![pi](https://img.shields.io/badge/pi-Extension-teal)](https://github.com/earendil-works/pi-coding-agent)
  9. [![Version](https://img.shields.io/badge/version-2.3.0-blue.svg)](https://github.com/uditgoenka/autoresearch/releases)
  10. [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
  11. [![Based on](https://img.shields.io/badge/Based_on-Karpathy's_Autoresearch-orange)](https://github.com/karpathy/autoresearch)
  12. [![Follow @iuditg](https://img.shields.io/badge/Follow-@iuditg-000000?style=flat&logo=x&logoColor=white)](https://x.com/intent/follow?screen_name=iuditg)
  13. [![Support](https://img.shields.io/badge/Support-PayPal-00457C?style=flat&logo=paypal&logoColor=white)](https://paypal.me/uditgoenka)
  14. <br>
  15. *"Set the GOAL → The agent runs the LOOP → You wake up to results"*
  16. *You don't need AGI. You need a goal, a metric, and a loop that never quits.*
  17. **Supports Claude Code, OpenCode, OpenAI Codex, pi, and Muse for the core skill, bundled runtime, installation, and verification surface. Hook guardrails ship for Claude Code (native hooks) and pi (extension events).**
  18. > **Fork-and-flush:** metric loops escape plateaus the way [Cai et al. (arXiv:2610.07447)](https://arxiv.org/abs/2610.07447) do. At 33% and 66% of the budget, or after 5 iterations with no `keep`, the loop forks fresh-context branches from the best commit so far, runs each briefly, and keeps the winner. Half the branches pick their own ideas (randomness); the other half each get a distinct experiment from a stateless Advisor that reads the results log ([Zhang et al., arXiv:2610.07625](https://arxiv.org/abs/2610.07625)); `Guided: R` sets the split. The orchestrator answers a plateau with `FORK` before it gives up. See [guide/autoresearch.md](guide/autoresearch.md#getting-unstuck-fork-and-flush).
  19. > **v2.2.0 — Autonomous Orchestrator:** Type a plain-language goal to `/autoresearch` and it classifies your goal, derives a Success predicate, confirms it once, then loops across subcommands until done. No manual chaining required. `Metric:`/`Verify:` invocations run the classic loop unchanged. See [guide/autoresearch-orchestrator.md](guide/autoresearch-orchestrator.md).
  20. <br>
  21. [How It Works](#how-it-works) · [Commands](#commands) · [Quick Start](#quick-start) · [Guides](guide/) · [FAQ](#faq)
  22. </div>
  23. ---
  24. ```
  25. PLAN LOOP DEBUG FIX SECURE SHIP
  26. ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
  27. │ Goal │ │ Modify │ │ Find │ │ Fix │ │ STRIDE │ │ Stage │
  28. │ Metric │────▶│ Verify │────▶│ Bugs │────▶│ Errors │────▶│ OWASP │────▶│ Deploy │
  29. │ Scope │ │Keep/Drop │ │ Trace │ │ Repair │ │ Red Team │ │ Release │
  30. └──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘
  31. /autoresearch: /autoresearch /autoresearch: /autoresearch: /autoresearch: /autoresearch:
  32. plan debug fix security ship
  33. ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
  34. │ Probe │ │ Scenario │ │ Predict │ │ Reason │
  35. │ Require- │ │ Edge │ │ 5-Expert │ │ Debate │
  36. │ ments │ │ Cases │ │ Swarm │ │ Converge │
  37. └──────────┘ └──────────┘ └──────────┘ └──────────┘
  38. /autoresearch: /autoresearch: /autoresearch: /autoresearch:
  39. probe scenario predict reason
  40. ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
  41. │ Learn │ │ Improve │ │ Eval │ │ Baseline │
  42. │ Docs │ │ Research │ │ Analyze │ │ Diff │
  43. │ Gen │ │ PRDs │ │ Results │ │ Verdict │
  44. └──────────┘ └──────────┘ └──────────┘ └──────────┘
  45. /autoresearch: /autoresearch: /autoresearch: /autoresearch:
  46. learn improve evals regression
  47. ```
  48. ---
  49. ## Why This Exists
  50. [Karpathy's autoresearch](https://github.com/karpathy/autoresearch) demonstrated that a 630-line Python script could autonomously improve ML models overnight — **100 experiments per night** — by following simple principles: one metric, constrained scope, fast verification, automatic rollback, git as memory.
  51. **Claude Autoresearch generalizes these principles to ANY domain.** Not just ML — code, content, marketing, sales, HR, DevOps, or anything with a number you can measure.
  52. **v2.1.0 is a major architecture rebuild.** The monolithic SKILL.md (813 lines, ~100K tokens per invocation) is replaced with a thin 41-line routing file and 12 self-contained command files (94–120 lines each, ~5–8K tokens per invocation). That is a **95% token reduction** with the same capability surface.
  53. ---
  54. ## How It Works
  55. ```
  56. LOOP (N iterations or until done):
  57. 1. Review current state + git history + results log
  58. 2. Pick the next change (based on what worked, what failed, what's untried)
  59. 3. Make ONE focused change
  60. 4. Git commit (before verification)
  61. 5. Run mechanical verification (tests, benchmarks, scores)
  62. 6. If improved → keep. If worse → git revert. If crashed → fix or skip.
  63. 7. Log the result
  64. 8. Repeat until N iterations complete or goal is met.
  65. ```
  66. Every improvement stacks. Every failure auto-reverts. Progress is logged in TSV format.
  67. ### The Setup Phase
  68. Before looping, Claude performs a one-time setup:
  69. 1. **Read context** — reads all in-scope files
  70. 2. **Define goal** — extracts or asks for a mechanical metric
  71. 3. **Define scope** — which files can be modified vs read-only
  72. 4. **Establish baseline** — runs verification on current state (iteration #0)
  73. 5. **Confirm and go** — shows setup, then begins the loop
  74. ### 8 Critical Rules
  75. | # | Rule |
  76. |---|------|
  77. | 1 | **Bounded by default** — every command has a default iteration count; unlimited is opt-in via `Iterations: unlimited` |
  78. | 2 | **Read before write** — understand full context before modifying |
  79. | 3 | **One change per iteration** — atomic changes; if it breaks, you know why |
  80. | 4 | **Mechanical search signal, independent acceptance** — no subjective "looks good"; the optimized metric never certifies behavior preservation |
  81. | 5 | **Automatic rollback** — failed changes revert instantly |
  82. | 6 | **Simplicity wins** — equal results + less code = keep |
  83. | 7 | **Git is memory** — experiments committed with `experiment:` prefix; agent reads `git log` + `git diff` before each iteration |
  84. | 8 | **When stuck, think harder** — re-read, combine near-misses, try radical changes |
  85. ---
  86. ## Hooks & Safety
  87. Hooks are defense-in-depth guardrails, not a security sandbox. Claude Code ships the hook surface natively; pi ships the same guardrails as extension events (`tool_call`, `before_agent_start`, `input`, `session_start`, `session_shutdown`). OpenCode, Codex, and Muse ship the core skill/runtime/install surface without hook parity.
  88. ### What's Protected
  89. | Hook | What it does | Event |
  90. |------|-------------|-------|
  91. | **scout-block** | Blocks node_modules/, .git/, __pycache__/, etc. from filling your context | PreToolUse |
  92. | **privacy-block** | Blocks .env, SSH keys, credentials from being read in sessions | PreToolUse |
  93. | **dangerous-cmd-block** | Blocks force-push, `rm -rf`, `git reset --hard` | PreToolUse |
  94. | **iteration-context** | Injects recent TSV iteration data after context compaction | UserPromptSubmit |
  95. | **subagent-context** | Gives subagents awareness of active loop state | SubagentStart |
  96. | **dev-rules-reminder** | Re-injects plan path and code standards after compaction | UserPromptSubmit |
  97. | **simplify-gate** | Warns at 400 LOC, blocks at 800 LOC before shipping | UserPromptSubmit |
  98. | **session-init** | Sets up project context at session start | SessionStart |
  99. | **stop-notify** | Terminal notification + optional webhook on session end | SessionEnd |
  100. ### Configuration
  101. All hooks are **on by default**. Disable individually only for troubleshooting:
  102. ```bash
  103. # Disable a specific hook
  104. export AR_DISABLE_SCOUT_BLOCK=1
  105. export AR_DISABLE_PRIVACY_BLOCK=1
  106. export AR_DISABLE_DANGEROUS_CMD_BLOCK=1
  107. # ... etc for each hook name
  108. ```
  109. Optional webhook for session completion notifications:
  110. ```bash
  111. export AR_NOTIFY_WEBHOOK=https://hooks.slack.com/services/...
  112. ```
  113. Customize blocked directories with a `.ckignore` file (gitignore syntax) at your project root.
  114. See [guide/hooks.md](guide/hooks.md) for full reference.
  115. Release preparation and contributor verification live in [scripts/release.md](scripts/release.md).
  116. ---
  117. ## Commands
  118. | Command | What it does | Default Iterations |
  119. |---------|--------------|--------------------|
  120. | `/autoresearch` | **Classic:** Core iterate loop: modify → verify → keep/discard · **Orchestrator:** free-form goal → auto-select pipeline → loop until predicate met | 25 / goal-bounded |
  121. | `/autoresearch:plan` | Convert goal into validated config | one-shot |
  122. | `/autoresearch:debug` | Hunt bugs via hypothesis iteration | 15 |
  123. | `/autoresearch:fix` | Crush errors one-by-one to zero | 20 |
  124. | `/autoresearch:security` | STRIDE + OWASP audit with red-team | 15 |
  125. | `/autoresearch:ship` | Ship through 8 phases | linear |
  126. | `/autoresearch:scenario` | Generate edge cases across 12 dimensions | 20 |
  127. | `/autoresearch:predict` | 5 expert personas debate | one-shot |
  128. | `/autoresearch:learn` | Scout → generate docs → validate → fix | 10 |
  129. | `/autoresearch:reason` | Adversarial debate with blind judges | 8 |
  130. | `/autoresearch:probe` | 8 personas interrogate requirements | 15 |
  131. | `/autoresearch:improve` | Research ICP, discover improvements, generate PRDs | 15 |
  132. | `/autoresearch:evals` | Analyze iteration results: trends, plateaus | one-shot |
  133. | `/autoresearch:regression` | Stability gate: baseline vs candidate, verdict STABLE/UNSTABLE | one-shot |
  134. **Universal flags:** `Iterations: N`, `Iterations: unlimited`, `--evals`, `--evals-interval N`, `--chain <targets>`, `--<subcommand>` shorthand.
  135. **All commands use interactive setup when invoked without arguments.** Just type the command — the agent asks for what it needs with smart defaults based on your codebase.
  136. > **OpenCode users:** Commands use underscore naming (`/autoresearch_debug`, `/autoresearch_fix`, etc.). All 14 commands available.
  137. >
  138. > **Codex users:** Invoke via `$autoresearch` mention syntax. Subcommands are keywords: `$autoresearch debug`, `$autoresearch plan`, etc.
  139. ### Quick Decision Guide
  140. | I want to... | Use |
  141. |--------------|-----|
  142. | Give a plain-language goal, let it self-orchestrate | `/autoresearch <goal>` (bare, no Metric/Verify) |
  143. | Improve test coverage / reduce bundle size / any metric | `/autoresearch` |
  144. | Run bounded iterations | Add `Iterations: N` to any command |
  145. | Don't know what metric to use | `/autoresearch:plan` |
  146. | Run a security audit | `/autoresearch:security` |
  147. | Ship a PR / deployment / release | `/autoresearch:ship` |
  148. | Optimize without breaking existing tests | Add `Guard: npm test` |
  149. | Hunt all bugs in a codebase | `/autoresearch:debug` |
  150. | Fix all errors (tests, types, lint) | `/autoresearch:fix` |
  151. | Debug then auto-fix | `/autoresearch:debug --fix` |
  152. | Check if something is ready to ship | `/autoresearch:ship --checklist-only` |
  153. | Explore edge cases for a feature | `/autoresearch:scenario` |
  154. | Generate test scenarios | `/autoresearch:scenario --format test-scenarios` |
  155. | Get expert opinions before starting | `/autoresearch:predict` |
  156. | Analyze from multiple angles then debug | `/autoresearch:predict --chain debug` |
  157. | Generate docs for a new codebase | `/autoresearch:learn --mode init` |
  158. | Update existing docs after changes | `/autoresearch:learn --mode update` |
  159. | Debate an architecture decision | `/autoresearch:reason --domain software` |
  160. | Surface hidden constraints before starting | `/autoresearch:probe` |
  161. | Pre-flight a fuzzy goal then loop | `/autoresearch:probe --chain plan,autoresearch` |
  162. | Discover what to build next for your ICP | `/autoresearch:improve` |
  163. | Research competitors and generate PRDs | `/autoresearch:improve --depth deep` |
  164. | Probe requirements then research improvements | `/autoresearch:probe --improve` |
  165. | Analyze trends and plateaus across past runs | `/autoresearch:evals` |
  166. | Check if a run has stalled | `/autoresearch:evals --file *-results.tsv` |
  167. | Verify a change won't regress before pushing | `/autoresearch:regression` |
  168. | Gate a PR: predict, fix, re-gate, then ship | `/autoresearch:regression --predict --fix --ship` |
  169. ---
  170. ## Quick Start
  171. ### Claude Code
  172. **Option A — npx install (recommended):**
  173. ```bash
  174. npx skills add uditgoenka/autoresearch
  175. ```
  176. All 14 commands are available after restarting Claude Code.
  177. **Option B — Plugin install:**
  178. ```
  179. /plugin marketplace add uditgoenka/autoresearch
  180. /plugin install autoresearch@autoresearch
  181. ```
  182. > **Note:** Start a new Claude Code session after installing. Reference files aren't resolvable in the same session where installation happened — this is a Claude Code platform limitation.
  183. **Updating (no reinstall needed):**
  184. ```
  185. /plugin update autoresearch
  186. ```
  187. Run `/reload-plugins` to activate. No need to uninstall or re-clone.
  188. **Option C — Manual copy:**
  189. ```bash
  190. git clone https://github.com/uditgoenka/autoresearch.git
  191. # Copy skill + subcommands to your project
  192. cp -r autoresearch/.claude/skills/autoresearch .claude/skills/autoresearch
  193. cp -r autoresearch/.claude/commands/autoresearch .claude/commands/autoresearch
  194. cp autoresearch/.claude/commands/autoresearch.md .claude/commands/autoresearch.md
  195. ```
  196. Or install globally:
  197. ```bash
  198. cp -r autoresearch/.claude/skills/autoresearch ~/.claude/skills/autoresearch
  199. cp -r autoresearch/.claude/commands/autoresearch ~/.claude/commands/autoresearch
  200. cp autoresearch/.claude/commands/autoresearch.md ~/.claude/commands/autoresearch.md
  201. ```
  202. **Option D — Guided installer:**
  203. ```bash
  204. git clone https://github.com/uditgoenka/autoresearch.git
  205. cd autoresearch
  206. ./scripts/install.sh --claude --global
  207. ```
  208. ### OpenCode Quick Start
  209. **Option A — Guided installer (recommended):**
  210. ```bash
  211. git clone https://github.com/uditgoenka/autoresearch.git
  212. cd autoresearch
  213. ./scripts/install.sh --opencode --global
  214. ```
  215. **Option B — Manual copy:**
  216. ```bash
  217. git clone https://github.com/uditgoenka/autoresearch.git
  218. cp -r autoresearch/.opencode/skills/autoresearch .opencode/skills/autoresearch
  219. cp autoresearch/.opencode/commands/autoresearch*.md .opencode/commands/
  220. ```
  221. Or globally:
  222. ```bash
  223. cp -r autoresearch/.opencode/skills/autoresearch ~/.config/opencode/skills/autoresearch
  224. cp autoresearch/.opencode/commands/autoresearch*.md ~/.config/opencode/commands/
  225. ```
  226. > All 14 commands available as `/autoresearch_debug`, `/autoresearch_fix`, `/autoresearch_improve`, etc.
  227. ### Codex Quick Start
  228. **Option A — Guided installer (recommended):**
  229. ```bash
  230. git clone https://github.com/uditgoenka/autoresearch.git
  231. cd autoresearch
  232. ./scripts/install.sh --codex --global
  233. ```
  234. **Option B — Manual copy:**
  235. ```bash
  236. git clone https://github.com/uditgoenka/autoresearch.git
  237. cp -r autoresearch/.agents/skills/autoresearch ~/.codex/skills/autoresearch
  238. ```
  239. > Invoke via `$autoresearch` mention syntax. Subcommands are keywords: `$autoresearch plan`, `$autoresearch debug`, `$autoresearch evals`, etc.
  240. > The installed Codex package includes the bundled orchestrator and regression helpers under `plugins/autoresearch/skills/autoresearch/` and `.agents/skills/autoresearch/`.
  241. ### pi Quick Start
  242. **Option A — Guided installer (recommended):**
  243. ```bash
  244. git clone https://github.com/uditgoenka/autoresearch.git
  245. cd autoresearch
  246. ./scripts/install.sh --pi --global
  247. ```
  248. **Option B — pi package install:**
  249. ```bash
  250. pi install git:github.com/uditgoenka/autoresearch
  251. ```
  252. **Option C — Try without installing:**
  253. ```bash
  254. pi -e ./pi-extension
  255. ```
  256. > Invoke commands as pi prompt templates: `/autoresearch`, `/autoresearch_debug`, `/autoresearch_fix`, … (underscore names). The dispatcher is a skill: `/skill:autoresearch`.
  257. > The pi extension ports the Claude hook guardrails (scout/privacy/dangerous-cmd block, iteration context, simplify gate, stop-notify) to pi extension events. See [`pi-extension/README.md`](pi-extension/README.md) for the full hook→event mapping and `AR_DISABLE_*` flags.
  258. ### Muse Quick Start
  259. **Option A — Guided installer (recommended):**
  260. ```bash
  261. git clone https://github.com/uditgoenka/autoresearch.git
  262. cd autoresearch
  263. ./scripts/install.sh --muse --global
  264. ```
  265. **Option B — Manual install with the Muse CLI:**
  266. ```bash
  267. git clone https://github.com/uditgoenka/autoresearch.git
  268. muse skills install autoresearch/.muse/skills/autoresearch --scope user
  269. ```
  270. Or project-local, so every Muse session in that project picks it up. Run it from your project folder:
  271. ```bash
  272. cd /path/to/your-project
  273. bash /path/to/autoresearch/scripts/install.sh --muse --local
  274. ```
  275. > Invoke by asking Muse to use the autoresearch skill with a subcommand keyword: `autoresearch plan`, `autoresearch debug`, `autoresearch fix`, etc.
  276. > The installed Muse package includes the bundled orchestrator and regression helpers under `.muse/skills/autoresearch/`.
  277. ### Run It
  278. ```
  279. /autoresearch
  280. Goal: Increase test coverage from 72% to 90%
  281. Scope: src/**/*.test.ts, src/**/*.ts
  282. Metric: coverage % (higher is better)
  283. Verify: npm test -- --coverage | grep "All files"
  284. Iterations: 25
  285. ```
  286. Claude reads all files, establishes a baseline, and starts iterating — one change at a time. Keeps improvements, auto-reverts failures, logs everything. Stops after N iterations or when you interrupt.
  287. ---
  288. ## /autoresearch:plan — Goal to Config
  289. The hardest part isn't the loop — it's defining Scope, Metric, and Verify correctly. `/autoresearch:plan` converts your plain-language goal into a validated, ready-to-execute configuration.
  290. ```
  291. /autoresearch:plan
  292. Goal: Make the API respond faster
  293. ```
  294. Walks through 5 steps: capture goal → define scope → define metric → define direction → validate verify command (dry-run). Every gate is mechanical — scope must resolve to files, metric must output a number, verify must pass a dry-run. Emits a `handoff.json` for chaining.
  295. ---
  296. ## /autoresearch:debug — Autonomous Bug Hunter
  297. Scientific method meets autoresearch loop. Doesn't stop at one bug — iteratively hunts ALL bugs using falsifiable hypotheses, evidence-based investigation, and 7 investigation techniques.
  298. ```
  299. /autoresearch:debug
  300. Scope: src/api/**/*.ts
  301. Symptom: API returns 500 on POST /users
  302. Iterations: 15
  303. ```
  304. **How it works:** Gather symptoms → Recon → Hypothesize (specific, testable) → Test (one experiment per iteration) → Classify (confirmed/disproven/inconclusive) → Log → Repeat.
  305. Every finding requires code evidence (file:line + reproduction steps). Every disproven hypothesis is logged — equally valuable.
  306. | Flag | Purpose |
  307. |------|---------|
  308. | `--fix` | After hunting, auto-switch to `/autoresearch:fix` |
  309. | `--scope <glob>` | Limit investigation scope |
  310. | `--symptom "<text>"` | Pre-fill symptom |
  311. | `--severity <level>` | Minimum severity to report |
  312. ---
  313. ## /autoresearch:fix — Autonomous Error Crusher
  314. Takes a broken state and iteratively repairs it until everything passes. ONE fix per iteration. Atomic, committed, verified, auto-reverted on failure.
  315. ```
  316. /autoresearch:fix
  317. Iterations: 20
  318. ```
  319. Auto-detects what's broken (tests, types, lint, build) → Prioritizes (blockers first) → Fixes ONE thing → Commits → Verifies error count decreased → Guard check → Keep/Revert → Repeat. **Stops automatically when error count hits zero.**
  320. | Flag | Purpose |
  321. |------|---------|
  322. | `--target <command>` | Explicit verify command |
  323. | `--guard <command>` | Safety command that must always pass |
  324. | `--category <type>` | Only fix specific type (test, type, lint, build) |
  325. | `--from-debug` | Read findings from latest debug session |
  326. **Chain them:** `/autoresearch:debug` → `/autoresearch:fix --from-debug`
  327. ---
  328. ## /autoresearch:security — Autonomous Security Audit
  329. Read-only security audit using STRIDE threat modeling, OWASP Top 10 sweeps, and red-team adversarial analysis with 4 hostile personas.
  330. ```
  331. /autoresearch:security
  332. Iterations: 15
  333. ```
  334. Codebase recon → asset inventory → trust boundaries → STRIDE threat model → attack surface map → autonomous testing loop → structured report. Every finding requires code evidence (file:line + attack scenario).
  335. | Flag | Purpose |
  336. |------|---------|
  337. | `--diff` | Only audit files changed since last audit |
  338. | `--fix` | Auto-fix confirmed Critical/High findings |
  339. | `--fail-on <severity>` | Exit non-zero for CI/CD gating |
  340. **Output:** Creates `security/{date}-{slug}/` with 7 structured report files.
  341. ---
  342. ## /autoresearch:ship — Universal Shipping Workflow
  343. Ship anything through 8 phases: **Identify → Inventory → Checklist → Prepare → Dry-run → Ship → Verify → Log.**
  344. ```
  345. /autoresearch:ship --auto
  346. ```
  347. Auto-detects what you're shipping (code PR, deployment, blog post, email campaign, sales deck, research paper, design assets) and generates domain-specific checklists — every item mechanically verifiable.
  348. | Flag | Purpose |
  349. |------|---------|
  350. | `--dry-run` | Validate everything but don't ship |
  351. | `--auto` | Auto-approve if checklist passes |
  352. | `--force` | Skip non-critical items (blockers still enforced) |
  353. | `--rollback` | Undo last ship action |
  354. | `--monitor N` | Post-ship monitoring for N minutes |
  355. | `--checklist-only` | Just check readiness |
  356. **9 supported types:** code-pr, code-release, deployment, content, marketing-email, marketing-campaign, sales, research, design.
  357. ---
  358. ## /autoresearch:scenario — Scenario Explorer
  359. Autonomous scenario exploration engine. Takes a seed scenario and iteratively generates situations across 12 dimensions — happy paths, errors, edge cases, abuse, scale, concurrency, temporal, data variation, permissions, integrations, recovery, and state transitions.
  360. ```
  361. /autoresearch:scenario
  362. Scenario: User attempts to checkout with multiple payment methods
  363. Iterations: 20
  364. ```
  365. Seed analysis → Decompose into 12 dimensions → Generate ONE situation per iteration → Classify (new/variant/duplicate) → Expand edge cases → Log → Repeat.
  366. | Flag | Purpose |
  367. |------|---------|
  368. | `--domain <type>` | software, product, business, security, marketing |
  369. | `--depth <level>` | shallow (10), standard (20), deep (50+) |
  370. | `--format <type>` | use-cases, user-stories, test-scenarios, threat-scenarios |
  371. | `--focus <area>` | edge-cases, failures, security, scale |
  372. ---
  373. ## /autoresearch:predict — Multi-Persona Prediction
  374. Before you debug, fix, or ship — get 5 expert perspectives in 2 minutes.
  375. Simulates a team (Architect, Security Analyst, Performance Engineer, Reliability Engineer, Devil's Advocate) who independently analyze your code, debate findings, and reach consensus.
  376. ```
  377. /autoresearch:predict --chain debug
  378. ```
  379. - `--chain debug` — pre-ranked hypotheses before debugging
  380. - `--chain security` — multi-persona red team analysis
  381. - `--chain scenario,debug,fix` — full quality pipeline
  382. ---
  383. ## /autoresearch:learn — Autonomous Documentation Engine
  384. Scout codebase → generate docs → validate → fix → repeat. 4 modes: init (create from scratch), update (refresh existing), check (read-only health report), summarize (quick overview).
  385. ```
  386. /autoresearch:learn --mode init --depth deep
  387. Iterations: 10
  388. ```
  389. Dynamic doc discovery, project-type detection, validation-fix loop, git-diff scoping for updates, selective single-doc update with `--file`. Auto-generates Mermaid architecture diagrams, API reference, testing guide, config guide, and cross-reference links.
  390. ---
  391. ## /autoresearch:reason — Adversarial Refinement
  392. Extends autoresearch to **subjective domains** where no objective metric exists. The blind judge panel is the fitness function.
  393. ```
  394. /autoresearch:reason
  395. Task: Should we use event sourcing for our order management system?
  396. Domain: software
  397. Iterations: 8
  398. ```
  399. **How it works:** Generate-A → Critic attacks → Author-B responds → Synthesizer merges → Blind judge panel (randomized labels) picks winner → Winner becomes new A → Repeat until convergence. Every agent is a cold-start fresh invocation — no history bleed.
  400. | Flag | Purpose |
  401. |------|---------|
  402. | `--judges N` | Judge count (3-7, odd preferred) |
  403. | `--convergence N` | Consecutive wins to converge (default 3) |
  404. | `--mode <mode>` | convergent (default), creative, debate |
  405. | `--domain <type>` | software, product, business, security, research, content |
  406. | `--chain <targets>` | Chain converged output to any autoresearch command |
  407. **Output:** Creates `reason/{date}-{slug}/` with lineage.md, candidates.md, judge-transcripts.md, reason-results.tsv, handoff.json.
  408. ---
  409. ## /autoresearch:probe — Adversarial Requirement Interrogation
  410. Eight adversarial personas interrogate user and codebase together until net-new constraints saturate. Output is the 5 autoresearch primitives (Goal/Scope/Metric/Direction/Verify) plus a `handoff.json` ready to feed any downstream command.
  411. ```
  412. /autoresearch:probe --chain plan,autoresearch
  413. Topic: Add multi-tenant isolation to the database layer
  414. ```
  415. **The 8 personas:** Skeptic, Edge-Case Hunter, Scope Sentinel, Ambiguity Detective, Contradiction Finder, Prior-Art Investigator, Success-Criteria Auditor, Constraint Excavator.
  416. | Flag | Purpose |
  417. |------|---------|
  418. | `--depth <level>` | shallow (5 rounds), standard (15), deep (30) |
  419. | `--adversarial` | Rotate Skeptic + Contradiction Finder + Edge-Case Hunter to front |
  420. | `--mode <mode>` | interactive (default) or autonomous |
  421. | `--chain <targets>` | plan, predict, debug, scenario, reason, fix, ship, learn |
  422. **Output:** Creates `probe/{date}-{slug}/` with probe-spec.md, constraints.tsv, autoresearch-config.yml, handoff.json.
  423. ---
  424. ## /autoresearch:improve — Product Improvement Engine
  425. Research what to build next. Discovers ICP challenges via deep multi-source research, scores and ranks improvements, generates per-feature PRDs with evidence chains.
  426. ```
  427. /autoresearch:improve
  428. Goal: Improve onboarding conversion
  429. ICP: B2B SaaS product managers at 50-500 person companies
  430. ```
  431. **How it works:** Resolve product context → Research across 5 categories (ICP challenges, competitor gaps, market trends, UX & experience, revenue & growth) → Saturate → ICP binary gate → Tiered ranking (Must-have / Nice-to-have / Moonshot) → User selects features → Generate PRDs.
  432. | Flag | Purpose |
  433. |------|---------|
  434. | `--icp "<text>"` | Ideal customer profile |
  435. | `--discover` | Force codebase scan even with existing context |
  436. | `--no-discover` | Skip auto-discover |
  437. | `--depth <level>` | shallow (5), standard (15), deep (30+) |
  438. | `--seeds <categories>` | Override default research categories |
  439. **Output:** Creates `improve/{date}-{slug}/` with research-findings.md, improvement-plan.md, per-feature PRDs, summary.md, improve-results.tsv, handoff.json.
  440. **Terminal emitter** — improve is the last link in any autoresearch chain. PRDs are consumed by external tools (`/ck:plan`, `/ck:cook`), not by other autoresearch commands.
  441. **Chain into improve:** `/autoresearch:probe --improve`, `/autoresearch:predict --improve`, `/autoresearch:debug --improve`.
  442. ---
  443. ## /autoresearch:evals — Results Analyzer
  444. Analyzes `*-results.tsv` files from any autoresearch run. Surfaces trends, plateau detection, convergence signals, and iteration efficiency. Backward compatible with v2.0.x TSV format.
  445. ```
  446. /autoresearch:evals
  447. /autoresearch:evals --file coverage-results.tsv
  448. ```
  449. **Adaptive checkpoints:** floor(max_iterations/3), minimum 1 checkpoint. Reports per-checkpoint delta, stall detection, best iteration, and a recommendation (continue / stop / change strategy).
  450. **Inline evals during a run:**
  451. ```
  452. /autoresearch
  453. Goal: Reduce bundle size below 200kb
  454. Iterations: 30
  455. --evals-interval 10
  456. ```
  457. Prints a checkpoint report every 10 iterations without interrupting the loop.
  458. ---
  459. ## /autoresearch:regression — Stability Gate
  460. Before you push, prove the change didn't break what already worked. Captures baseline behavior from a `git worktree` of the base ref, diffs the candidate across **8 dimensions**, and emits a single **STABLE / UNSTABLE** verdict.
  461. ```
  462. /autoresearch:regression --predict --evals --fix --ship
  463. ```
  464. **Core invariant:** a regression is a **green→red transition only**. Pre-existing failures (red→red), new tests (absent→red), and flaky tests (flake→red) are classified and excluded — never counted as regressions.
  465. **Tiered verdict:**
  466. - **HARD gate** (any green→red = UNSTABLE): `functional`, `api-contract`, `data-migration`, `integration-e2e`
  467. - **SCORE** (0–100, noise-tolerant, weighted; UNSTABLE below threshold 95): `flakiness` .30, `performance` .30, `resource` .20, `visual-ui` .20
  468. | Flag | Purpose |
  469. |------|---------|
  470. | `--select auto` | Use detected affected-test mapper (jest `--findRelatedTests`, nx affected) else FULL suite — never a silent subset |
  471. | `--samples N` / `--noise-band %` | Tune the perf statistical gate (default 7 samples/side, Mann–Whitney U) |
  472. | `--fix` / `--fix-cycles N` | Re-gate after fixing; each cycle must strictly shrink the blocking-set (max 3) |
  473. | `--predict` | Pre-empt likely regressions before the gate runs |
  474. | `--reason` | Adversarial root-cause when a regression's cause is ambiguous |
  475. | `--debug` | Force the bisect Hunter (HARD dims passing 3/3 reproduction) |
  476. | `--max-runs N` | Ceiling on dims×axes×samples×cells (warn+confirm past 200) |
  477. **Output:** Creates `regression/{date}-{slug}/` with regression-results.tsv, stability-report.md, dimensions/<dim>.md, baseline/, evals-summary.md, handoff.json.
  478. > **data-migration is hard-guarded:** opt-in, and refuses any DB URL that isn't ephemeral/allowlisted (`*test*`, `*ci*`, container). Migrations are forward-only by default.
  479. ---
  480. ## Guard — Prevent Regressions
  481. When optimizing a metric, the loop might break existing behavior. **Guard** is an optional safety net.
  482. ```
  483. /autoresearch
  484. Goal: Reduce API response time to under 100ms
  485. Verify: npm run bench:api | grep "p95"
  486. Guard: npm test
  487. ```
  488. - **Verify** = "Did the metric improve?" (the goal)
  489. - **Guard** = "Did anything else break?" (the safety net)
  490. If the metric improves but the guard fails, Claude reworks the optimization (up to 2 attempts). Guard/test files are never modified.
  491. > **Credit:** Guard was contributed by [@pronskiy](https://github.com/pronskiy) (JetBrains) in [PR #7](https://github.com/uditgoenka/autoresearch/pull/7).
  492. ---
  493. ## Results Tracking
  494. Every iteration is logged in TSV format:
  495. ```tsv
  496. iteration commit metric delta status description
  497. 0 a1b2c3d 85.2 0.0 baseline initial state
  498. 1 b2c3d4e 87.1 +1.9 keep add tests for auth edge cases
  499. 2 - 86.5 -0.6 discard refactor test helpers (broke 2 tests)
  500. 3 c3d4e5f 88.3 +1.2 keep add error handling tests
  501. ```
  502. Run `/autoresearch:evals` at any time to analyze trends across any TSV file. Adaptive checkpoints fire at floor(max_iterations/3) intervals.
  503. ---
  504. ## Crash Recovery
  505. | Failure | Response |
  506. |---------|----------|
  507. | Syntax error | Fix immediately, don't count as iteration |
  508. | Runtime error | Attempt fix (max 3 tries), then move on |
  509. | Resource exhaustion | Revert, try smaller variant |
  510. | Infinite loop / hang | Kill after timeout, revert |
  511. | External dependency | Skip, log, try different approach |
  512. ---
  513. ## Repository Structure
  514. ```
  515. autoresearch/
  516. ├── README.md
  517. ├── COMPARISON.md ← Karpathy's vs Claude Autoresearch
  518. ├── guide/ ← Guides — one per command + advanced patterns
  519. ├── scripts/
  520. │ ├── install.sh ← Guided installer (Claude Code + OpenCode + Codex + Muse)
  521. │ ├── transform.sh ← Single transform: .claude/ → .opencode/ + .agents/ + .muse/
  522. │ ├── release.sh ← Release automation
  523. │ └── release.md ← Release checklist
  524. ├── .claude/
  525. │ ├── skills/autoresearch/
  526. │ │ ├── SKILL.md ← Thin routing table (41 lines)
  527. │ │ └── references/ ← 3 focused reference files
  528. │ │ ├── security-checklist.md ← STRIDE + OWASP
  529. │ │ ├── predict-personas.md ← 5 personas + adversarial set
  530. │ │ └── reason-judge-protocol.md ← Adversarial refinement loop
  531. │ └── commands/
  532. │ ├── autoresearch.md ← Core loop (self-contained, ~100 lines)
  533. │ └── autoresearch/ ← 13 subcommand files (self-contained)
  534. │ ├── plan.md
  535. │ ├── debug.md
  536. │ ├── fix.md
  537. │ ├── security.md
  538. │ ├── ship.md
  539. │ ├── scenario.md
  540. │ ├── predict.md
  541. │ ├── learn.md
  542. │ ├── reason.md
  543. │ ├── improve.md
  544. │ ├── probe.md
  545. │ ├── evals.md
  546. │ └── regression.md
  547. ├── .opencode/ ← OpenCode port (via transform.sh)
  548. │ ├── skills/autoresearch/
  549. │ └── commands/ ← 14 command files (autoresearch_*.md)
  550. ├── .agents/ ← Codex port (via transform.sh)
  551. │ └── skills/autoresearch/
  552. ├── .muse/ ← Muse port (via transform.sh)
  553. │ └── skills/autoresearch/
  554. ├── plugins/ ← Codex plugin metadata
  555. │ └── openai.yaml
  556. └── LICENSE
  557. ```
  558. ---
  559. ## FAQ
  560. **Q: I don't know what metric to use.**
  561. A: Run `/autoresearch:plan` — it analyzes your codebase, suggests metrics, and dry-runs the verify command before you launch.
  562. **Q: What changed in v2.2.0?**
  563. A: The root `/autoresearch` command now supports an autonomous orchestrator mode. Type a plain-language goal (e.g., `/autoresearch help me fix the login bug`) instead of `Metric:`/`Verify:` and the orchestrator classifies your goal, derives a verifiable Success predicate, confirms it once, then loops across subcommands until done. Classic metric-loop behavior is unchanged when `Metric:` or `Verify:` are present.
  564. **Q: What changed in v2.1.0?**
  565. A: Architecture rebuild. The monolithic SKILL.md (813 lines, ~100K tokens) is replaced with a thin routing file + 12 self-contained command files (~5–8K tokens each). 95% token reduction. A new `/autoresearch:evals` command analyzes iteration results. Every looping command now has a bounded default instead of running unlimited.
  566. **Q: How do bounded defaults work?**
  567. A: Every looping command ships with a sensible default (e.g., `/autoresearch` defaults to 25 iterations). Override inline: `Iterations: 50` for more, `Iterations: unlimited` for the old unbounded behavior.
  568. **Q: How does /autoresearch:evals work?**
  569. A: Point it at any `*-results.tsv` file from a previous run. It reports trends, plateau detection, and a recommendation. Use `--evals-interval N` during a live run to get checkpoint reports without interrupting the loop.
  570. **Q: Does this work with any project?**
  571. A: Yes. Any language, framework, or domain. Install via plugin (Claude Code), installer script, or manual copy.
  572. **Q: Does this work with OpenCode?**
  573. A: Yes. Run `./scripts/install.sh --opencode --global` or manually copy `.opencode/` files. Commands use underscore naming (`/autoresearch_debug`, `/autoresearch_evals`, etc.). All 14 commands available.
  574. **Q: Does this work with OpenAI Codex?**
  575. A: Yes. Run `./scripts/install.sh --codex --global` or copy `.agents/skills/autoresearch/` to `~/.codex/skills/autoresearch`. Invoke via `$autoresearch` mention syntax.
  576. **Q: Does this work with Muse?**
  577. A: Yes. Run `./scripts/install.sh --muse --global`, or `muse skills install autoresearch/.muse/skills/autoresearch --scope user`. Ask Muse to use the autoresearch skill with a subcommand keyword (`autoresearch plan`, `autoresearch debug`, etc.). Hook guardrails are Claude Code-only.
  578. **Q: How do I stop the loop?**
  579. A: `Ctrl+C` or add `Iterations: N` to your inline config. Claude commits before verifying, so your last successful state is always in git.
  580. **Q: Can I use this for non-code tasks?**
  581. A: Absolutely. Sales emails, marketing copy, HR policies, runbooks — anything with a measurable metric. See [Examples by Domain](guide/examples-by-domain.md).
  582. **Q: Does /autoresearch:security modify my code?**
  583. A: No. Read-only by default. Use `--fix` to opt into auto-remediation of confirmed Critical/High findings.
  584. **Q: What's the difference between /autoresearch:predict and /autoresearch:reason?**
  585. A: Predict is a one-shot analysis — 5 experts debate your existing code. Reason is an iterative refinement loop — competing candidates are generated, critiqued, synthesized, and blind-judged over multiple rounds until convergence. Use predict for analysis before acting; use reason for decisions where no objective metric exists.
  586. **Q: What is handoff.json?**
  587. A: A structured file emitted by plan, probe, reason, and other commands that carries Goal/Scope/Metric/Verify config for downstream commands. When you `--chain plan,autoresearch`, the chain reads handoff.json automatically.
  588. ---
  589. ## Contributing
  590. Contributions welcome. See [CONTRIBUTING.md](CONTRIBUTING.md).
  591. Areas of interest: new domain examples, verification script templates, CI/CD integrations, real-world benchmarks. All guides are in [guide/](guide/).
  592. ---
  593. ## Star History
  594. <a href="https://www.star-history.com/?repos=uditgoenka%2Fautoresearch&type=timeline&legend=top-left">
  595. <picture>
  596. <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/image?repos=uditgoenka/autoresearch&type=timeline&theme=dark&legend=bottom-right&v=20260319" />
  597. <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/image?repos=uditgoenka/autoresearch&type=timeline&legend=bottom-right&v=20260319" />
  598. <img alt="Star History Chart" src="https://api.star-history.com/image?repos=uditgoenka/autoresearch&type=timeline&legend=bottom-right&v=20260319" />
  599. </picture>
  600. </a>
  601. ---
  602. ## License
  603. MIT — see [LICENSE](LICENSE).
  604. ---
  605. ## Credits
  606. - **[Andrej Karpathy](https://github.com/karpathy)** — for [autoresearch](https://github.com/karpathy/autoresearch)
  607. - **Cai, Ziakas, Kontonis, Pearce, Sen, Krishnamurthy, Garg, Papailiopoulos** — for [Fork-and-Flush](https://arxiv.org/abs/2610.07447)
  608. - **Zhang, Ji, Sun, Li, et al.** — for [Stateless Language Agents](https://arxiv.org/abs/2610.07625) (the Advisor behind guided branches)
  609. - **[Anthropic](https://anthropic.com)** — for [Claude Code](https://docs.anthropic.com/en/docs/claude-code) and the skills system
  610. - **[OpenCode](https://opencode.ai)** — for the OpenCode terminal agent
  611. - **[OpenAI](https://openai.com)** — for [Codex](https://developers.openai.com/codex) and the agent skills standard
  612. ---
  613. <div align="center">
  614. ## About the Author
  615. <a href="https://udit.co">
  616. <img src="https://avatars.githubusercontent.com/uditgoenka" width="80" style="border-radius: 50%;" alt="Udit Goenka" />
  617. </a>
  618. **[Udit Goenka](https://udit.co)** — AI Product Expert, Founder & Angel Investor
  619. Self-taught builder who went from a slow internet connection in India to founding multiple companies and helping 700+ startups generate over ~$25m in revenue.
  620. **Building:** [TinyCheque](https://tinycheque.com) (India's first agentic AI venture studio) · [Firstsales.io](https://firstsales.io) (sales automation)
  621. **Investing:** 38 startups backed, 6 exits. Focused on early-stage AI and SaaS.
  622. **Connect:** [udit.co](https://udit.co) · [@iuditg](https://x.com/iuditg) · [@uditgoenka](https://github.com/uditgoenka) · [Newsletter](https://udit.co/blog)
  623. > *"Autonomy scales when you constrain scope, clarify success, mechanize verification, and let agents optimize tactics while humans optimize strategy."*
  624. </div>
.agents/skills/autoresearch/scripts/orchestrate.sh · Bundled script · 614 lines · 3 findings marked
  1. #!/usr/bin/env bash
  2. # orchestrate.sh — deterministic seam for the autoresearch orchestrator loop.
  3. #
  4. # classify <goal-string> → Goal archetype label (keyword heuristics)
  5. # next-hop <state.json> → Next subcommand from router decision table
  6. # units <results.json> → Units-remaining scalar (lower_is_better)
  7. # plateau <state.json> → Exit 0 if last N computed values are flat-or-worse
  8. # screen-cmd <shell-string> → "ok" exit 0 | "refuse" exit 1 safety gate
  9. # verdict <state.json> → CONVERGED|FORK|PLATEAU|CEILING|BLOCKED + ship-gate
  10. #
  11. # All subcommands are pure and CI-usable via exit codes.
  12. set -uo pipefail
  13. # Fork-and-flush rounds a plateaued loop runs before verdict gives up. The
  14. # fork-and-flush paper (arXiv:2610.07447) schedules two rounds per run, at 33%
  15. # and 66% of the budget; a third rarely pays for its ~24% of budget.
  16. FORK_ROUNDS_BEFORE_PLATEAU=2
  17. # ---------------------------------------------------------------------------
  18. # classify: map a goal string to one of the 9 Goal archetype labels.
  19. # Priority order matters: higher-stakes archetypes checked first so that
  20. # "fix and add the broken feature" → fix-broken, not build-feature.
  21. # ---------------------------------------------------------------------------
  22. classify() {
  23. local goal="${1:?usage: classify <goal-string>}"
  24. local g
  25. g=$(printf '%s' "$goal" | tr '[:upper:]' '[:lower:]')
  26. # Keywords match at a word start, so "prefix" is not "fix" and "address" is not "add".
  27. # Compounds that should still match (debug, hotfix, rebuild) are listed by name.
  28. # Security/hardening — above build because "secure" is higher stakes than "add"
  29. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])(secure|insecure|harden|vuln)'; then
  30. echo "harden"; return 0
  31. fi
  32. # Broken/bugfix
  33. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])(fix|bug|broken|hotfix|bugfix)|^[[:space:]]*debug'; then
  34. echo "fix-broken"; return 0
  35. fi
  36. # debug anywhere else, with no build verb present, is still a bug hunt
  37. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])debug' \
  38. && ! printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])((re)?(build|implement)|add(s|ed|ing)?([^[:alnum:]_]|$))'; then
  39. echo "fix-broken"; return 0
  40. fi
  41. # Ship/release/deploy
  42. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])(re)?(ship|release|deploy)'; then
  43. echo "ship-ready"; return 0
  44. fi
  45. # Product direction — requires a "what …" question so bare "next"/"build" in a
  46. # build-feature goal (e.g. "build the next-gen parser") doesn't mis-route here.
  47. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])(what.*build|what.*next)'; then
  48. echo "what-to-build"; return 0
  49. fi
  50. # Build/implement/add — "feature" alone is insufficient; any of these words qualify
  51. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])((re)?(build|implement)|add(s|ed|ing)?([^[:alnum:]_]|$))'; then
  52. echo "build-feature"; return 0
  53. fi
  54. # Metric optimization
  55. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])(faster|smaller|reduce|optimize|coverage)'; then
  56. echo "optimize-metric"; return 0
  57. fi
  58. # Documentation
  59. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])(document|docs)'; then
  60. echo "document"; return 0
  61. fi
  62. # Design decision — "should we" / "decide" / "approach"
  63. if printf '%s' "$g" | grep -qE '(^|[^[:alnum:]_])(should we|decide|approach)'; then
  64. echo "decide-design"; return 0
  65. fi
  66. # Default: open-ended investigation
  67. echo "explore"
  68. }
  69. # ---------------------------------------------------------------------------
  70. # next-hop: cheap router over fields in a state JSON file.
  71. # Decision order: errors → regression → untested gaps → ship/DONE.
  72. # ---------------------------------------------------------------------------
  73. next-hop() {
  74. local state_file="${1:?usage: next-hop <state.json>}"
  75. if [[ ! -f "$state_file" ]]; then
  76. echo "ERROR: missing state file" >&2; return 2
  77. fi
  78. # Parse with sed/grep — no jq dependency (score-regression.sh doesn't use jq)
  79. local errors regression gaps archetype
  80. errors=$(grep -o '"errors_remaining"[[:space:]]*:[[:space:]]*[0-9]*' "$state_file" \
  81. | grep -o '[0-9]*$')
  82. regression=$(grep -o '"regression_verdict"[[:space:]]*:[[:space:]]*"[^"]*"' "$state_file" \
  83. | grep -o '"[^"]*"$' | tr -d '"')
  84. gaps=$(grep -o '"untested_gaps"[[:space:]]*:[[:space:]]*[0-9]*' "$state_file" \
  85. | grep -o '[0-9]*$')
  86. archetype=$(grep -o '"archetype"[[:space:]]*:[[:space:]]*"[^"]*"' "$state_file" \
  87. | grep -o '"[^"]*"$' | tr -d '"')
  88. # Optional: pending_verify gates an independent acceptance check before DONE/ship.
  89. # Absent (or false) → routing is identical to prior behavior.
  90. local pending
  91. pending=$(grep -o '"pending_verify"[[:space:]]*:[[:space:]]*[a-z]*' "$state_file" \
  92. | grep -o '[a-z]*$')
  93. # Guard: missing required fields
  94. if [[ -z "$errors" || -z "$regression" || -z "$gaps" ]]; then
  95. echo "ERROR: malformed state file" >&2; return 2
  96. fi
  97. if [[ "$errors" -gt 0 ]]; then
  98. echo "fix"; return 0
  99. fi
  100. if [[ "$regression" == "UNSTABLE" ]]; then
  101. echo "regression"; return 0
  102. fi
  103. if [[ "$gaps" -gt 0 ]]; then
  104. echo "debug"; return 0
  105. fi
  106. # Gaps clear but an accepted high-impact change still needs a fresh, independent
  107. # acceptance check (separate from the signal used to choose it) → verify first.
  108. if [[ "$pending" == "true" ]]; then
  109. echo "verify"; return 0
  110. fi
  111. # All clear: ship if archetype has ship in the pipeline, else DONE
  112. if [[ "$archetype" == "ship-ready" ]]; then
  113. echo "ship"; return 0
  114. fi
  115. echo "DONE"
  116. }
  117. # ---------------------------------------------------------------------------
  118. # units: compute Units-remaining scalar from a results JSON file.
  119. # Formula: failing_tests + open_hard_regressions + (metric_delta / metric_target)
  120. # Prints "unknown" and exits 2 when inputs are missing or uncomputable.
  121. # ---------------------------------------------------------------------------
  122. units() {
  123. local results_file="${1:?usage: units <results.json>}"
  124. if [[ ! -f "$results_file" ]]; then
  125. echo "unknown"; return 2
  126. fi
  127. local ft regressions delta target
  128. ft=$(grep -o '"failing_tests"[[:space:]]*:[[:space:]]*[0-9.]*' "$results_file" \
  129. | grep -o '[0-9.]*$')
  130. regressions=$(grep -o '"open_hard_regressions"[[:space:]]*:[[:space:]]*[0-9.]*' "$results_file" \
  131. | grep -o '[0-9.]*$')
  132. delta=$(grep -o '"metric_delta"[[:space:]]*:[[:space:]]*[0-9.]*' "$results_file" \
  133. | grep -o '[0-9.]*$')
  134. target=$(grep -o '"metric_target"[[:space:]]*:[[:space:]]*[0-9.]*' "$results_file" \
  135. | grep -o '[0-9.]*$')
  136. if [[ -z "$ft" || -z "$regressions" || -z "$delta" || -z "$target" ]]; then
  137. echo "unknown"; return 2
  138. fi
  139. # Integer check: metric_target must be non-zero to avoid divide-by-zero
  140. if awk -v t="$target" 'BEGIN { exit !(t + 0 == 0) }'; then
  141. echo "unknown"; return 2
  142. fi
  143. # awk handles floating point; strip trailing .0 for clean integer output
  144. awk -v ft="$ft" -v r="$regressions" -v d="$delta" -v t="$target" '
  145. BEGIN {
  146. val = ft + r + (d / t)
  147. # Strip unnecessary trailing zeros (e.g. 4.500 → 4.5, 0.000 → 0)
  148. if (val == int(val)) printf "%d\n", val
  149. else printf "%g\n", val
  150. }
  151. '
  152. }
  153. # ---------------------------------------------------------------------------
  154. # plateau: read state.units_remaining; determine if progress has stalled.
  155. # Skips interleaved "unknown" cycles; N=5 consecutive trailing unknowns = BLOCKED.
  156. # Exit 0 = plateau (no net progress); exit 1 = still improving; exit 3 = BLOCKED.
  157. # ---------------------------------------------------------------------------
  158. plateau() {
  159. local state_file="${1:?usage: plateau <state.json>}"
  160. node - "$state_file" <<'NODE'
  161. const fs = require('fs');
  162. const n = 5;
  163. const blocked = () => { console.log('BLOCKED'); process.exit(3); };
  164. let history;
  165. try {
  166. const state = JSON.parse(fs.readFileSync(process.argv[2], 'utf8'));
  167. history = state.units_remaining;
  168. if (!Array.isArray(history) || history.some(value =>
  169. value !== 'unknown' && (typeof value !== 'number' || !Number.isFinite(value) || value < 0))) {
  170. blocked();
  171. }
  172. } catch {
  173. blocked();
  174. }
  175. let trailingUnknown = 0;
  176. for (let i = history.length - 1; i >= 0 && history[i] === 'unknown'; i--) trailingUnknown++;
  177. const values = history.filter(value => value !== 'unknown');
  178. // An uncomputable runner is BLOCKED; unknown cycles never count as zero progress.
  179. if (trailingUnknown >= n || values.length === 0) blocked();
  180. if (values.length < n) process.exit(1);
  181. // Lower is better. Oscillation with no net improvement is a plateau too.
  182. process.exit(values[values.length - 1] < values[values.length - n] ? 1 : 0);
  183. NODE
  184. }
  185. # ---------------------------------------------------------------------------
  186. # screen-cmd: safety gate for shell strings before execution.
  187. # Prints "ok" / "refuse". Database hosts other than localhost, 127.0.0.1 or a
  188. # single-label name that starts with a letter need a _test or _ci dbname suffix.
  189. # Bare substring "test" inside words like "latest" or "precision" must NOT qualify.
  190. # ---------------------------------------------------------------------------
  191. screen-cmd() {
  192. local cmd="${1:?usage: screen-cmd <shell-string>}"
  193. # A control byte (other than tab and newline) can confuse the readers below, and a
  194. # backslash before a newline joins two lines into one word (r\<newline>m is rm).
  195. # Both are refused as a whole kind.
  196. if printf '%s' "$cmd" | tr -d '\t\n' | LC_ALL=C grep -q '[[:cntrl:]]' \
  197. || [[ "$cmd" == *\\$'\n'* ]]; then
  198. echo "refuse"; return 1
  199. fi
  200. # The shell drops quotes and backslashes before it runs a name, so 'rm' and \rm are rm.
  201. local bare=${cmd//[\"\'\\]/}
  202. # $IFS and brace lists ({rm,-rf,/}) turn into spaces when bash runs them.
  203. bare=${bare//\$\{IFS\}/ }
  204. bare=${bare//\$IFS/ }
  205. bare=${bare//[\{\},]/ }
  206. # This check cannot expand variables, substitutions or brace lists, and each one can
  207. # build a name (${X}m, $X, r``m, `printf rm`, $'\x72m', r{m,}). So it refuses the whole
  208. # kind: any backtick, and any $ outside single quotes unless a closing double quote, a
  209. # space or the line end follows it (grep "ok$" is fine). A brace list outside quotes is
  210. # refused too. The scan below reads quotes the way bash does: a backslash escapes the
  211. # next character, and a single quote inside double quotes is just a character.
  212. # $"..." is a quoted string to bash, so it is marked like a backtick and refused.
  213. # Each quoted part is also kept (joined by \002) so a script passed to a nested shell
  214. # can be checked on its own.
  215. local scan live unquoted parts
  216. scan=$(printf '%s' "$cmd" | awk -v sq="'" -v dq='"' -v bs='\\' 'BEGIN { RS = "\001" } {
  217. q = ""; a = ""; b = ""; part = ""; parts = ""
  218. for (i = 1; i <= length($0); i++) {
  219. c = substr($0, i, 1)
  220. if (q == sq) { if (c == sq) { q = ""; a = a c; b = b c; parts = parts part "\002"; part = "" } else part = part c; continue }
  221. if (c == bs) { i++; a = a "_"; if (q == "") b = b "_"; else part = part c substr($0, i, 1); continue }
  222. if (q == dq) { a = a c; if (c == dq) { q = ""; b = b c; parts = parts part "\002"; part = "" } else part = part c; continue }
  223. if (c == "$" && substr($0, i + 1, 1) == dq) a = a "`"
  224. if (c == sq || c == dq) q = c
  225. a = a c; b = b c
  226. }
  227. printf "%s\001%s\001%s", a, b, parts part
  228. }')
  229. live=${scan%%$'\001'*}
  230. unquoted=${scan#*$'\001'}
  231. parts=${unquoted#*$'\001'}
  232. unquoted=${unquoted%%$'\001'*}
  233. if printf '%s' "$live" | grep -qE '`|\$[^"[:space:]]' \
  234. || printf '%s' "$unquoted" | grep -qE '\{[^[:space:]{}]*(,|\.\.)[^[:space:]{}]*\}'; then
  235. echo "refuse"; return 1
  236. fi
  237. # A glob in a command name (r?, /bin/r[m], sudo r*) can match any program, so a glob
  238. # in the first word of a command, after assignments, reserved words, flags and
  239. # numbers, is refused. After a launcher, every later word is checked, because an
  240. # option value (sudo -u root) can come before the name. Globs in arguments
  241. # (ls *.txt) and the test commands [ and [[ stay fine.
  242. # Every line is read, and a redirect target (>x, 2>/dev/null) is dropped first so it
  243. # cannot stand in for the command name.
  244. if printf '%s' "$unquoted" | awk '{
  245. line = $0; gsub(/[0-9]*[<>]+[&|]?[[:space:]]*[^[:space:];&|(){}<>]*/, " ", line)
  246. n = split(line, segs, /[;&|(){}<>]/)
  247. for (s = 1; s <= n; s++) {
  248. w = split(segs[s], words, /[[:space:]]+/)
  249. launched = 0
  250. for (k = 1; k <= w; k++) {
  251. x = words[k]
  252. if (x == "" || x ~ /^[A-Za-z_][A-Za-z0-9_]*=/ || x ~ /^-/ || x ~ /^[0-9]+$/) continue
  253. if (x ~ /^(if|then|else|elif|do|while|until|!)$/) continue
  254. y = x; sub(/^[^[:space:]]*[\/]/, "", y)
  255. if (y ~ /^(sudo|doas|env|exec|command|builtin|nohup|time|nice|ionice|timeout|xargs|stdbuf|chrt|taskset|eval)$/) { launched = 1; continue }
  256. if (x == "[" || x == "[[") break
  257. if (x ~ /[*?[]/) found = 1
  258. if (!launched) break
  259. }
  260. }
  261. }
  262. END { exit !found }'; then
  263. echo "refuse"; return 1
  264. fi
  265. # A shell or eval runs its quoted script as a new command, so each quoted part is
  266. # screened on its own too: bash -c 'X=rm; $X -rf /' and eval 'r? -rf /' are seen.
  267. # The outer shell drops the backslash in \$ and \`, so the inner script sees a live
  268. # expansion; those are refused outright when a nested shell is present.
  269. if printf '%s' "$bare" | grep -qE '(^|[[:space:];&|(){`<>])([^[:space:];&|(){`<>]*/)?(eval|sh|bash|zsh|dash|ksh|fish|source|su)([[:space:];&|()<>]|$)' \
  270. || printf '%s' "$bare" | grep -qE '(^|[;&|(){`])[[:space:]]*\.[[:space:]]'; then
  271. if [[ "$cmd" == *'\$'* || "$cmd" == *'\`'* ]]; then echo "refuse"; return 1; fi
  272. local part
  273. while IFS= read -r -d $'\002' part; do
  274. [[ -n "${part//[[:space:]]/}" ]] || continue
  275. screen-cmd "$part" >/dev/null || { echo "refuse"; return 1; }
  276. done <<< "$parts"$'\002'
  277. fi
  278. # env -S splits one argument into a new command line that the matchers cannot read.
  279. if printf '%s' "$bare" | grep -qE '(^|[[:space:];&|(){`<>])([^[:space:];&|(){`<>]*/)?env([[:space:]]+[^[:space:];&|()<>]+)*[[:space:]]+(-[a-zA-Z]*S|--split-string)'; then
  280. echo "refuse"; return 1
  281. fi
  282. # Each command matcher below starts at a space, ; & | ( ) { ` $ < or >, so `true;rm -rf /`
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 318 on GitHub
  283. # and `$(rm -rf ~)` are seen too. Flags are read from $bare, so rm "-rf" / is too.
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 319 on GitHub
  284. # rm with recursive AND force, in any flag arrangement: bundled (-rf/-Rf/-fr),
  285. # separate (-r -f), or long (--recursive --force). Both flags must be present.
  286. # The optional path prefix catches path-qualified invocations (/bin/rm, ./rm,
  287. # /usr/local/bin/rm) that a bare command-name anchor would miss.
  288. if printf '%s' "$bare" | grep -qE '(^|[[:space:];&|(){`\\$<>])([^[:space:];&|(){`<>]*/)?rm([[:space:];&|()<>]|$)'; then
  289. local rm_rec=0 rm_force=0
  290. printf '%s' "$bare" | grep -qE -- '(^|[[:space:]])-[a-zA-Z]*[rR]|(^|[[:space:]])--r(e(c(u(r(s(i(ve?)?)?)?)?)?)?)?([^[:alnum:]_-]|$)' && rm_rec=1
  291. printf '%s' "$bare" | grep -qE -- '(^|[[:space:]])-[a-zA-Z]*[fF]|(^|[[:space:]])--f(o(r(ce?)?)?)?([^[:alnum:]_-]|$)' && rm_force=1
  292. if [[ "$rm_rec" -eq 1 && "$rm_force" -eq 1 ]]; then
  293. echo "refuse"; return 1
  294. fi
  295. fi
  296. # curl/wget piped to an interpreter (sh/bash/zsh/dash/fish/ksh/python/perl/ruby/
  297. # node/php), including a path-qualified one (| /bin/bash). Enumerated interpreters
  298. # rather than "refuse any curl pipe" so a legitimate derived predicate that pipes
  299. # curl output to a parser (jq/grep/awk) is not falsely refused.
  300. if printf '%s' "$bare" | grep -qE '(curl|wget)[^|]*\|[[:space:]]*([^[:space:]]*/)?(sh|bash|zsh|dash|fish|ksh|python[0-9.]*|perl|ruby|node|php)([[:space:];&|()<>]|$)'; then
  301. echo "refuse"; return 1
  302. fi
  303. # curl/wget piped through a launcher (sudo, env, exec, ...) into an interpreter.
  304. if printf '%s' "$bare" | grep -qE '(curl|wget)[^|]*\|[[:space:]]*(sudo|doas|env|exec|command|nohup|time)([[:space:]]+[^[:space:]|]+)*[[:space:]]+([^[:space:]]*/)?(sh|bash|zsh|dash|fish|ksh|python[0-9.]*|perl|ruby|node|php)([[:space:];&|()<>]|$)'; then
  305. echo "refuse"; return 1
  306. fi
  307. # A remote payload run without a pipe: bash <(curl ...), source <(curl ...),
    • Note: Runs a downloaded script through process substitution (bash <(curl …))
    • Line 346 on GitHub
  308. # bash -c "$(curl ...)", eval "$(curl ...)".
  309. if printf '%s' "$bare" | grep -qE '((sh|bash|zsh|dash|fish|ksh|python[0-9.]*|perl|ruby|node|php)|source|\.)[[:space:]]+(<[[:space:]]*)?<\([[:space:]]*(curl|wget)'; then
  310. echo "refuse"; return 1
  311. fi
  312. if printf '%s' "$bare" | grep -qE '((sh|bash|zsh|dash|fish|ksh|python[0-9.]*|perl|ruby|node|php)([[:space:]]+([^[:space:];&|]|[<>]&|&>)+)*[[:space:]]+-[a-zA-Z]*c[a-zA-Z]*|eval)[[:space:]]+["'"'"']?(\$\(|`)[[:space:]]*(curl|wget)'; then
  313. echo "refuse"; return 1
  314. fi
  315. # curl/wget routed through xargs into an interpreter. The xargs wrapper sidesteps the
  316. # direct pipe matcher above, so a remote payload still reaches a shell.
  317. if printf '%s' "$bare" | grep -qE '(curl|wget)[^|]*\|.*xargs.*[[:space:]]([^[:space:]]*/)?(sh|bash|zsh|dash|ksh|python[0-9.]*|perl|ruby|node|php)([[:space:];&|()<>]|$)'; then
  318. echo "refuse"; return 1
  319. fi
  320. # Output piped to netcat exfiltrates data off-host.
  321. if printf '%s' "$bare" | grep -qE '\|[[:space:]]*([^[:space:]]*/)?(nc|ncat|netcat)([[:space:];&|()<>]|$)'; then
  322. echo "refuse"; return 1
  323. fi
  324. # Raw block-device write — dd target or shell redirect onto a disk device wipes it.
  325. # Scoped to real device families (incl. SD/eMMC mmcblk, mdadm md, device-mapper dm-)
  326. # so dd/redirect to /dev/null or a regular file stays ok.
  327. if printf '%s' "$bare" | grep -qE '(of=|>[[:space:]]*)/dev/(sd|hd|vd|nvme|disk|mapper|loop|xvd|mmcblk|md|dm-)'; then
  328. echo "refuse"; return 1
  329. fi
  330. # Filesystem format destroys everything on a partition. Optional path prefix catches a
  331. # path-qualified invocation (/sbin/mkfs.ext4) that a bare-name anchor would miss.
  332. if printf '%s' "$bare" | grep -qE '(^|[[:space:];&|(){`\\$<>])([^[:space:];&|(){`<>]*/)?(mkfs|mke2fs)'; then
  333. echo "refuse"; return 1
  334. fi
  335. # find ... -delete mass-removes matched files. Both tokens required so a plain find
  336. # search (no -delete) is not refused; optional path prefix catches /usr/bin/find.
  337. if printf '%s' "$bare" | grep -qE '(^|[[:space:];&|(){`\\$<>])([^[:space:];&|(){`<>]*/)?find([[:space:];&|()<>]|$)' \
  338. && printf '%s' "$bare" | grep -qE '[[:space:]]-delete([[:space:];&|()<>]|$)'; then
  339. echo "refuse"; return 1
  340. fi
  341. # shred overwrites then unlinks — irrecoverable.
  342. if printf '%s' "$bare" | grep -qE '(^|[[:space:];&|(){`\\$<>])([^[:space:];&|(){`<>]*/)?shred([[:space:];&|()<>]|$)'; then
  343. echo "refuse"; return 1
  344. fi
  345. # truncate to zero size destroys file contents in place. Non-zero sizes are allowed.
  346. # Optional path prefix catches /usr/bin/truncate; size matcher covers -s 0, -s0,
  347. # --size 0, and --size=0.
  348. if printf '%s' "$bare" | grep -qE '(^|[[:space:];&|(){`\\$<>])([^[:space:];&|(){`<>]*/)?truncate([[:space:];&|()<>]|$)' \
  349. && printf '%s' "$bare" | grep -qE '(-s[[:space:]]*0|--size[[:space:]]*=?[[:space:]]*0)([[:space:];&|()<>]|$)'; then
  350. echo "refuse"; return 1
  351. fi
  352. # Recursive chmod to a zero mode locks an entire tree out of access. Scoped to the
  353. # zero lock-out (000/00/0 octal short forms) so ordinary recursive permission changes
  354. # are not refused; optional path prefix catches /bin/chmod.
  355. if printf '%s' "$bare" | grep -qE '(^|[[:space:];&|(){`\\$<>])([^[:space:];&|(){`<>]*/)?chmod([[:space:];&|()<>]|$)' \
  356. && printf '%s' "$bare" | grep -qE '(-R|--recursive)([[:space:];&|()<>]|$)' \
  357. && printf '%s' "$bare" | grep -qE '(^|[[:space:]])(000|00|0)([[:space:];&|()<>]|$)'; then
  358. echo "refuse"; return 1
  359. fi
  360. # Fork bomb pattern
  361. if printf '%s' "$cmd" | grep -qF ':(){ :|:'; then
  362. echo "refuse"; return 1
  363. fi
  364. if printf '%s' "$cmd" | grep -qE ':\(\)\{'; then
  365. echo "refuse"; return 1
  366. fi
  367. # AWS credential patterns (key IDs start with AKIA, secret keys are 40-char base64)
  368. if printf '%s' "$cmd" | grep -qE 'AKIA[0-9A-Z]{16}'; then
  369. echo "refuse"; return 1
  370. fi
  371. # PASSWORD= credential pattern
  372. if printf '%s' "$cmd" | grep -qE 'PASSWORD[[:space:]]*='; then
  373. echo "refuse"; return 1
  374. fi
  375. # Private key headers — pattern starts with dashes so pass -- to avoid flag misparse
  376. if printf '%s' "$cmd" | grep -qE -- 'BEGIN (RSA |EC |OPENSSH |DSA )?PRIVATE KEY'; then
  377. echo "refuse"; return 1
  378. fi
  379. # Database URL safety: extract host and dbname from postgres:// or postgresql:// URIs
  380. # Pattern: postgres(ql)://user:pass@HOST/DBNAME or postgres(ql)://HOST/DBNAME
  381. # A quote or backslash inside the scheme ('postgres'ql://) still gives the shell a URL,
  382. # so every URL the shell sees must also appear as plain text here.
  383. local seen_urls plain_urls
  384. seen_urls=$(printf '%s' "$bare" | grep -oE 'postgres(ql)?://' | wc -l)
  385. plain_urls=$(printf '%s' "$cmd" | grep -oE 'postgres(ql)?://' | wc -l)
  386. if [[ "$seen_urls" -ne "$plain_urls" ]]; then
  387. echo "refuse"; return 1
  388. fi
  389. if printf '%s' "$cmd" | grep -qE 'postgres(ql)?://'; then
  390. # Check every URL: one safe URL must not vouch for a second, real one.
  391. # A URL ends at whitespace, quotes or shell punctuation, so `A=url1;B=url2` is two URLs.
  392. # Each URL is read twice: cut at the first punctuation, and with the user:pass part kept
  393. # whole. `p&ss@prod` and `prod;x@localhost` each fail one of the two readings.
  394. # Known limit: a dotted user name with & in the password is refused too; run it by hand.
  395. # Refuse what this check cannot read safely: encoded text (%), several hosts (,),
  396. # a backslash (the shell drops it, so ho\st= is host=), and query keys that move
  397. # the connection (host, hostaddr, dbname, service).
  398. if printf '%s' "$cmd" | grep -qE 'postgres(ql)?://[^[:space:];|<>()]*[%,#\\]' \
  399. || printf '%s' "$cmd" | grep -qiE 'postgres(ql)?://[^[:space:];|<>()]*[?&](host|hostaddr|dbname|service)='; then
  400. echo "refuse"; return 1
  401. fi
  402. # A URL with nothing after the scheme leaves the host to PGHOST or -h, and a quoted
  403. # URL with a space in it (a password such as "p ss") is cut before its real host.
  404. if printf '%s' "$cmd" | grep -qE 'postgres(ql)?://([[:space:];&|<>()"'"'"'`]|$)' \
  405. || printf '%s' "$cmd" | grep -qE '["'"'"']postgres(ql)?://[^"'"'"']*[[:space:]]'; then
  406. echo "refuse"; return 1
  407. fi
  408. # A quote inside the URL (not just around it) hides where the host really ends.
  409. if printf '%s' "$cmd" | grep -qE 'postgres(ql)?://[^[:space:];&|<>()]*["'"'"'`][^[:space:];&|<>()"'"'"'`]'; then
  410. echo "refuse"; return 1
  411. fi
  412. local url rest authority db_path db_host db_name host_ok
  413. while IFS= read -r url; do
  414. # Split scheme://[user@]host[:port][/dbname][?query]; the query never feeds dbname.
  415. rest=${url#*://}
  416. authority=${rest%%/*}
  417. # Two @ signs leave it unclear which part is the host.
  418. if [[ "$authority" == *@*@* ]]; then echo "refuse"; return 1; fi
  419. # Some URL parsers end the host part at ? or #, so prod?@localhost may mean prod.
  420. if [[ "$authority" == *[?#]*@* ]]; then echo "refuse"; return 1; fi
  421. db_host=${authority##*@}
  422. # A ? after the host starts the query, so any later / is not a dbname path.
  423. if [[ "$db_host" == *\?* ]]; then db_path=""; else db_path=${rest:${#authority}}; fi
  424. db_host=${db_host%%[:?]*}
  425. db_path=${db_path%%\?*}
  426. db_name=${db_path#/}
  427. # Shell expansion in the name ($_test is an empty variable) hides the real name.
  428. if [[ "$db_name" == *[\$\{\}\*\[~]* ]]; then echo "refuse"; return 1; fi
  429. # Allowed hosts: localhost, 127.0.0.1, or a plain single-label name that starts
  430. # with a letter, such as a container. Anything else is remote: IPv6, a number
  431. # (2130706433 and 0x7f000001 are IPv4 addresses), and an empty host, which psql
  432. # fills from PGHOST or -h.
  433. host_ok=0
  434. if [[ "$db_host" == "localhost" || "$db_host" == "127.0.0.1" ]]; then
  435. host_ok=1
  436. elif [[ "$db_host" =~ ^[A-Za-z][A-Za-z0-9_-]*$ ]]; then
  437. host_ok=1
  438. fi
  439. # Non-allowlisted host: dbname must end with _test or _ci (anchored suffix, not substring)
  440. if [[ "$host_ok" -eq 0 ]] && ! printf '%s' "$db_name" | grep -qE '_test$|_ci$'; then
  441. echo "refuse"; return 1
  442. fi
  443. done < <(printf '%s' "$cmd" | grep -oE 'postgres(ql)?://[^[:space:];&|<>()"'"'"'`]+'
  444. printf '%s' "$cmd" | grep -oE 'postgres(ql)?://([^[:space:]/@]*@)?[^[:space:];&|<>()"'"'"'`]+')
  445. fi
  446. echo "ok"; return 0
  447. }
  448. # ---------------------------------------------------------------------------
  449. # verdict: synthesize a convergence verdict from state JSON.
  450. # Reads: units, plateau, ceiling, fork_rounds_since_progress fields. Prints
  451. # verdict + ship-gate line. A plateau answers FORK (run a fork-and-flush round)
  452. # until FORK_ROUNDS_BEFORE_PLATEAU rounds have run since the last progress,
  453. # then PLATEAU. A reached ceiling is checked first and always answers CEILING.
  454. # Exit 0 = CONVERGED; exit 1 = not converged; exit 2 = error.
  455. # ---------------------------------------------------------------------------
  456. verdict() {
  457. local state_file="${1:?usage: verdict <state.json>}"
  458. if [[ ! -f "$state_file" ]]; then
  459. echo "BLOCKED"; echo "ship=no"; return 2
  460. fi
  461. local units_val plateau_val ceiling_val fork_rounds
  462. units_val=$(grep -o '"units"[[:space:]]*:[[:space:]]*[0-9.]*' "$state_file" \
  463. | grep -o '[0-9.]*$')
  464. plateau_val=$(grep -o '"plateau"[[:space:]]*:[[:space:]]*[a-z]*' "$state_file" \
  465. | grep -o '[a-z]*$')
  466. ceiling_val=$(grep -o '"ceiling"[[:space:]]*:[[:space:]]*[a-z]*' "$state_file" \
  467. | grep -o '[a-z]*$')
  468. fork_rounds=$(grep -o '"fork_rounds_since_progress"[[:space:]]*:[[:space:]]*"\{0,1\}[0-9][0-9]*' "$state_file" \
  469. | grep -o '[0-9][0-9]*$')
  470. if [[ -z "$units_val" ]]; then
  471. echo "BLOCKED"; echo "ship=no"; return 2
  472. fi
  473. if [[ "$ceiling_val" == "true" ]]; then
  474. echo "CEILING"; echo "ship=no"; return 1
  475. fi
  476. if [[ "$plateau_val" == "true" ]]; then
  477. if (( ${fork_rounds:-0} < FORK_ROUNDS_BEFORE_PLATEAU )); then
  478. echo "FORK"; echo "ship=no"; return 1
  479. fi
  480. echo "PLATEAU"; echo "ship=no"; return 1
  481. fi
  482. # units==0 with no plateau/ceiling → converged
  483. if awk -v u="$units_val" 'BEGIN { exit (u == 0 ? 0 : 1) }'; then
  484. echo "CONVERGED"; echo "ship=yes"; return 0
  485. fi
  486. # units > 0, no plateau/ceiling signal yet → still running
  487. echo "BLOCKED"; echo "ship=no"; return 1
  488. }
  489. # ---------------------------------------------------------------------------
  490. # validate-state: schema gate for orchestrator-state.json. The ledger is the
  491. # loop's evidence trail; a malformed one must not be trusted to route from.
  492. # Prints "valid" exit 0 | "invalid" exit 2. Node is a checked installation
  493. # prerequisite, so use its JSON parser rather than approximating JSON with grep.
  494. # ---------------------------------------------------------------------------
  495. validate-state() {
  496. local state_file="${1:?usage: validate-state <state.json>}"
  497. if [[ ! -f "$state_file" ]]; then
  498. echo "invalid"; return 2
  499. fi
  500. if ! node -e '
  501. const fs = require("fs");
  502. const state = JSON.parse(fs.readFileSync(process.argv[1], "utf8"));
  503. const strings = ["goal", "archetype", "predicate", "terminal_choice"];
  504. if (!strings.every((key) => typeof state[key] === "string" && state[key].length > 0)) process.exit(1);
  505. if (!Number.isInteger(state.cycle) || state.cycle < 0) process.exit(1);
  506. if (!Array.isArray(state.units_remaining) || !Array.isArray(state.pipeline_log)) process.exit(1);
  507. ' "$state_file" 2>/dev/null; then
  508. echo "invalid"; return 2
  509. fi
  510. echo "valid"; return 0
  511. }
  512. # ---------------------------------------------------------------------------
  513. # screen-state-predicate: extract the pinned predicate from a persisted state
  514. # file and re-run it through screen-cmd. Persisted commands are never trusted —
  515. # a poisoned state file must not re-enter the loop with an unscreened command.
  516. # Delegates the verdict (ok/refuse + exit) to screen-cmd; "invalid" exit 2 when
  517. # the state has no pinned predicate.
  518. # ---------------------------------------------------------------------------
  519. screen-state-predicate() {
  520. local state_file="${1:?usage: screen-state-predicate <state.json>}"
  521. if [[ ! -f "$state_file" ]]; then
  522. echo "invalid"; return 2
  523. fi
  524. local pred
  525. if ! pred=$(node -e '
  526. const fs = require("fs");
  527. const state = JSON.parse(fs.readFileSync(process.argv[1], "utf8"));
  528. if (typeof state.predicate !== "string" || state.predicate.length === 0 || state.predicate.includes("\0")) process.exit(1);
  529. process.stdout.write(state.predicate);
  530. ' "$state_file" 2>/dev/null); then
  531. echo "invalid"; return 2
  532. fi
  533. screen-cmd "$pred"
  534. }
  535. case "${1:-}" in
  536. classify) shift; classify "$@" ;;
  537. next-hop) shift; next-hop "$@" ;;
  538. units) shift; units "$@" ;;
  539. plateau) shift; plateau "$@" ;;
  540. screen-cmd) shift; screen-cmd "$@" ;;
  541. verdict) shift; verdict "$@" ;;
  542. validate-state) shift; validate-state "$@" ;;
  543. screen-state-predicate) shift; screen-state-predicate "$@" ;;
  544. *) echo "usage: $0 {classify|next-hop|units|plateau|screen-cmd|verdict|validate-state|screen-state-predicate}" >&2; exit 64 ;;
  545. esac

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.