Skip to content
Back to search

rerun

Rerun: when an HCP Terraform run fails, agents diagnose it through the Terraform MCP server, verify a fix, and open the PR.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/tayden-b--rerun/badge.svg)](/listings/tayden-b--rerun)

HTML

<a href="/listings/tayden-b--rerun"><img src="/api/listings/tayden-b--rerun/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Demo mode needs no credentials. It runs the real incident workflow on the recorded `apply-time-exit-1` run, started through the signed webhook: a mock MCP server serves the recorded fixtures, Jev answers are replayed, the agents are scripted, and GitHub and the verifier are fakes. The incident reaches Awaiting approval in about 20 seconds.

git clone https://github.com/tayden-b/rerun && cd rerun
npm ci
npm run demo -- --fresh   # app on http://localhost:3000, mock Terraform MCP server on :8081; --fresh starts a new incident

Live mode needs an HCP Terraform workspace that applies from a GitHub repository (fork [rerun-fixtures](https://github.com/tayden-b/rerun-fixtures)), Docker, and the keys listed in `.env.example`.

cp .env.example .env.local          # TFE_TOKEN, TFE_ORG, GITHUB_TOKEN, JEV_API_KEY, ANTHROPIC_API_KEY
docker compose up -d terraform-mcp  # Terraform MCP server on 127.0.0.1:8080 with the allowlist
npm run db:migrate
npm run dev                         # polls the workspace every 30 s
# in the fixtures repository: scripts/break.sh http-404-postcondition

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 275 lines
  1. # Rerun
  2. When an HCP Terraform run fails, Rerun reads the failure through the Terraform MCP server, classifies it with a typed decision model, writes the smallest fix, proves it with a speculative plan and opens a pull request that a human approves from a dashboard.
  3. [![CI](https://github.com/tayden-b/rerun/actions/workflows/ci.yml/badge.svg?branch=dev)](https://github.com/tayden-b/rerun/actions/workflows/ci.yml)
  4. [![Triage eval](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Ftayden-b%2Frerun%2Fdev%2Fdocs%2Fbadges%2Feval.json)](#evals)
  5. [![Terraform MCP tools](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Ftayden-b%2Frerun%2Fdev%2Fdocs%2Fbadges%2Fmcp-tools.json)](#why-the-terraform-mcp-server)
  6. ![Demo mode: a failed apply goes from collection through triage, diagnosis, fix and verification to Awaiting approval](docs/media/rerun.gif)
  7. ## Why the Terraform MCP server
  8. Every piece of Terraform data Rerun uses comes from the [Terraform MCP server](https://github.com/hashicorp/terraform-mcp-server), the same server a platform team already runs for its engineers' coding agents. Rerun adds no HCP Terraform API client of its own. That makes the integration a configuration decision rather than code:
  9. - **Read-only by construction.** The server is started with `--tools=<allowlist>`, so only the tools below are registered. `ENABLE_TF_OPERATIONS` is never set; Rerun cannot queue, apply or discard a run.
  10. - **One workspace.** The client in `src/mcp/terraform.ts` refuses any call that names a workspace, run, plan or state version outside `RERUN_WORKSPACE` (`WorkspaceScopeError`).
  11. - **One token holder.** `TFE_TOKEN` is read by the MCP sidecar and by the Docker verifier for the speculative plan, nowhere else.
  12. - **Human merge.** Rerun opens pull requests. Approving in the dashboard merges one; the repository's own pipeline applies it.
  13. ```mermaid
  14. flowchart LR
  15. poll["Poller<br/>list_runs every 30 s"] --> wf
  16. hook["Webhook<br/>HMAC-verified"] --> wf
  17. wf["Incident workflow<br/>Workflow SDK, one run per incident"]
  18. wf -->|read-only tools| mcp["Terraform MCP server<br/>allowlist, one workspace"]
  19. mcp --> hcp[("HCP Terraform")]
  20. wf -->|four questions| jev["Jev<br/>typed decisions"]
  21. wf --> agents["Config and Reality agents<br/>Fix agent"]
  22. agents -->|provider docs| mcp
  23. wf --> verify["Docker verifier<br/>fmt, validate, speculative plan"]
  24. verify --> hcp
  25. wf --> pr["GitHub pull request<br/>rerun-fixtures"]
  26. wf --> db[("libSQL<br/>incidents, events")]
  27. db --> ui["Dashboard<br/>approve = merge"]
  28. ui --> pr
  29. pr -->|Actions apply| hcp
  30. ```
  31. Every Terraform MCP tool on the allowlist, generated from `src/mcp/allowlist.ts` by `npm run eval:readme`:
  32. <!-- mcp-tools:start -->
  33. | Tool | Toolset | What Rerun uses it for | Called from |
  34. | ----------------------------- | --------- | ------------------------------------------------------------------------------------------------ | --------------------- |
  35. | `list_workspaces` | terraform | Not called; the scope guard only allows a filter on the watched workspace. | — |
  36. | `get_workspace_details` | terraform | Resolves the workspace id that state reads need; part of every incident context. | `collectRunArtifacts` |
  37. | `list_runs` | terraform | Polls for newly errored runs and for the run after a merge. | `src/watch/runs.ts` |
  38. | `get_run_details` | terraform | Run status, message and the plan and apply ids; proves the run belongs to the watched workspace. | `collectRunArtifacts` |
  39. | `get_plan_details` | terraform | Plan status: did the run fail in init/plan or get past it. | `collectRunArtifacts` |
  40. | `get_plan_logs` | terraform | The failing log, from which the 120-line error excerpt is cut. | `collectRunArtifacts` |
  41. | `get_plan_json_output` | terraform | Planned resource changes for the blast-radius question. Scrubbed of private keys first. | `collectRunArtifacts` |
  42. | `get_apply_details` | terraform | Apply status, when the plan succeeded. | `collectRunArtifacts` |
  43. | `get_apply_logs` | terraform | The apply-time error (provisioners, API calls during apply). | `collectRunArtifacts` |
  44. | `list_state_versions` | terraform | Not called by the incident flow; the scope guard uses it to check an explicit state version id. | — |
  45. | `get_state_version` | terraform | Current state outputs and resources, for the Reality agent. | `collectRunArtifacts` |
  46. | `get_run_comments` | terraform | Not called yet; exposed for a comment-back surface. | — |
  47. | `whoami` | terraform | Identity check when recording fixtures. | `scripts/record.ts` |
  48. | `search_provider_docs` | registry | Finds the provider doc for an argument or attribute named in the error. | Config and Fix agents |
  49. | `get_provider_details` | registry | Reads that doc page, e.g. which tls v4 attributes are read-only. | Config and Fix agents |
  50. | `get_latest_provider_version` | registry | Not called yet; exposed for version-conflict fixes. | — |
  51. | `get_provider_capabilities` | registry | Not called yet; exposed for version-conflict fixes. | — |
  52. <!-- mcp-tools:end -->
  53. ## How a failed run becomes a merged fix
  54. One workflow run per incident (`workflows/`); every step writes events that the dashboard renders as a live trace.
  55. 1. **Collect.** The poller (or webhook) sees an `errored` run. Rerun pulls the run, plan, logs, plan JSON, apply and current state through MCP, and cuts a 120-line error excerpt with secrets redacted.
  56. 2. **Triage.** One Jev request answers four questions: failure class (12 closed classes), whether a configuration change alone can fix it, blast radius, and whether a human must decide. The answers are probabilities, not prose.
  57. <img src="docs/media/triage.png" alt="Triage card: failure class and the three routing signals as probability bars, above the error excerpt" width="360">
  58. 3. **Diagnose.** The Config agent reads the repository and the log; the Reality agent also reads state and apply output when the failure happened at apply time or outside the configuration. Both cite evidence (file and line, log line, provider doc).
  59. <img src="docs/media/subagents.png" alt="Two agent panels with their tool calls and cited evidence" width="720">
  60. 4. **Fix.** The Fix agent edits a working copy of the configuration and gives a rationale per file.
  61. 5. **Verify.** `terraform fmt -check`, `terraform validate` and a speculative remote plan run in `hashicorp/terraform:1.14`. On failure the fix is retried once with the verifier output; then the incident goes to a human.
  62. <img src="docs/media/fix-verify.png" alt="The proposed diff next to the verifier output for fmt, validate and plan" width="720">
  63. 6. **Propose.** Rerun opens the pull request with the evidence in its body. The router (`autoPr` iff `config_fixable` ≥ 0.70, `blast_radius` ≤ 1.0 and `needs_human` ≤ 0.40) decides between a ready-for-review PR and a draft flagged for a human.
  64. 7. **Approve.** A human reads the war room and presses Approve, which merges the PR, or Request changes, which posts a PR comment.
  65. ![The approval bar: Approve and merge or Request changes on the fix branch](docs/media/approve.png)
  66. 8. **Confirm.** The fixtures repository's GitHub Actions applies the merge. Rerun watches the next run: `applied` resolves the incident, `errored` opens a new incident linked to this one.
  67. ## Evals
  68. `npm run eval` scores triage on the eleven recorded scenarios of [rerun-fixtures](https://github.com/tayden-b/rerun-fixtures) (real MCP outputs and real Jev answers, replayed offline). `npm run eval -- --live` breaks each scenario on the real workspace and runs the whole flow. `npm run eval:readme` renders `evals/results.json` below; nothing between the markers is edited by hand.
  69. Accuracy is against `expected.json` in each scenario. Calibration compares stated probabilities with outcomes in 0.1 bins. The flip rate counts verdicts that change when the input is perturbed without changing its meaning (renamed resources, an extra UUID line, reordered plan entries). Cost covers the triage call only; agent and verifier costs are in the live table and on each incident.
  70. <!-- evals:start -->
  71. Generated 2026-11-21 from `evals/results.json`: 11 scenario(s), 2 perturbation(s) per scenario, prices as of 2026-11-20.
  72. ### Triage accuracy
  73. | Provider | Scenarios | Errors | Class accuracy | Route accuracy | Mean latency | Mean input tokens | Triage cost / incident |
  74. | -------- | --------- | ------ | -------------- | -------------- | ------------ | ----------------- | ---------------------- |
  75. | jev | 11 | 0 | 100% | 100% | 53 ms | 1410 | $0.000059 |
  76. | llm | 11 | 0 | 100% | 91% | 2931 ms | 3733 | $0.000373 |
  77. ### Accuracy per failure class
  78. Correct `failure_class` answers out of the scenarios of that class. Each cell is one provider on the same recorded MCP data.
  79. | Class | Scenarios | jev | llm |
  80. | --------------------- | --------- | --- | --- |
  81. | `apply_runtime` | 1 | 1/1 | 1/1 |
  82. | `provider_api_error` | 1 | 1/1 | 1/1 |
  83. | `precondition_failed` | 2 | 2/2 | 2/2 |
  84. | `dependency_cycle` | 1 | 1/1 | 1/1 |
  85. | `config_syntax` | 2 | 2/2 | 2/2 |
  86. | `module_source` | 1 | 1/1 | 1/1 |
  87. | `provider_version` | 1 | 1/1 | 1/1 |
  88. | `config_reference` | 1 | 1/1 | 1/1 |
  89. | `missing_input` | 1 | 1/1 | 1/1 |
  90. ### Question rewording: before and after
  91. Same scenarios and SPEC §4 thresholds (`autoPr` iff `config_fixable` ≥ 0.70, `blast_radius` ≤ 1.0, `needs_human` ≤ 0.40); only the question wording changed. Before: recorded Jev answers of 2026-11-21. After: live jev answers of 2026-11-21.
  92. | Scenario | Expected | `config_fixable` before | after | `needs_human` before | after | Route before | after |
  93. | --------------------------- | --------- | ----------------------- | ----- | -------------------- | ----- | ------------ | --------- |
  94. | `apply-time-exit-1` | `autoPr` | 0.88 | 0.92 | 0.69 | 0.27 | `draftPr` | `autoPr` |
  95. | `bad-http-url` | `autoPr` | 0.77 | 0.90 | 0.56 | 0.29 | `draftPr` | `autoPr` |
  96. | `check-block-failure` | `autoPr` | 0.83 | 0.95 | 0.77 | 0.30 | `draftPr` | `autoPr` |
  97. | `dependency-cycle` | `autoPr` | 0.88 | 0.92 | 0.69 | 0.16 | `draftPr` | `autoPr` |
  98. | `http-404-postcondition` | `autoPr` | 0.73 | 0.91 | 0.69 | 0.27 | `draftPr` | `autoPr` |
  99. | `missing-required-arg` | `autoPr` | 0.97 | 0.98 | 0.49 | 0.20 | `draftPr` | `autoPr` |
  100. | `module-source-typo` | `draftPr` | 0.78 | 0.81 | 0.55 | 0.41 | `draftPr` | `draftPr` |
  101. | `provider-version-conflict` | `draftPr` | 0.68 | 0.84 | 0.63 | 0.81 | `draftPr` | `draftPr` |
  102. | `removed-argument` | `autoPr` | 0.94 | 0.96 | 0.25 | 0.12 | `autoPr` | `autoPr` |
  103. | `undeclared-reference` | `autoPr` | 0.96 | 0.97 | 0.43 | 0.16 | `draftPr` | `autoPr` |
  104. | `unset-variable` | `autoPr` | 0.59 | 0.95 | 0.56 | 0.31 | `draftPr` | `autoPr` |
  105. | Route matches expected | | | | | | 3/11 | 11/11 |
  106. ### Calibration
  107. Outcome: `autoPr` in `expected.json` means config-fixable = 1 and needs-human = 0. ECE = expected calibration error over 0.1-wide bins (lower is better).
  108. | Provider | Signal | n | ECE |
  109. | -------- | ---------------- | --- | ---- |
  110. | jev | `config_fixable` | 11 | 0.20 |
  111. | jev | `needs_human` | 11 | 0.26 |
  112. | llm | `config_fixable` | 11 | 0.13 |
  113. | llm | `needs_human` | 11 | 0.14 |
  114. | Provider | Signal | Bin | n | Mean predicted | Observed |
  115. | -------- | ---------------- | ------- | --- | -------------- | -------- |
  116. | jev | `config_fixable` | 0.8–0.9 | 2 | 0.82 | 0.00 |
  117. | jev | `config_fixable` | 0.9–1.0 | 9 | 0.94 | 1.00 |
  118. | jev | `needs_human` | 0.1–0.2 | 3 | 0.15 | 0.00 |
  119. | jev | `needs_human` | 0.2–0.3 | 4 | 0.26 | 0.00 |
  120. | jev | `needs_human` | 0.3–0.4 | 2 | 0.30 | 0.00 |
  121. | jev | `needs_human` | 0.4–0.5 | 1 | 0.41 | 1.00 |
  122. | jev | `needs_human` | 0.8–0.9 | 1 | 0.81 | 1.00 |
  123. | llm | `config_fixable` | 0.7–0.8 | 2 | 0.75 | 0.50 |
  124. | llm | `config_fixable` | 0.8–0.9 | 3 | 0.84 | 0.67 |
  125. | llm | `config_fixable` | 0.9–1.0 | 6 | 0.93 | 1.00 |
  126. | llm | `needs_human` | 0.0–0.1 | 5 | 0.05 | 0.00 |
  127. | llm | `needs_human` | 0.2–0.3 | 3 | 0.22 | 0.00 |
  128. | llm | `needs_human` | 0.3–0.4 | 1 | 0.30 | 0.00 |
  129. | llm | `needs_human` | 0.8–0.9 | 2 | 0.82 | 1.00 |
  130. ### Robustness
  131. Each scenario state is perturbed without changing its meaning (consistent resource renames, an extra UUID line, reordered plan entries, whitespace). A flip is a run whose class or route differs from the unperturbed run.
  132. | Provider | Perturbed runs | Flips | Flip rate |
  133. | -------- | -------------- | ----- | --------- |
  134. | jev | 22 | 0 | 0% |
  135. | llm | 22 | 0 | 0% |
  136. ### End-to-end (live)
  137. Each scenario broken on the real workspace, then run through Rerun. 9 of 11 reached a passing verifier and an open PR.
  138. | Scenario | Class (expected → actual) | Route (expected → actual) | Reached status | Verifier passed | PR | fixTouches respected | Duration | Cost |
  139. | ------------------------- | ------------------------- | ------------------------- | --------------------------------- | --------------- | ---------------------------------------------------------------- | -------------------- | -------- | ------ |
  140. | apply-time-exit-1 | apply_runtime → — | autoPr → — | blocked: Anthropic credit balance | — | — | — | — | — |
  141. | bad-http-url | provider_api_error → — | autoPr → — | blocked: Anthropic credit balance | — | — | — | — | — |
  142. | check-block-failure | precondition_failed | autoPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/15) | yes | 54 s | $0.073 |
  143. | dependency-cycle | dependency_cycle | autoPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/16) | yes | 52 s | $0.062 |
  144. | http-404-postcondition | precondition_failed | autoPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/14) | yes | 52 s | $0.060 |
  145. | missing-required-arg | config_syntax | autoPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/9) | yes | 58 s | $0.082 |
  146. | module-source-typo | module_source | draftPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/17) (draft) | yes | 50 s | $0.057 |
  147. | provider-version-conflict | provider_version | draftPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/12) (draft) | yes | 60 s | $0.119 |
  148. | removed-argument | config_syntax | autoPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/13) | yes | 48 s | $0.060 |
  149. | undeclared-reference | config_reference | autoPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/10) | yes | 64 s | $0.057 |
  150. | unset-variable | missing_input | autoPr | awaiting_approval | yes | [PR](https://github.com/tayden-b/rerun-fixtures/pull/11) | yes | 54 s | $0.057 |
  151. Approve → resolved (apply-time-exit-1): blocked: Anthropic credit balance.
  152. - LLM decision cost counts input tokens only (DecisionProvider does not report output tokens).
  153. <!-- evals:end -->
  154. ### What the question rewording changed
  155. The first live recording got the class and stage right in all 11 scenarios but put `needs_human` above 0.40 in 10 of them. The old question asked whether a fix needed "a human decision (ambiguous intent, security-sensitive change, or more than one reasonable fix)". Almost every Terraform change has more than one reasonable fix, so the model answered yes for a missing argument and a misspelled reference alike.
  156. The rewrite in `src/decisions/questions.ts` leaves the class question alone. Most of it is general: each routing question now says what Rerun can change (any `.tf` or `.tfvars` file, through a pull request a human approves) and what it cannot (credentials, quota, systems outside Terraform), and `needs_human` says that review always happens, so several equivalent ways to write a fix no longer counts as needing a human. The blast-radius question gained one clarification: replacing a resource that holds no data is the second level, not higher.
  157. One sentence is domain policy chosen with the fixtures in view: a fix that edits a module source or version, or a `required_providers` or `required_version` constraint, counts as an owner decision because it changes which dependency code gets installed. We think that is the right default for a platform team, but it is also what keeps the two init-stage scenarios on `draftPr`, so part of the 11/11 is that policy rather than better judgment. `apply-time-exit-1` has no margin: its `blast_radius` sits at the 1.0 limit (Jev 1.00, LLM 1.02). Eleven scenarios is a small set; the wording needs its own scenarios on any real estate before `autoPr` is trusted.
  158. ## Things we learned
  159. Dated 2026-11-21; versions as pinned in this repository.
  160. - **`TRANSPORT_MODE` disables the tool allowlist.** In `hashicorp/terraform-mcp-server:1.4`, setting `TRANSPORT_MODE` makes the server ignore `--tools` and register every toolset. Rerun starts it with the `streamable-http` subcommand and `--transport-host=0.0.0.0 --transport-port=8080 --tools=<allowlist>` instead (`docker-compose.yml`). Check `tools/list` after any change to how the server is started.
  161. - **Init failures never reach HCP Terraform from a CLI-driven workflow.** When `terraform init` fails on the CI runner, no remote run is queued, so there is nothing for Rerun to see. The fixtures repository's Actions workflow uploads the configuration through the HCP Terraform API in that case, which produces a remote run that fails at init. HCP Terraform reports it inside the plan phase: `Operation failed: failed running terraform init`.
  162. - **`check` blocks only warn.** A failing `check` assertion does not fail a run. The `check-block-failure` scenario enforces the same guardrail as a resource `precondition`, so its class is `precondition_failed`.
  163. - **Plan JSON carries private keys in clear text.** The baseline's `tls_private_key.platform` writes its PEM private key into the raw plan JSON returned by `get_plan_json_output`. Unfiltered, that key would have gone to both models and into the recorded fixtures; a code review of the recording PR caught it. `src/mcp/scrub.ts` now replaces any `-----BEGIN … PRIVATE KEY-----` block with `[REDACTED:private_key]` before trace emission, recording or model input, and `test/fixtures/redaction.test.ts` fails on any PEM key under `fixtures/`. The key had been readable in one unmerged commit, so it was rotated with an HCP Terraform replace run of `tls_private_key.platform` and `tls_self_signed_cert.platform`. Plan JSON and state are secret-bearing: scrub them before any model sees them, whatever the provider marks as sensitive.
  164. - **Two log formats.** CLI-driven runs log JSON lines (`{"@level":"error","@message":"Error: …"}`); API-queued runs log plain text with `│ Error:` blocks. The excerpt builder normalises both.
  165. ## Quickstart
  166. Demo mode needs no credentials. It runs the real incident workflow on the recorded `apply-time-exit-1` run, started through the signed webhook: a mock MCP server serves the recorded fixtures, Jev answers are replayed, the agents are scripted, and GitHub and the verifier are fakes. The incident reaches Awaiting approval in about 20 seconds.
  167. ```bash
  168. git clone https://github.com/tayden-b/rerun && cd rerun
  169. npm ci
  170. npm run demo -- --fresh # app on http://localhost:3000, mock Terraform MCP server on :8081; --fresh starts a new incident
  171. ```
  172. Live mode needs an HCP Terraform workspace that applies from a GitHub repository (fork [rerun-fixtures](https://github.com/tayden-b/rerun-fixtures)), Docker, and the keys listed in `.env.example`.
  173. ```bash
  174. cp .env.example .env.local # TFE_TOKEN, TFE_ORG, GITHUB_TOKEN, JEV_API_KEY, ANTHROPIC_API_KEY
  175. docker compose up -d terraform-mcp # Terraform MCP server on 127.0.0.1:8080 with the allowlist
  176. npm run db:migrate
  177. npm run dev # polls the workspace every 30 s
  178. # in the fixtures repository: scripts/break.sh http-404-postcondition
  179. ```
  180. ## What a customer would extend
  181. - **Drift with Search & Import.** `drift` is already a class and always routes to a draft PR. HCP Terraform Search & Import (Terraform 1.14 `list` blocks and `terraform query`) can generate the `import` blocks that adopt a drifted resource, so the Fix agent proposes an import instead of a rewrite.
  182. - **Sentinel and OPA policy failures.** A policy check failure is a new class. HCP Terraform exports Sentinel mock data for each plan; the verifier can run `sentinel test` against the mocks before the PR is opened.
  183. - **Vercel Sandbox verifier.** `src/verify/vercel-sandbox.ts` implements the `Verifier` interface as a stub. A sandbox removes the local Docker dependency and isolates each plan.
  184. - **Slack or MCP Apps surface.** The war room's Approve and Request changes map onto an interactive Slack message or an MCP App, so the approval happens where the platform team already works.
  185. - **More workspaces.** The scope guard takes one workspace name; a list and a per-workspace repository map make it an organization-wide watcher.
  186. ## Limitations
  187. - One workspace and one repository per deployment.
  188. - The dashboard has no authentication; anyone who can reach it can approve a merge.
  189. - A speculative plan cannot prove an apply-time fix: for `apply_runtime` failures the verifier shows the plan succeeds, and only the post-merge apply confirms the fix.
  190. - The routing thresholds and question wording were calibrated on eleven scenarios built on no-cloud providers. Real estates need their own scenarios before `autoPr` is trusted.
  191. - The live eval is run by hand against a real workspace; CI runs demo mode only.
  192. - Polling adds up to 30 s of latency; the webhook route removes it.
  193. ## Repository
  194. | Path | Contents |
  195. | ------------------------------------------ | -------------------------------------------------------------- |
  196. | `src/mcp/` | Scoped Terraform MCP client, allowlist, scrubbing, mock server |
  197. | `src/decisions/` | Jev and LLM decision providers, triage questions, router |
  198. | `src/agents/` | Config, Reality and Fix agents |
  199. | `src/verify/`, `src/github/`, `src/watch/` | Docker verifier, GitHub client, run watcher |
  200. | `workflows/` | The incident workflow |
  201. | `app/`, `components/` | Dashboard |
  202. | `evals/` | Eval harness and the README renderer |
  203. | `fixtures/scenarios/` | Recorded MCP outputs and Jev answers per scenario |
  204. The specification is [`handoff/SPEC.md`](handoff/SPEC.md); contributor rules and commands are in [`AGENTS.md`](AGENTS.md).
  205. ## License
  206. [MIT](LICENSE)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.