Skip to content
Back to search

opslane-verify-marketplace

MarketplaceWorks with Claude CodeGitHub

Verify changes against approved criteria by driving the real system.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/opslane-verify/badge.svg)](/listings/opslane-verify)

HTML

<a href="/listings/opslane-verify"><img src="/api/listings/opslane-verify/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

/plugin marketplace add opslane/verify
/plugin install opslane-verify

Needs Node.js on your PATH. This registers `/verify` and `/verify-setup`, plus a Playwright MCP server for browser criteria. In your repository it writes only to `.verify/`, and adds that directory to `.gitignore` if it is not there already.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 18 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "opslane-verify-marketplace",
  3. "description": "Verify changes against approved criteria by driving the real system.",
  4. "owner": {
  5. "name": "opslane"
  6. },
  7. "plugins": [
  8. {
  9. "name": "opslane-verify",
  10. "description": "Verify a change works by driving the real system the way a user does. Approved acceptance criteria, a recorded run, and a four-axis report.",
  11. "version": "2.8.0",
  12. "source": "./",
  13. "author": {
  14. "name": "opslane"
  15. }
  16. }
  17. ]
  18. }
README.md · README · 111 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Verify Skill
  2. Don't trust your agent. Verify it.
  3. Coding agents write more code than anyone can read, and reviewing the diff does not run it. So the agent says done and the tests are green, and both of those came from the agent. Claude checking Claude's own work has one likely answer, and it is that everything works.
  4. That leaves one real check, and it is you. Boot the app, click through it, and find out what the change actually did. Every time.
  5. With Verify, you approve the acceptance criteria first, from what you meant the change to do. Verify boots your stack, drives the change end to end, and hands you a report that shows whether it works, with the evidence behind every line.
  6. It is a Claude Code plugin. You run it locally, after the agent finishes implementing and before you open the PR.
  7. ## How it works
  8. - **It reads your plan, not the diff.** The criteria come from what the change was meant to do, so the implementation gets no say in what "correct" means.
  9. - **A second model reviews the criteria.** Codex when installed, otherwise a fresh Claude session that has seen only the plan. It catches the checks that broken code would still pass.
  10. - **You approve them.** Nothing runs until you say go. Fix any criterion that does not match what you meant.
  11. - **It boots your whole stack and drives every criterion.** Real HTTP, real rows, real browser. Nothing mocked in process.
  12. - **It hands you the receipts.** Every command it ran, what came back, and the evidence behind each verdict.
  13. ```mermaid
  14. flowchart TD
  15. plan["Your plan<br>what you meant"]
  16. old["The old code<br>how it behaved before"]
  17. diff["The diff<br>what the agent built"]
  18. ac["Acceptance criteria"]
  19. codex["Second model<br>catches checks broken code would pass"]
  20. you["You approve"]
  21. run["Boot the stack<br>run every criterion against it"]
  22. rep["Report<br>what ran, what happened, the proof"]
  23. plan --> ac
  24. old --> ac
  25. diff -.->|"gaps only, as questions for you"| ac
  26. ac --> codex --> you --> run --> rep
  27. ```
  28. ## Install
  29. ```
  30. /plugin marketplace add opslane/verify
  31. /plugin install opslane-verify
  32. ```
  33. Needs Node.js on your PATH. This registers `/verify` and `/verify-setup`, plus a Playwright MCP server for browser criteria. In your repository it writes only to `.verify/`, and adds that directory to `.gitignore` if it is not there already.
  34. ## Quick start
  35. From your repository, on a branch with a change and a plan:
  36. ```
  37. /verify
  38. ```
  39. The first time in a repository, it asks you to run `/verify-setup`. Setup looks for a compose file, seed scripts, and env files, asks one question per thing it could not decide, and writes `.verify/setup.json`. It never asks for or stores a secret. Commit that file and your teammates skip the interview.
  40. After that, `/verify` writes the criteria, has them reviewed, and stops. Read them. If one is wrong, say so. When they look right, say:
  41. ```
  42. go
  43. ```
  44. It boots the stack, seeds it, drives each criterion, tears the stack down, and prints the report. Everything it produced is in `.verify/runs/<timestamp>/`.
  45. ## We test the intent, not the code
  46. Every expectation comes from the plan, or from how the code behaved before the change. Never from the diff.
  47. A check written from the implementation confirms the implementation against itself, so it cannot fail. This is also why the agent's own tests pass. The test and the code came from the same understanding, so they agree even when both are wrong.
  48. Verify reads the diff once, to find what the plan does not explain. Each of those becomes a question for you rather than an answer it invents. If it cannot find a plan at all, it asks for one and stops.
  49. Two guards sit on top of that. Each criterion records whether the old code would have passed it, so one meant to prove new behaviour that the old code already passes gets flagged before you approve. And the second model reads the whole set looking for checks that broken code would still pass. On a real run it caught one that any code labelling every job as access-denied would have satisfied, and another whose check compared the worker's output against the worker's own count.
  50. ## We run your whole stack for every change
  51. Verify does not test your code in isolation. Every run boots the whole thing.
  52. With a compose file, each run gets its own compose project, so it never collides with your dev stack or with another run. Boot is `up -d --wait`. Teardown is `down -v`, so the volumes go too and nothing leaks into the next run. Without compose, a start command and a health URL work the same way.
  53. Then it seeds. Your repository's own seed scripts run first. After them, a per-run script creates whatever the criteria need through the app's front door, with a marker unique to the run woven into every record. You read that script before anything runs.
  54. Then it probes every part the criteria depend on: a marker round trip for the database, a request for the API, your one-line liveness command for a worker, a queue, or storage. A part that is down turns its criteria into "could not run" rather than a failure blamed on your change.
  55. Nothing is mocked in process. The bugs that survive unit tests live in the parts that usually get mocked: the webhook that fires twice, the migration that runs against real rows, the page that renders before its data resolves.
  56. ## We give you the receipts
  57. A verdict you cannot check is just another agent saying "done". So every criterion carries the commands that ran and what came back. It also carries an artifact proving the check happened in this run: a row bearing the run's marker, a screenshot taken at the moment of observation, or a value read live. A status code on its own is not evidence, and a criterion with no way to prove it ran is not run.
  58. ![A Verify report. The headline reads 4 of 5 proven, 1 failed, above a criterion card with its plain-language claim, its pass badges, and the two commands that ran with their exit codes](assets/report-receipts.png)
  59. The report also separates four questions that one word would blur.
  60. - **Proven** is how many verdicts the evidence supports. A pass with no receipt behind it does not count.
  61. - **Behaviour** is what your code did. This is the pass or fail line.
  62. - **Ran** is whether the check itself could run. A dead database container shows up here, not as a failing test.
  63. - **Covered** is whether every changed file has at least one criterion touching it.
  64. Every report ends with a section called "Not checked", printed even when it is empty. It lists what did not run and why, along with anything the run noticed on its way past. In the run above, that section is where Verify reported that the plan and the code disagreed about how one failure should be classed. Nothing had asked it to compare the two.
  65. ## Going deeper
  66. - [A run, walked through](docs/example-run.md): five criteria on a real change, what the second model flagged, what failed, and what the report found on its own.
  67. - [What it can test](docs/what-it-can-test.md): APIs, databases, CLIs, queues and webhooks, and web UIs in a real browser, and what each one needs from you.
  68. - [How it runs your stack](docs/how-it-runs-your-stack.md): the setup contract, boot modes, seeding, probes, teardown, login capture, and what makes a repository easy to verify.
  69. - [`examples/agent-fail-v2/`](examples/agent-fail-v2/): the raw criteria, report, and HTML report from that run.
  70. ## License
  71. MIT
scripts/review.sh · Bundled script · 67 lines · 2 findings marked
  1. #!/usr/bin/env bash
  2. # Second-opinion review of the criteria. Codex when installed (different
  3. # vendor, different blind spots); otherwise a fresh `claude -p` call that
  4. # receives ONLY the spec and the plan — nothing from the drafting session.
  5. # Fallback chain: codex -> fresh claude -> unavailable (loud, never fatal).
  6. set -e
  7. SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
  8. if [ "${VERIFY_ALLOW_DANGEROUS:-0}" != "1" ]; then
  9. echo "✗ This script runs claude with --dangerously-skip-permissions."
  10. echo " Set VERIFY_ALLOW_DANGEROUS=1 to proceed."
  11. exit 1
  12. fi
  13. CODEX="${CODEX_BIN:-codex}"
  14. CLAUDE="${CLAUDE_BIN:-claude}"
  15. if command -v gtimeout >/dev/null 2>&1; then TIMEOUT_CMD="gtimeout"
  16. elif command -v timeout >/dev/null 2>&1; then TIMEOUT_CMD="timeout"
  17. else echo "✗ timeout command not found. Install: brew install coreutils"; exit 1; fi
  18. RUN_ID="$(cat .verify/current-run 2>/dev/null || echo "")"
  19. [ -n "$RUN_ID" ] || { echo "✗ .verify/current-run missing — create the run first"; exit 1; }
  20. RUN_DIR=".verify/runs/$RUN_ID"
  21. [ -f "$RUN_DIR/criteria.json" ] || { echo "✗ $RUN_DIR/criteria.json not found"; exit 1; }
  22. SPEC_PATH=$(cat .verify/.spec_path 2>/dev/null || echo "")
  23. PROMPT_FILE="$RUN_DIR/reviewer-prompt.txt"
  24. {
  25. cat "$SCRIPT_DIR/prompts/reviewer.txt"
  26. printf "\n\nTHE SPEC:\n"
  27. [ -f "$SPEC_PATH" ] && cat "$SPEC_PATH" || echo "(no spec file)"
  28. printf "\n\nTHE CRITERIA:\n"
  29. cat "$RUN_DIR/criteria.json"
  30. } > "$PROMPT_FILE"
  31. valid_review() { # $1 = json string; every plan id covered, enums correct
  32. echo "$1" | jq -e --slurpfile plan "$RUN_DIR/criteria.json" '
  33. (.criteria | type == "array") and (.missing | type == "array") and
  34. (($plan[0].criteria | map(.id) | sort) == (.criteria | map(.id) | sort)) and
  35. ([.criteria[] | .keep | IN("load-bearing","redundant","unreachable")] | all) and
  36. ([.criteria[] | .codify | type == "boolean"] | all)' > /dev/null 2>&1
  37. }
  38. accept() { # $1 = name, $2 = raw output
  39. local clean
  40. clean=$(echo "$2" | sed '/^```/d' | tr -d '\r')
  41. valid_review "$clean" || return 1
  42. echo "$clean" | jq --arg r "$1" '. + {reviewer: $r}' > "$RUN_DIR/review.json"
  43. }
  44. RAW=""
  45. DONE=0
  46. if command -v "$CODEX" > /dev/null 2>&1; then
  47. # codex exec reads the prompt from stdin when given "-"
  48. RAW=$("$TIMEOUT_CMD" 300 "$CODEX" exec -s read-only - < "$PROMPT_FILE" 2>/dev/null) || RAW=""
  49. [ -n "$RAW" ] && accept "codex" "$RAW" && DONE=1
  50. fi
  51. if [ "$DONE" = "0" ]; then
  52. RAW=$("$TIMEOUT_CMD" 300 "$CLAUDE" -p --model opus --dangerously-skip-permissions < "$PROMPT_FILE" 2>/dev/null) || RAW=""
  53. [ -n "$RAW" ] && accept "fresh-claude" "$RAW" && DONE=1
  54. fi
  55. if [ "$DONE" = "0" ]; then
  56. echo "⚠ NO SECOND OPINION AVAILABLE — criteria were reviewed only by the model that wrote them."
  57. jq -n '{reviewer: "unavailable", criteria: [], missing: []}' > "$RUN_DIR/review.json"
  58. exit 0
  59. fi
  60. echo "✓ Second opinion ($(jq -r '.reviewer' "$RUN_DIR/review.json")) → $RUN_DIR/review.json"
  61. jq -r '.criteria[] | " \(.id): \(.keep) — \(.why)"' "$RUN_DIR/review.json"
  62. jq -r '.missing[]? | " missing: \(.)"' "$RUN_DIR/review.json"

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.