Skip to content
Back to search

claude-security-skills

SkillWorks with Claude CodeGitHub

Claude skills + agent for first-pass third-party (vendor) security reviews, with a built-in skeptic. NIST CSF 2.0 / 800-53 / ISO 27001 mapped, evals included.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 21 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/jasaan/tprm-review-agent with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/jasaan/tprm-review-agent'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/jasaan--claude-security-skills/badge.svg)](/listings/jasaan--claude-security-skills)

HTML

<a href="/listings/jasaan--claude-security-skills"><img src="/api/listings/jasaan--claude-security-skills/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Option 1: Claude Code (no API key needed)

The skills are linked into `.claude/skills/`, so Claude Code picks them up automatically.

git clone https://github.com/jasaan/claude-security-skills.git
cd claude-security-skills
claude

Then ask: *"Run a vendor security review on samples/vendors/brightpath-analytics."*

Option 2: The agent (Claude API)

Requires Python 3.10+ and Claude API credentials (`ANTHROPIC_API_KEY`, or `ant auth login`). A review of a sample vendor makes four API calls and takes about 6 to 8 minutes.

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

python agent/review.py samples/vendors/brightpath-analytics
python agent/review.py samples/vendors/ledgerly-payroll
python evals/score.py --all

**Keep the API key out of files.** On macOS, store it in the keychain once and load it only for the command that needs it:

security add-generic-password -a "$USER" -s anthropic-api-key -w   # prompts; nothing echoed
ANTHROPIC_API_KEY="$(security find-generic-password -a "$USER" -s anthropic-api-key -w)" \
  python agent/review.py samples/vendors/brightpath-analytics

On Linux, `secret-tool` or `pass` do the same job. Never commit a key; `.env` and `*.key` are in `.gitignore`.

Reports land in `output/<vendor>/report.md`. To review a real vendor, make a folder with an `intake.md` and the vendor's documents (PDFs work) and point the agent at it. Use `--dry-run` to see what would be sent without calling the API.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 124 lines
  1. # claude-security-skills
  2. **Claude skills and an agent that run a first-pass third-party security review, with a built-in skeptic.** Point it at a vendor's SOC 2 report, security questionnaire and policies, plus a short intake note about what the business wants to buy, and it produces a risk tier, a control-by-control assessment mapped to NIST CSF 2.0, NIST SP 800-53 Rev. 5 and ISO/IEC 27001:2022, findings with remediation deadlines, contradictions between what the vendor claims and what the evidence shows, follow-up questions, and a recommended outcome for a human risk owner to decide on.
  3. > All sample vendors, people and documents in this repo are fictional.
  4. ## Why this exists
  5. Vendor security reviews are slow, repetitive and important. Most of an analyst's time goes to reading: finding the audit period in a 90-page SOC 2 report, checking whether the product being bought is even in scope, and noticing that the questionnaire says "MFA everywhere" while the auditor found admin accounts without it. That reading is exactly what a language model is good at, as long as the method is explicit and the output is checked.
  6. I've run third-party risk programs in regulated healthcare, and automating this first pass with Claude cut my assessment time by more than 80%. This repo is a public, from-scratch version of that approach: the method is written down as skills, the output is a fixed schema, and an eval checks that the agent catches what it should.
  7. ## How it works
  8. ```
  9. intake note ─┐ 1. DRAFT 2. CHALLENGE 3. RESOLVE
  10. SOC 2 report ─┼─► vendor-security-review ─► skeptic-review ─────► reviewer accepts / rejects /
  11. questionnaire ┤ ├ soc2-report-reader questions the intake, marks "needs an answer" for
  12. policies/DPA ─┘ └ questionnaire-gap-check the vendor and the every challenge ─► findings.json
  13. draft itself report.md
  14. evals/score.py
  15. ```
  16. 1. **Tier first.** The vendor is tiered from the intake note (data, access, criticality) before any vendor evidence is read, so a polished trust center can't talk the tier down.
  17. 2. **Read the audit report.** The `soc2-report-reader` skill pulls out type, period, opinion, scope, exceptions, subservice organizations and the controls we must run ourselves (CUECs), and checks that the purchased product is in scope.
  18. 3. **Assess 12 control domains.** Each is marked met, partially met, not met or not evidenced, with a citation. A questionnaire answer is treated as a claim, not evidence. Domains are modeled on SIG Lite and the CSA CAIQ and mapped to NIST and ISO controls (`skills/vendor-security-review/references/control-catalog.md`).
  19. 4. **Cross-check claims.** The `questionnaire-gap-check` skill flags every claim the evidence contradicts.
  20. 5. **Decide.** Findings get a severity and due date from a written rubric; the outcome is approve, approve with conditions, escalate or reject, with a re-review date by tier.
  21. 6. **Challenge it.** A separate skeptic step, with its own instructions (`skills/skeptic-review/`), questions three parties: the **requesting team** (is the data really "de-identified"? is the tool really "not critical"?), the **vendor** (do the numbers add up? what backs the absolutes?), and the **draft review** itself (does the evidence say what the draft claims? is each severity justified, in either direction?).
  22. 7. **Resolve every challenge.** The reviewer must accept each challenge and change the review, reject it with a reason, or mark it "needs an answer" and turn it into a question for the vendor or the requesting team. The report includes the full challenge log, so a risk owner can see what was questioned and why it was decided the way it was.
  23. The agent never contacts a vendor or approves anything; it writes a report for a person.
  24. ## What's in the repo
  25. | Path | What it is |
  26. |---|---|
  27. | `skills/vendor-security-review/` | The main skill, with risk tiering, severity rubric, control catalog (NIST/ISO mapping) and report template |
  28. | `skills/soc2-report-reader/` | Extracts what reviewers need from a SOC 2 report |
  29. | `skills/questionnaire-gap-check/` | Finds contradictions between vendor claims and evidence |
  30. | `skills/skeptic-review/` | The "naysayer": challenges the requesting team, the vendor and the draft review |
  31. | `agent/` | Python runner: loads the skills, sends the vendor documents (Markdown, text or PDF) to Claude, enforces the output schema, writes `findings.json` and `report.md` |
  32. | `samples/vendors/` | Two fictional vendors: **Brightpath Analytics** (healthcare analytics, nine planted issues) and **Ledgerly Payroll** (a well-run payroll vendor that should pass) |
  33. | `evals/` | Answer keys for each sample vendor and a scorer that checks the agent's output against them |
  34. | `examples/` | Real reports, final `findings.json` and the skeptic's `challenges.json` from the runs below |
  35. ## Quickstart
  36. ### Option 1: Claude Code (no API key needed)
  37. The skills are linked into `.claude/skills/`, so Claude Code picks them up automatically.
  38. ```bash
  39. git clone https://github.com/jasaan/claude-security-skills.git
  40. cd claude-security-skills
  41. claude
  42. ```
  43. Then ask: *"Run a vendor security review on samples/vendors/brightpath-analytics."*
  44. ### Option 2: The agent (Claude API)
  45. Requires Python 3.10+ and Claude API credentials (`ANTHROPIC_API_KEY`, or `ant auth login`). A review of a sample vendor makes four API calls and takes about 6 to 8 minutes.
  46. ```bash
  47. python3 -m venv .venv && source .venv/bin/activate
  48. pip install -r requirements.txt
  49. python agent/review.py samples/vendors/brightpath-analytics
  50. python agent/review.py samples/vendors/ledgerly-payroll
  51. python evals/score.py --all
  52. ```
  53. **Keep the API key out of files.** On macOS, store it in the keychain once and load it only for the command that needs it:
  54. ```bash
  55. security add-generic-password -a "$USER" -s anthropic-api-key -w # prompts; nothing echoed
  56. ANTHROPIC_API_KEY="$(security find-generic-password -a "$USER" -s anthropic-api-key -w)" \
  57. python agent/review.py samples/vendors/brightpath-analytics
  58. ```
  59. On Linux, `secret-tool` or `pass` do the same job. Never commit a key; `.env` and `*.key` are in `.gitignore`.
  60. Reports land in `output/<vendor>/report.md`. To review a real vendor, make a folder with an `intake.md` and the vendor's documents (PDFs work) and point the agent at it. Use `--dry-run` to see what would be sent without calling the API.
  61. ## Results
  62. Real runs on 2026-11-20 with Claude Opus 5.5 (high effort): four API calls per vendor, about 6 to 8 minutes each. Full output, including the skeptic's raw challenges, is in [`examples/`](examples/).
  63. | Vendor | Tier | Outcome | Findings | Skeptic challenges | Eval |
  64. |---|---|---|---|---|---|
  65. | Brightpath Analytics | Tier 1 | Escalate | 1 critical, 7 high, 2 medium, 1 low | 10: 4 accepted, 1 rejected, 5 need an answer | **14/14** |
  66. | Ledgerly Payroll | Tier 2 | Approve with conditions | 0 critical, 0 high, 3 medium, 6 low | 9: 2 accepted, 1 rejected, 6 need an answer | **7/7** |
  67. **What the skeptic caught:**
  68. - **The requesting team.** Brightpath's requester called the data "de-identified"; the skeptic pointed out that the field list includes names, dates of birth and phone numbers. For Ledgerly it asked whether the payroll vendor will get debit authority over the company bank account (not in the intake), and noted that the employee self-service portal will serve all 180 employees, not just the 6 SSO users.
  69. - **The draft review.** It caught Ledgerly controls marked "met" on the strength of a DPA that hasn't been signed yet, and a Brightpath severity that didn't match the rubric. Both were accepted and fixed.
  70. - **My own answer key.** I had planted "MFA for all users" as a contradiction of the SOC 2 MFA exception. The skeptic noted the questionnaire is dated after the vendor's claimed fix, so it isn't a direct contradiction; the real issue is that the fix is unverified. The final review kept the High finding, asked the vendor for evidence, and I updated the eval to accept either treatment.
  71. It also argues for leniency when warranted: it challenged several Ledgerly findings as possibly reflecting missing pages of an abridged report rather than real gaps.
  72. ## Design choices
  73. - **A skeptic with a separate brief.** The same model that wrote a review is a poor judge of it. The challenge step runs as its own call with its own instructions and a mandate to push back in both directions, and the reviewer has to answer every challenge in writing. Inaccurate information often comes from the requesting team, not just the vendor, so the skeptic tests the intake note too.
  74. - **Method in skills, not in code.** The review procedure, rubric and control catalog are plain Markdown that a security team can read, argue with and change without touching Python.
  75. - **Structured output.** The model must answer in a fixed JSON schema (`agent/schema.py`), validated again with Pydantic. Reports are rendered from data, so they're consistent and the results can feed a tracker or GRC tool.
  76. - **Evals, not vibes.** Each sample vendor has an answer key of planted issues, and things that should *not* be flagged. The scorer is deliberately simple keyword-and-field matching so anyone can see why a check passed or failed.
  77. - **Vendor documents are untrusted input.** The Brightpath trust-center page hides an instruction telling automated reviewers to approve the vendor. The agent is told to treat vendor text as data and report attempts like this as findings; the eval checks that it does.
  78. - **Model and safety defaults.** Claude Opus 5.5 with adaptive thinking at high effort. The server-side refusal fallback is on by default (`--no-fallback` turns it off).
  79. ## Limitations
  80. - It's a first pass. A person still decides, and still reads the evidence behind any high or critical finding.
  81. - It only knows what's in the documents. It doesn't check breach databases, attack-surface scans or news.
  82. - The NIST and ISO mappings are indicative cross-references, not an audit opinion.
  83. - The keyword scorer can mark a correct but unusually worded finding as missed; read the report when a check fails.
  84. ## Frameworks referenced
  85. NIST Cybersecurity Framework 2.0 · NIST SP 800-53 Rev. 5 · ISO/IEC 27001:2022 Annex A · Shared Assessments SIG Lite · CSA CAIQ · AICPA SOC 2 Trust Services Criteria · NIST AI RMF and ISO/IEC 42001 (for AI vendors)
  86. ## License
  87. MIT

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.