Skip to content
Back to search

defending-code-reference-harness

SkillWorks with Claude CodeOfficial: AnthropicGitHub

Skills for threat modeling, scanning, triage, patching, plus an autonomous scanning harness you can /customize

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/anthropics-defending-code-reference-harness/badge.svg)](/listings/anthropics-defending-code-reference-harness)

HTML

<a href="/listings/anthropics-defending-code-reference-harness"><img src="/api/listings/anthropics-defending-code-reference-harness/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

git clone https://github.com/anthropics/defending-code-reference-harness
cd defending-code-reference-harness
claude

# 30-sec intro + guided first run on the canary target
> /quickstart

> /quickstart how do I port the pipeline to Java?
> /quickstart how do I triage all these bugs?

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 358 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Defending Code Reference Harness
  2. A reference implementation for autonomous vulnerability discovery and
  3. remediation with Claude, based on our learnings from [partnering with security
  4. teams at several organizations](https://www.anthropic.com/glasswing)
  5. since launching Claude Mythos Preview. For a write up of these learnings along with
  6. best practices, see the [accompanying blog post](https://claude.com/blog/using-llms-to-secure-source-code)
  7. (also available in [`blog-post.md`](docs/blog-post.md)). For a lightweight SDK-only
  8. walkthrough of the same recon → find → triage → report → patch loop, see the
  9. [companion cookbook](https://platform.claude.com/cookbook/claude-agent-sdk-06-the-vulnerability-detection-agent).
  10. This repo is not maintained and is not accepting contributions.
  11. > 🔒 **Want a managed option?** Anthropic offers
  12. > [Claude Security](https://claude.com/product/claude-security), a hosted product
  13. > that finds and fixes vulnerabilities in your source code across multiple
  14. > projects. Claude Security scans your repository for vulnerabilities,
  15. > applies a multi-stage verification pipeline to reduce false positives, and
  16. > lets you manage findings through their lifecycle: triage, fix validation,
  17. > and rapid fix generation.
  18. >
  19. > This repository is an open-source reference implementation based on general
  20. > best practices for finding vulnerabilities using Claude. You can use it to
  21. > build your own vulnerability finding pipeline, customize the logic, and it
  22. > can be used with whatever access you have to Claude APIs (including
  23. > Bedrock, Vertex, or Azure).
  24. ## Contents
  25. - **Claude Code skills**: `/quickstart`, `/threat-model`, `/vuln-scan`,
  26. `/triage`, `/patch`, `/customize`: interactive scoping, scanning, triage,
  27. and patching. Open this repo in Claude Code and run `/quickstart` to get
  28. oriented.
  29. - **`harness/`**: the autonomous reference pipeline (recon → find → verify
  30. → report → patch), configured for finding C/C++ memory vulnerabilities
  31. using Docker and ASAN. This harness is a **reference, not a product**.
  32. The general shape, prompts, and sandboxing are reusable, but the harness
  33. will not work on every codebase out of the box. Run `/customize` to port it
  34. to your language, detector, or vuln class.
  35. - **Detection & response**: the `/dnr-hunt` and `/dnr-respond` skills plus
  36. `dnr_harness/`, their autonomous mirror (`dnr-pipeline`). Everything else
  37. in this repo is preventive; this track assumes an attacker is already in
  38. the logs — hunt the corpus, scope the damage, and propose a response.
  39. Demo target: `targets/dnrcanary/`. See
  40. [docs/detection-response.md](docs/detection-response.md).
  41. > ⚠️ **Security:** `/quickstart`, `/threat-model`, `/vuln-scan`, and `/triage`
  42. > only read and write files. Running `/patch` on static findings (`TRIAGE.json`
  43. > or `VULN-FINDINGS.json`) is likewise read- and write-only. `/customize` edits
  44. > the harness code and runs validation commands. Any of these skills are safe to
  45. > run unsandboxed, as long as you review and approve each tool use in Claude Code.
  46. > The autonomous pipelines (`vuln-pipeline`, `dnr-pipeline`, and `/patch` on
  47. > pipeline results) **execute target code**, so they refuse to run outside of
  48. > a gVisor sandbox
  49. > unless explicitly overridden. To get set up, run `scripts/setup_sandbox.sh` once,
  50. > then invoke the pipeline via `bin/vp-sandboxed`. The detection & response
  51. > skills (`/dnr-hunt`, `/dnr-respond`) additionally run the demo app on
  52. > `127.0.0.1` to verify PoCs. See [docs/security.md](docs/security.md)
  53. > and [docs/agent-sandbox.md](docs/agent-sandbox.md) for more details.
  54. ## Getting Started
  55. ```bash
  56. git clone https://github.com/anthropics/defending-code-reference-harness
  57. cd defending-code-reference-harness
  58. claude
  59. # 30-sec intro + guided first run on the canary target
  60. > /quickstart
  61. > /quickstart how do I port the pipeline to Java?
  62. > /quickstart how do I triage all these bugs?
  63. ```
  64. ## Further Reading
  65. - [**Blog Post**](docs/blog-post.md) · The accompanying blog post with learnings + best practices
  66. - [**Pipeline**](docs/pipeline.md) · How it works: diagram, stages, CLI flags
  67. - [**Security**](docs/security.md) · Sandboxing, what not to mount
  68. - [**Agent sandbox**](docs/agent-sandbox.md) · gVisor isolation + egress allowlist for every agent
  69. - [**Best practices**](docs/best-practices.md) · Field-tested principles: verification, severity, iteration, large codebases
  70. - [**Prompting**](docs/prompting.md) · Prompting the model for defensive security tasks
  71. - [**Threat model**](docs/threat-model.md) · Why a threat model cuts false positives, and the `/threat-model` skill
  72. - [**Detection & response**](docs/detection-response.md) · Hunting an attacker already in the logs; the D&R skills and pipeline
  73. - [**Customize**](docs/customizing.md) · Port to my stack; which files change and why
  74. - [**Patching**](docs/patching.md) · Generate and verify fixes for verified crashes
  75. - [**Other use cases**](docs/other-use-cases.md) · Binary analysis, embedded, bug chains, threat intel
  76. - [**Troubleshooting**](docs/troubleshooting.md) · Duplicates, rate limits, subagent model pinning
  77. - [**Safeguards**](https://support.claude.com/en/articles/14604842-real-time-cyber-safeguards-on-claude) · Block for dangerous cyber work
  78. ---
  79. ## Ramp Up
  80. The most successful security teams we've partnered with are those
  81. that have gotten hands-on the fastest. Though it's tempting to
  82. spend months designing the perfect pipeline, we recommend starting
  83. small on Day 1 and building from there as learnings come. The
  84. steps below follow that pattern and set an ambitious (but reasonable)
  85. pace based on what we've seen.
  86. | | | |
  87. |-------------------------------------------------------------------------------------|--------------|--------------------------------------------------------------|
  88. | [Step 1](#step-1-day-1-build-a-threat-model-and-run-your-first-static-scan--triage) | **Day 1** | Build a threat model and run your first static scan + triage |
  89. | [Step 2](#step-2-day-2-run-the-reference-pipeline-on-a-cc-library) | **Day 2** | Run the reference pipeline on a C/C++ library |
  90. | [Step 3](#step-3-days-3-5-customize-the-pipeline-for-your-target) | **Days 3-5** | Customize the pipeline for your target |
  91. | [Step 4](#step-4-week-2-start-autonomous-scanning-triage-and-patching) | **Week 2** | Start autonomous scanning, triage, and patching |
  92. | [Step 5](#step-5-optional-detection--response) | **Optional** | Hunt a planted campaign in logs (detection & response) |
  93. ### Step 1 (Day 1): Build a threat model and run your first static scan + triage
  94. Day 1 is focused on seeing the whole loop end-to-end. Using only the
  95. interactive skills, you'll build a threat model, run a static scan scoped
  96. by it, triage what comes back, and draft candidate fixes. You'll finish
  97. the day with a threat model, a ranked list of static findings, and candidate
  98. patches.
  99. The relevant skills **only read and write files** in your repo. As long as you
  100. run Claude Code interactively and approve each tool use, no sandbox is needed.
  101. ```bash
  102. # Pin every subagent to the model you want
  103. export CLAUDE_CODE_SUBAGENT_MODEL=<model-id>
  104. claude
  105. # 0. intro + guided first run
  106. > /quickstart
  107. # 1. Build a threat model (aim before you shoot)
  108. > /threat-model bootstrap targets/canary
  109. # 2. Run a static scan, scoped by that threat model
  110. > /vuln-scan targets/canary
  111. # 3. Verify, dedupe, and rank what came back
  112. > /triage targets/canary/VULN-FINDINGS.json
  113. # 4. Generate candidate fixes for the verified findings
  114. > /patch ./TRIAGE.json --repo targets/canary
  115. ```
  116. This flow produces `THREAT_MODEL.md`, `VULN-FINDINGS.{json,md}`,
  117. `TRIAGE.{json,md}`, and `PATCHES/`.
  118. The vulnerability candidates produced in Step 1 come from Claude's static
  119. review of the source (nothing is built or run), so expect more false positives on
  120. any non-canary targets. In Step 2, you'll produce *execution-verified* findings.
  121. > **Note:** on the canary target, `/triage` may dismiss the scan's findings
  122. > as false positives. `entry.c` announces itself as deliberately vulnerable
  123. > demo code, and `/triage` correctly excludes bugs in test / fixture code.
  124. > To see the full confirm / dedupe / false positive flow, run it on the
  125. > curated fixture instead (`/triage .claude/skills/triage/fixtures/canary-findings.json
  126. > --repo targets/canary`) or point the Step 1 skills at your own code.
  127. ### Step 2 (Day 2): Run the reference pipeline on a C/C++ library
  128. On Day 2, you'll move from interactive skills to your first autonomous
  129. run using the reference pipeline. You'll run the full recon → find →
  130. verify → report loop in your environment on a known-vulnerable open-source
  131. library, then generate a candidate patch for what it finds. You'll finish
  132. with a set of reproducible crashes, exploitability reports, and candidate patches,
  133. along with a feel for how the pipeline works.
  134. Running the pipeline is simple:
  135. ```bash
  136. # One-time setup
  137. python3 -m venv .venv && .venv/bin/pip install -e .
  138. ./scripts/setup_sandbox.sh # installs gVisor, builds the agent images, and verifies isolation; note: requires Docker
  139. export ANTHROPIC_API_KEY=sk-ant-... # or CLAUDE_CODE_OAUTH_TOKEN, or Bedrock — see docs/agent-sandbox.md
  140. # Run the recon → find → verify → report loop
  141. bin/vp-sandboxed run drlibs --model <model-id> --runs 3 --parallel --stream --auto-focus
  142. # Generate a candidate patch for each finding
  143. bin/vp-sandboxed patch results/drlibs/<timestamp>/ --model <model-id>
  144. # Or, ask Claude Code to launch the pipeline and watch the run for you
  145. claude
  146. > run the pipeline on drlibs and explain findings as they come
  147. ```
  148. Results from the loop land in a `results/drlibs/<timestamp>/` directory. With
  149. the `--stream` flag, the first report will appear in minutes under `reports/bug_NN/`.
  150. > ⚠️ **`run` spawns autonomous agents.** The pipeline runs each agent
  151. > inside a gVisor container with egress restricted to the Claude API.
  152. > Agent-spawning subcommands refuse to start outside it unless explicitly
  153. > overridden. For more information, see [docs/security.md](docs/security.md)
  154. > and [docs/agent-sandbox.md](docs/agent-sandbox.md).
  155. Under the hood, the pipeline walks through seven stages:
  156. 1. **Build**: Compiles the target into a Docker image with ASAN (the memory
  157. error detector for C and C++). The pipeline builds this image automatically
  158. on first run using the target's `Dockerfile`.
  159. 2. **Recon**: A lightweight agent reads the source inside a network-isolated
  160. container and proposes a partition, i.e., *"here are N distinct input-parsing
  161. subsystems worth attacking separately"*, so that parallel find agents explore
  162. different areas instead of converging on the same bug. Without the `--auto-focus`
  163. flag, the pipeline uses the `focus_areas` list from the target's `config.yaml`.
  164. 3. **Find**: N agents run in parallel, each in its own isolated container.
  165. Each agent reads the source, crafts malformed inputs, and runs the ASAN
  166. binary until a given input produces a crash 3 out of 3 times.
  167. 4. **Verify**: A separate grader agent reproduces each crash in a fresh
  168. container that the find agent hasn't touched. The only thing that crosses over
  169. from the find agent to the grader is the proof of concept it produced.
  170. 5. **Dedupe**: A judge agent compares verified crashes against bugs already
  171. reported and decides whether each is a new bug, a better example of a known
  172. bug, or a duplicate to skip.
  173. 6. **Report**: A report agent writes a structured exploitability analysis per
  174. unique bug, including details on primitive class, reachability, escalation
  175. path, and severity.
  176. 7. **Patch** (the separate patch command above): A patch agent writes a proposed
  177. fix, and a grader agent confirms that the new code builds, that the original
  178. proof of concept input no longer crashes, that the target's test suite still
  179. passes, and that a fresh find agent can't find a way around the fix.
  180. For more details, see [docs/pipeline.md](docs/pipeline.md).
  181. ### Step 3 (Days 3-5): Customize the pipeline for your target
  182. On Days 3-5, you'll customize the harness for your own target. First, you'll
  183. point the Step 1 skills at your code, then you'll use `/customize` to port the
  184. pipeline to your stack. By the end of the week, you'll have a `targets/<your-service>/`
  185. directory that the pipeline can run against, validated with a single smoke run
  186. of the pipeline, and ready to scale up in Step 4.
  187. While the reference pipeline is designed for finding memory vulnerabilities in C and C++
  188. code, its shape is generic. Porting it to a new vuln class or language just means
  189. answering the following questions for your target stack:
  190. | Question | C/C++ Reference | Your target (examples) |
  191. |-----------------------------------------|-----------------------------------|------------------------------------------------|
  192. | What signals a finding? | ASAN crash signature | exception / canary file / DNS callback |
  193. | What does a proof of concept look like? | crashing input file | HTTP request sequence / tx list / test harness |
  194. | How is the target built and run? | `Dockerfile` (using clang + ASAN) | your language's build in a container |
  195. Before customizing, point the Step 1 skills at your own code. As a reminder,
  196. they're read- and write-only, so they can run unsandboxed.
  197. ```bash
  198. claude
  199. > /quickstart how do I customize this for ~/code/my-service?
  200. > /threat-model bootstrap-then-interview ~/code/my-service
  201. > /vuln-scan ~/code/my-service
  202. > /triage ~/code/my-service/VULN-FINDINGS.json --repo ~/code/my-service
  203. ```
  204. Then, use the artifacts produced by those skills in the `/customize` skill,
  205. which modifies the harness for your codebase.
  206. ```bash
  207. > /customize use ~/code/my-service/{THREAT_MODEL.md,VULN-FINDINGS.json} and ./TRIAGE.md
  208. ```
  209. When `/customize` is done, you'll have a `targets/my-service/` directory
  210. set up. Validate it with a smoke run of the pipeline before scaling up.
  211. ```bash
  212. bin/vp-sandboxed run my-service --model <model-id> --runs 1
  213. ```
  214. For more details, see [docs/customizing.md](docs/customizing.md).
  215. ### Step 4 (Week 2): Start autonomous scanning, triage, and patching
  216. In Week 2, you'll use the pipeline you customized in Step 3 on your own
  217. targets, adding an *outer* loop to the inner pipeline loop - run multiple
  218. pipeline scans, triage the findings from across those runs, patch based
  219. on prioritization, and repeat.
  220. ```bash
  221. # Scan - run a wave of parallel runs against your target
  222. bin/vp-sandboxed run my-service --model <model-id> --runs 5 --parallel --stream --auto-focus
  223. # Triage - dedupe and rank every finding across all waves using your threat model
  224. > /triage results/my-service/ --repo ~/code/my-service --auto --votes 5
  225. # Patch - generate and validate fixes, starting with what triage ranked the highest
  226. > /patch results/my-service/<timestamp>/ --model <model-id>
  227. ```
  228. > ⚠️ Follow the same sandboxing guidelines as in
  229. > [Step 2](#step-2-day-2-run-the-reference-pipeline-on-a-cc-library)
  230. A given pipeline run already verifies and deduplicates its own findings.
  231. `/triage` works across many pipeline runs. When pointed at the `results/`
  232. directory, it collapses duplicates across all runs (and any static findings
  233. from `/vuln-scan` if present), recalibrates severity ratings against your
  234. threat model, and attempts to route every finding to the component owner.
  235. When possible, patching findings quickly helps keep the outer loop as
  236. productive as possible. When findings are fixed, the model can't re-find
  237. them, and instead will surface net new, typically deeper issues. As you run
  238. more pipeline waves, the number of findings will likely go down, but the
  239. complexity will likely also go up. If quick patching isn't possible, even
  240. just recording prior findings in the target's `known_bugs` can help steer
  241. future runs toward newer bugs.
  242. Autonomous triage and patching are still open issues, and this reference
  243. harness doesn't fully solve them. The verification strategies in `/patch`
  244. help raise the bar, but severity and prioritization are ultimately
  245. judgments about your environment, and verified patches are not always
  246. upstreamable. Many partners have reported these steps as their current
  247. bottlenecks, and you should budget real engineering time for them.
  248. For more details, see [docs/triage.md](docs/triage.md) and
  249. [docs/patching.md](docs/patching.md).
  250. ### Step 5 (Optional): Detection & response
  251. Everything above is about finding vulnerabilities before an attacker does.
  252. The detection & response track is an example of how you may use Claude if an
  253. attacker is already in your systems and shows up in your logs: find them,
  254. scope the damage, and propose the response.
  255. The demo target is `targets/dnrcanary`, a deliberately vulnerable web app
  256. with a planted attack campaign hiding in a week of generated logs.
  257. ```bash
  258. pip install flask pyyaml # flask runs the app; pyyaml runs the grader
  259. python3 targets/dnrcanary/generate_logs.py --seed 42 # required first — logs are local-only
  260. > /dnr-hunt # no alert in hand: hunt the corpus
  261. > /dnr-respond INC-1 # lead in hand: verdict, blast radius, proposed plan
  262. python3 targets/dnrcanary/grade.py results/dnrcanary/<ts>/INCIDENTS.json # self-score against ground truth
  263. ```
  264. The same exercise also runs unattended, mirroring how `vuln-pipeline`
  265. mirrors the interactive scanning skills:
  266. ```bash
  267. bin/vp-sandboxed dnr-pipeline run targets/dnrcanary --model <model-id>
  268. ```
  269. Confirmed vulnerabilities flow into the same `/triage` → `/patch` loop as
  270. the static track.
  271. → Deeper: [docs/detection-response.md](docs/detection-response.md)
  272. ## Looking Forward
  273. After the initial ramp up, the teams we've worked with have tended to invest in a
  274. few directions:
  275. 1. Reviewing all their internal repos and key open-source dependencies,
  276. ranking which are the most important to scan (e.g., based on their exposure,
  277. history of CVEs, business-criticality), then working through scanning the
  278. list in priority order.
  279. 2. Setting up bespoke infrastructure for scanning to move scans off of laptops
  280. or one-off VMs. The most successful teams resist the urge to build the perfect
  281. scanning platform before scaling up.
  282. 3. Incorporating scans into their SDLC. Some teams have set up recurring scans
  283. (e.g., daily, weekly) or have added scanning into their CI pipelines.
  284. 4. Testing and experimenting with the models to find what works best for them.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.