Skip to content
Back to search

ai-dfir-toolkit

A vendor-neutral collection of Sigma, YARA, and Suricata rules for detecting compromise of LLM applications, MCP servers, ML supply chains, AI infrastructure, AI-powered insider threats, and RAG/vector database attacks.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/depalmar/ai-dfir-toolkit with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/depalmar/ai-dfir-toolkit'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/depalmar--ai-dfir-toolkit/badge.svg)](/listings/depalmar--ai-dfir-toolkit)

HTML

<a href="/listings/depalmar--ai-dfir-toolkit"><img src="/api/listings/depalmar--ai-dfir-toolkit/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Elastic / Kibana

pip install sigma-cli pysigma-backend-elasticsearch
sigma convert -t lucene --without-pipeline ai-dfir-toolkit/**/*.yml > ai-dfir-toolkit.lucene

The rules are vendor-neutral Sigma — see the [pySigma backends](https://github.com/SigmaHQ/pySigma) list to convert to any other SIEM query language.

> **Case sensitivity:** the keyword-based rules (prompt injection, jailbreak, > system-prompt extraction, etc.) follow the Sigma convention that `contains` > matching is case-insensitive — attacker text varies in case, so the rules are > written in lowercase and rely on the backend to fold case. On **Elastic**, > ensure the matched content fields are mapped as analyzed `text` (the default > for string fields), not `keyword`: a `keyword` mapping matches > case-sensitively and will miss capitalized input. Do **not** try to force this > with the `re|i` modifier — Lucene's regex engine does not support the `(?i)` > flag and the resulting query is invalid.

YARA scanning

# Scan a model directory
yara -r ai-dfir-toolkit/03-model-supply-chain/*.yar /path/to/models/

# Scan an MCP config
yara ai-dfir-toolkit/02-mcp-attacks/mcp_tool_poisoning.yar \
  ~/Library/Application\ Support/Claude/claude_desktop_config.json

Suricata

cp ai-dfir-toolkit/**/*.rules /etc/suricata/rules/
echo 'rule-files: [ai-dfir.rules]' >> /etc/suricata/suricata.yaml
suricata -T -c /etc/suricata/suricata.yaml  # validate
systemctl reload suricata

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 302 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # AI/ML DFIR Detection Pack
  2. **Open-source detection signatures for AI/ML attacks and breaches.**
  3. <a href="https://www.sans.org/"><picture>
  4. <source media="(prefers-color-scheme: dark)" srcset="docs/brand/sans-logo-white.svg">
  5. <img src="docs/brand/sans-logo.svg" alt="SANS Institute" height="40">
  6. </picture></a>
  7. Developed with support from [SANS Institute](https://www.sans.org/). The project stays independent and vendor-neutral; see the disclaimer below.
  8. A vendor-neutral collection of Sigma, YARA, and Suricata rules for detecting compromise of LLM applications, MCP servers, ML supply chains, AI infrastructure, AI-powered insider threats, and RAG/vector database attacks.
  9. **Browse it at [depalmar.github.io/ai-dfir-toolkit](https://depalmar.github.io/ai-dfir-toolkit/)** — the
  10. artifact catalog, the detection rules, the ATLAS/OWASP mappings, the case
  11. studies, and the investigation guide, rebuilt from this repository on every
  12. push so the page cannot drift from the content.
  13. ### See it in use
  14. **Start from the alert.** Name the tool the alert mentioned, say where the AI ran and what role it played, and get its traces sorted by how fast each one disappears.
  15. ![Screen recording: Cursor is typed into the landing page, a role is chosen, and its evidence appears grouped as live, rotating and stable](docs/media/start-from-the-alert.gif)
  16. **Collect in the right order.** Pick rows into a plan, group it by volatility, and export it for the collector your team already runs. Credentials are hashed, not copied.
  17. ![Screen recording: a collection plan for Cursor grouped by volatility, with the export switched between a native script, Velociraptor and KAPE](docs/media/collection-plan.gif)
  18. Full-size video: [start from the alert](docs/media/start-from-the-alert.mp4) · [collection plan](docs/media/collection-plan.mp4). How they were recorded: [`docs/media/`](docs/media/README.md).
  19. For the companion investigation guide with attack background, forensic artifacts, Mermaid attack-chain diagrams, and hands-on investigation procedures, see [`docs/ai-dfir-investigation-guide.md`](./docs/ai-dfir-investigation-guide.md).
  20. License: Apache 2.0.
  21. ---
  22. ## Disclaimer
  23. This is an independent personal project. It is not affiliated with, endorsed by, or produced on behalf of any employer. All research is based on public sources — published CVEs, vendor advisories, academic papers, and vendor-neutral security research. Contributions are welcome from the community.
  24. ---
  25. ## Why this exists
  26. Most existing detection content is either locked behind vendor SIEMs or scattered across blog posts. AI/ML attacks need detection coverage that spans:
  27. - **Endpoint** (Claude Desktop / Cursor / Copilot config tampering)
  28. - **Cloud SaaS logs** (Bedrock, Azure OpenAI, M365 Copilot)
  29. - **Network** (vector DB exfil, model exfil, ShadowRay C2)
  30. - **File artifacts** (poisoned pickle models, malicious MCP configs)
  31. This pack uses **open standards only** so the rules can be deployed in any modern detection stack via open Sigma/YARA/Suricata tooling.
  32. ---
  33. ## Structure
  34. ```
  35. ai-dfir-toolkit/
  36. ├── 01-llm-prompt-injection/ # Prompt injection, jailbreaks, indirect injection
  37. ├── 02-mcp-attacks/ # MCP tool poisoning, config tampering, rug pulls
  38. ├── 03-model-supply-chain/ # Pickle exploits, HuggingFace, dependency confusion
  39. ├── 04-ai-infrastructure/ # ShadowRay, Triton, MLflow, GPU abuse
  40. ├── 05-copilot-assistant-abuse/ # M365 Copilot, GitHub Copilot, Claude, Cursor
  41. ├── 06-rag-vector-db/ # Vector DB exposure, RAG poisoning
  42. ├── 07-runtime-ai-malware/ # Malware calling an LLM API at runtime
  43. ├── 08-agentic-orchestration/ # Agent-as-operator, AI service API as C2
  44. ├── 09-agent-memory-forensics/ # Memory as persistence, context poisoning
  45. ├── collectors/ # Read-only acquisition with chain of custody
  46. ├── playbooks/ # CACAO v2.0 response playbooks
  47. ├── artifacts/ # Machine-readable AI agent artifact catalog
  48. ├── skills/ # Agent skills for maintaining the catalog
  49. ├── docs/ # Investigation guide
  50. ├── tests/ # Sample events / test files
  51. ├── MAPPINGS.md # ATLAS + OWASP cross-reference
  52. └── README.md
  53. ```
  54. Each category directory contains a `README.md` describing the threats covered and rule files in their native format (`.yml` for Sigma, `.yar` for YARA, `.rules` for Suricata).
  55. ---
  56. ## Rule formats
  57. | Format | Use Case | Where it Deploys |
  58. |--------|----------|------------------|
  59. | **Sigma** (`.yml`) | Generic log-based detection | Any SIEM via [pySigma](https://github.com/SigmaHQ/pySigma) backends |
  60. | **YARA** (`.yar`) | File / memory artifacts | EDR platforms, malware analysis, file scanning pipelines |
  61. | **Suricata** (`.rules`) | Network traffic | Suricata, Snort (compatible subset), Zeek (via translation) |
  62. **All rules use open formats.** No vendor-specific query languages, no proprietary field schemas. Convert to your platform using pySigma backends.
  63. ---
  64. ## Quick start
  65. ### Elastic / Kibana
  66. ```bash
  67. pip install sigma-cli pysigma-backend-elasticsearch
  68. sigma convert -t lucene --without-pipeline ai-dfir-toolkit/**/*.yml > ai-dfir-toolkit.lucene
  69. ```
  70. The rules are vendor-neutral Sigma — see the [pySigma backends](https://github.com/SigmaHQ/pySigma) list to convert to any other SIEM query language.
  71. > **Case sensitivity:** the keyword-based rules (prompt injection, jailbreak,
  72. > system-prompt extraction, etc.) follow the Sigma convention that `contains`
  73. > matching is case-insensitive — attacker text varies in case, so the rules are
  74. > written in lowercase and rely on the backend to fold case. On **Elastic**,
  75. > ensure the matched content fields are mapped as analyzed `text` (the default
  76. > for string fields), not `keyword`: a `keyword` mapping matches
  77. > case-sensitively and will miss capitalized input. Do **not** try to force this
  78. > with the `re|i` modifier — Lucene's regex engine does not support the `(?i)`
  79. > flag and the resulting query is invalid.
  80. ### YARA scanning
  81. ```bash
  82. # Scan a model directory
  83. yara -r ai-dfir-toolkit/03-model-supply-chain/*.yar /path/to/models/
  84. # Scan an MCP config
  85. yara ai-dfir-toolkit/02-mcp-attacks/mcp_tool_poisoning.yar \
  86. ~/Library/Application\ Support/Claude/claude_desktop_config.json
  87. ```
  88. ### Suricata
  89. ```bash
  90. cp ai-dfir-toolkit/**/*.rules /etc/suricata/rules/
  91. echo 'rule-files: [ai-dfir.rules]' >> /etc/suricata/suricata.yaml
  92. suricata -T -c /etc/suricata/suricata.yaml # validate
  93. systemctl reload suricata
  94. ```
  95. ---
  96. ## Artifact Catalog
  97. [`artifacts/`](artifacts/) is a machine-readable catalog of the endpoint
  98. artifacts left by AI coding agents, local LLM runtimes, agentic workflow
  99. engines, and Model Context Protocol components — install paths, credential
  100. storage, MCP configs, listening ports, process relationships, and registry keys,
  101. each rated by forensic value and sourcing confidence, with collection priorities
  102. for IR triage.
  103. Eleven tool categories are covered: local LLM runtimes, coding agents and IDEs,
  104. agent frameworks, coding-agent CLIs, workflow engines, browser agents,
  105. computer-use agents, MCP hosts, cloud agents, code-completion tools, and
  106. code-execution agents. Alongside them sit documented incident case studies, each
  107. carrying its own provenance and recording analyst disagreement in a `contested`
  108. field rather than resolving it.
  109. Ships vendor-neutral Sigma detection rules and an osquery inventory pack, plus
  110. generated feeds in four downstream formats:
  111. | Export | Consumed by |
  112. |--------|-------------|
  113. | [ForensicArtifacts](https://github.com/ForensicArtifacts/artifacts) | Plaso, GRR, Timesketch |
  114. | [KAPE](https://ericzimmerman.github.io/KapeDocs/) targets | KAPE triage collections |
  115. | [Velociraptor](https://docs.velociraptor.app/) artifacts | Velociraptor hunts |
  116. | CSV / JSON | anything else — see [`artifacts/docs/api/`](artifacts/docs/api/) |
  117. ### Threat intel
  118. On top of the catalog sits a threat-intel layer:
  119. - **Actor profiles:** 20 profiles, covering attributed groups and
  120. reporter-named clusters.
  121. - **Case studies:** 22, with 221 indicators.
  122. - **Pinned frameworks:** ATLAS 2026.10, ATT&CK 20.0, CISA KEV, and the OWASP
  123. LLM and Agentic 2026 lists.
  124. Each profile is derived from the cases and reported sightings that cite it. It
  125. gives the techniques the actor uses, the rule in this repository that detects
  126. each one, the catalogued tools the actor abused, and which of those tools'
  127. artifacts to collect first. The site adds:
  128. - a cross-filtering Threat intel view: ATLAS matrix with coverage gaps,
  129. victimology, timeline, technique heatmap, detection backlog, actor similarity
  130. and clusters
  131. - a Maltego-style graph workbench
  132. - a collection plan that generates a self-contained triage script (PowerShell or
  133. bash) or the equivalent Velociraptor / KAPE command
  134. Feeds: [`intel.json`](artifacts/docs/api/intel.json), a
  135. [STIX 2.1 bundle](artifacts/docs/api/stix/ai-dfir-intel.json) for OpenCTI or
  136. MISP, and [Maltego import CSVs](artifacts/docs/api/graph/). Two scheduled
  137. workflows keep it current:
  138. - the framework pins refresh weekly and auto-merge when only reference data
  139. changed
  140. - new public reporting arrives as `intel-candidate` issues for a person to
  141. research
  142. For current counts — catalogued tools, documented artifacts, credential
  143. locations, MCP config paths, case studies, and the risk and confidence
  144. breakdowns — see the Contents table in [`artifacts/README.md`](artifacts/README.md).
  145. That table is generated by `scripts/readme_counts.py` and gated in CI, so it is
  146. the one place those numbers are guaranteed current.
  147. ---
  148. ## Coverage overview
  149. 76 rule files containing 171 individual signatures — nine attack-class categories
  150. plus a cross-tool endpoint set:
  151. | Category | Files | Signatures | ATLAS Techniques | OWASP LLM 2026 | OWASP Agentic |
  152. |----------|-------|-----------:|------------------|-----------|---------------|
  153. | LLM Prompt Injection | 8 | 10 | T0051, T0054, T0029 | LLM01, LLM02, LLM06, LLM08 | ASI01 |
  154. | MCP Attacks | 5 | 14 | T0010, T0115.002, T0110, T0086 | LLM02, LLM03, LLM04 | ASI02, ASI03, ASI04, ASI05 |
  155. | Model Supply Chain | 8 | 25 | T0010, T0018, T0020 | LLM04 | ASI03, ASI04, ASI05 |
  156. | AI Infrastructure | 9 | 31 | T0011, T0049, T0029 | LLM06 | — |
  157. | Copilot/Assistant Abuse | 8 | 19 | T0086, T0024 | LLM02, LLM03, LLM04 | ASI01, ASI02, ASI03, ASI04, ASI05 |
  158. | RAG / Vector DB | 5 | 19 | T0020 | LLM01, LLM02, LLM09 | ASI01, ASI06 |
  159. | Runtime AI-Malware | 8 | 16 | T0096, T0086 | LLM01, LLM03 | — |
  160. | Agentic Orchestration & AI-Service C2 | 8 | 17 | T0096, T0086, T0054, T0012, T0055, T0116 | LLM03 | ASI02 |
  161. | Agent Memory & Context Poisoning | 2 | 5 | T0080, T0080.000, T0086 | LLM01, LLM02, LLM03 | ASI06 |
  162. | Endpoint (cross-tool) | 15 | 15 | T0053, T0081, T0083, T0098, T0010.001 | LLM02, LLM03, LLM04, LLM06 | ASI02, ASI03, ASI04, ASI05 |
  163. | **Total** | **76** | **171** | | | |
  164. The endpoint set lives in [`artifacts/detections/`](artifacts/detections/) and is
  165. scoped to agent behaviour on a host rather than to one attack class, so it applies
  166. across every tool in the artifact catalog. An osquery pack there answers the
  167. inventory question ("which hosts have this") that the Sigma rules cannot.
  168. *Signature count includes multi-document Sigma YAML, multiple `rule` blocks inside a single YARA file, and multiple `alert` lines inside a single Suricata `.rules` file, except one `flowbits:noalert` helper that only sets state for the rule after it. One file often covers several related variants.*
  169. See [MAPPINGS.md](./MAPPINGS.md) for per-rule mappings.
  170. ---
  171. ## Tuning notes
  172. These rules are written to err toward signal over noise, but every environment is different. Each rule includes:
  173. - `falsepositives:` section listing known FP scenarios
  174. - `level:` field (`low` / `medium` / `high` / `critical`) — start with `high`+ in production
  175. - Tunable selectors so you can scope to specific orgs/users/namespaces
  176. Recommended rollout:
  177. 1. Deploy to a test index/workspace for 7 days
  178. 2. Triage hits, tune `selection` and `filter` blocks
  179. 3. Promote to production with appropriate severity
  180. ---
  181. ## Testing
  182. The `tests/` directory contains sample artifacts (malicious and benign) for validating rule correctness. Run:
  183. ```bash
  184. cd tests/
  185. ./validate.sh
  186. ```
  187. Expected result: 15 passes, 0 failures.
  188. The suite hard-exits if the `yara` binary is missing rather than reporting a
  189. clean run — absent tooling would otherwise score every "expect no matches"
  190. assertion as a pass and the suite would look healthy while testing nothing.
  191. ---
  192. ## Contributing
  193. PRs welcome. Rule submission requirements:
  194. 1. Reference an attack technique — ATLAS ID, CVE, or published research
  195. 2. Include test data in `tests/` — sample log line, file, or PCAP
  196. 3. Document false positives in the rule
  197. 4. Use lowercase, snake_case filenames matching the threat
  198. 5. Tag with `attack.atlas.txxxx` in Sigma rules
  199. Sigma format: follow the [Sigma specification](https://github.com/SigmaHQ/sigma-specification).
  200. ---
  201. ## References
  202. - [MITRE ATLAS](https://atlas.mitre.org/)
  203. - [OWASP Top 10 for LLM Applications 2026](https://genai.owasp.org/llm-top-10/) — detection content maps to the **2026** list; eight of the ten IDs changed meaning from 2025, so an ID quoted from an older report names a different category here
  204. - [NIST AI RMF](https://www.nist.gov/itl/ai-risk-management-framework)
  205. - [Sigma HQ](https://github.com/SigmaHQ/sigma)
  206. - [AI Incident Database](https://incidentdatabase.ai/)
  207. ---
  208. ## License
  209. Apache License 2.0 — see [LICENSE](./LICENSE).
  210. You are free to use, modify, and redistribute these rules in commercial and non-commercial settings. Attribution appreciated but not required.
  211. Catalog data under `artifacts/catalog/`, `artifacts/case-studies/`, and
  212. `artifacts/docs/api/` is licensed CC BY 4.0; see
  213. [`artifacts/LICENSE-DATA`](./artifacts/LICENSE-DATA). All other content,
  214. including the scripts and schema under `artifacts/`, is under the repository's
  215. Apache-2.0 licence.
  216. ---
  217. **Maintainer:** Raymond DePalma — independent security researcher.
  218. This is a personal project and is not affiliated with any employer.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.