Skip to content
Back to search

Auto-Empirical-Research-Skills

🔬 A curated collection of 23,000+ agent skills for empirical research across 8 social science disciplines. | 精选 23,000+ AI Agent 技能库,覆盖8大社会科学学科的实证研究。CoPaper.AI 20分钟完成一篇可复现的规范实证论文,并支持用户上传 Skills。-- Maintained by CoPaper.AI from Stanford REAP.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026. Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/brycewang-stanford--auto-empirical-research-skills/badge.svg)](/listings/brycewang-stanford--auto-empirical-research-skills)

HTML

<a href="/listings/brycewang-stanford--auto-empirical-research-skills"><img src="/api/listings/brycewang-stanford--auto-empirical-research-skills/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 77 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "$schema": "https://json.schemastore.org/claude-code-marketplace.json",
  3. "name": "auto-empirical-research-skills",
  4. "version": "1.1.0",
  5. "description": "First-party, install-ready Claude Code plugins from the Auto-Empirical-Research-Skills collection (Stanford REAP x CoPaper.AI). Add this marketplace to one-command-install the empirical-research skill stacks (AER manuscript pipeline + explicit Python / Stata / R analysis pipelines); the wider catalog of 50+ third-party skills is browsable in the repo and consumed via copy-into-.claude/skills or --plugin-dir.",
  6. "owner": {
  7. "name": "Bryce Wang",
  8. "email": "brycew6m@stanford.edu",
  9. "url": "https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills"
  10. },
  11. "plugins": [
  12. {
  13. "name": "aer-skills",
  14. "source": "./skills/50-brycewang-aer-skills",
  15. "version": "1.0.0",
  16. "description": "Nine-skill stack for top-5 economics manuscripts (AER / AER: Insights / AEJ): topic selection, modern causal identification (DiD / IV / RDD / SCM / Bartik), referee-anticipating robustness, Keith-Head-style introductions, AER booktabs tables, AEA Data and Code Availability deposits (openICPSR-ready), submission preflight, and R&R rebuttal letters. Ships Stata / R / Python templates and classic-AER-paper exemplars.",
  17. "author": { "name": "Bryce Wang", "email": "brycew6m@stanford.edu" },
  18. "license": "MIT",
  19. "homepage": "https://github.com/brycewang-stanford/AER-skills",
  20. "repository": "https://github.com/brycewang-stanford/AER-skills",
  21. "category": "academic",
  22. "keywords": [
  23. "economics", "aer", "american-economic-review", "aej", "causal-inference",
  24. "difference-in-differences", "instrumental-variables", "regression-discontinuity",
  25. "synthetic-control", "replication", "academic-writing", "empirical-research"
  26. ]
  27. },
  28. {
  29. "name": "empirical-analysis-python",
  30. "source": "./plugins/empirical-analysis-python",
  31. "version": "1.0.0",
  32. "description": "Explicit 8-step empirical-analysis pipeline in the traditional Python econometrics stack (pandas + statsmodels + linearmodels + pyfixest + rdrobust + econml + causalml): cleaning -> variable construction -> Table 1 -> diagnostics -> estimation (OLS/IV/DID/RDD/PSM/SCM/DML/Causal Forest) -> robustness -> mechanism/heterogeneity/mediation -> publication-ready tables & figures. Includes epidemiology and ML-causal modes.",
  33. "author": { "name": "Bryce Wang", "email": "brycew6m@stanford.edu" },
  34. "license": "CC-BY-SA-4.0",
  35. "homepage": "https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills",
  36. "repository": "https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills",
  37. "category": "academic",
  38. "keywords": [
  39. "econometrics", "causal-inference", "python", "difference-in-differences",
  40. "instrumental-variables", "regression-discontinuity", "double-machine-learning",
  41. "empirical-research", "replication"
  42. ]
  43. },
  44. {
  45. "name": "empirical-analysis-stata",
  46. "source": "./plugins/empirical-analysis-stata",
  47. "version": "1.0.0",
  48. "description": "Explicit 8-step empirical-analysis pipeline in the traditional Stata ecosystem (reghdfe + ivreg2 + csdid + did_imputation + sdid + rdrobust + synth + psmatch2 + teffects + esttab + coefplot): from use/import to .tex/.rtf tables and .pdf figures, with the full modern DID toolkit (bacondecomp + honestdid + rwolf + ritest). Includes epidemiology and ML-causal modes. The first choice for referee-level Stata replication packs.",
  49. "author": { "name": "Bryce Wang", "email": "brycew6m@stanford.edu" },
  50. "license": "CC-BY-SA-4.0",
  51. "homepage": "https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills",
  52. "repository": "https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills",
  53. "category": "academic",
  54. "keywords": [
  55. "econometrics", "causal-inference", "stata", "reghdfe", "difference-in-differences",
  56. "instrumental-variables", "regression-discontinuity", "synthetic-control",
  57. "empirical-research", "replication"
  58. ]
  59. },
  60. {
  61. "name": "empirical-analysis-r",
  62. "source": "./plugins/empirical-analysis-r",
  63. "version": "1.0.0",
  64. "description": "Explicit 8-step empirical-analysis pipeline in the modern tidyverse + econometrics R stack (dplyr + haven + fixest + did + synthdid + rdrobust + MatchIt + WeightIt + grf + DoubleML + mediation + marginaleffects + modelsummary). All eight steps fit in a single .qmd; `quarto render` produces a unified PDF/HTML/Word reproducibility report. Includes epidemiology and ML-causal modes.",
  65. "author": { "name": "Bryce Wang", "email": "brycew6m@stanford.edu" },
  66. "license": "CC-BY-SA-4.0",
  67. "homepage": "https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills",
  68. "repository": "https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills",
  69. "category": "academic",
  70. "keywords": [
  71. "econometrics", "causal-inference", "r", "tidyverse", "fixest",
  72. "difference-in-differences", "regression-discontinuity", "quarto",
  73. "empirical-research", "reproducible-research"
  74. ]
  75. }
  76. ]
  77. }
README.md · README · 404 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)
  2. [![GitHub stars](https://img.shields.io/github/stars/brycewang-stanford/Auto-Empirical-Research-Skills?style=social)](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills)
  3. [![License: CC BY-SA 4.0](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg)](https://creativecommons.org/licenses/by-sa/4.0/)
  4. [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)
  5. [![Validate catalog](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/actions/workflows/validate-catalog.yml/badge.svg)](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/actions/workflows/validate-catalog.yml)
  6. [![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/brycewang-stanford/Auto-Empirical-Research-Skills/badge)](https://scorecard.dev/viewer/?uri=github.com/brycewang-stanford/Auto-Empirical-Research-Skills)
  7. [![Security audit: baseline 52/52 CLEAN](https://img.shields.io/badge/security%20audit-baseline%2052%2F52%20CLEAN-brightgreen)](SECURITY-SCAN-REPORT.md)
  8. [![Rigor coverage](https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fbrycewang-stanford%2FAuto-Empirical-Research-Skills%2Fmain%2Fdocs%2Fbadges%2Frigor-coverage.json)](docs/RIGOR_COVERAGE.md)
  9. [![Powered by StatsPAI](https://img.shields.io/badge/powered%20by-StatsPAI-orange)](https://github.com/brycewang-stanford/StatsPAI)
  10. # Auto-Empirical Research Skills (AERS)
  11. > ## 🔥 力荐 · [StatsPAI](https://github.com/brycewang-stanford/StatsPAI) — Stata 与 R 的 Python 平替,为 AI Agent 而生
  12. >
  13. > - 🔁 **Stata / R 平替**:`reghdfe`、`csdid`、`rdrobust`、`synth`、`fixest`、`did` 等常用命令在 Python 中一站式替代,一个 `import statspai as sp` 搞定 DID / RD / IV / SCM / DML / matching
  14. > - 🤖 **面向 Agent 设计**:结构化结果对象 + 内置 MCP Server,Claude / Codex 等智能体可直接调用、诊断、审计与出表
  15. > - ⚡ **运行速度更快**:无需启动 Stata / R 会话,免许可证,纯 Python 流水线端到端提速
  16. > - 🏛️ **同一团队维护**:StatsPAI 与 AERS 由同一团队(Stanford REAP)开发和持续维护,两者深度集成、同步迭代
  17. >
  18. > **[github.com/brycewang-stanford/StatsPAI](https://github.com/brycewang-stanford/StatsPAI)** · `pip install statspai` · MIT 开源 · 欢迎 ⭐ Star
  19. <div align="center">
  20. **🌐 语言: [English](README-en.md) | 简体中文(默认) | [繁體中文](README-zh-TW.md) | [日本語](README-ja.md) | [한국어](README-ko.md)**
  21. <br/>
  22. <table>
  23. <tr>
  24. <td align="center">
  25. <a href="https://github.com/brycewang-stanford/StatsPAI"><img src="images/statspai-logo.png" alt="StatsPAI" width="360" /></a>
  26. </td>
  27. <td width="72"></td>
  28. <td align="center">
  29. <img src="images/stanford-reap-logo.png" alt="Stanford REAP - Center on China's Economy & Institutions" width="440" />
  30. </td>
  31. </tr>
  32. </table>
  33. <br/>
  34. <strong>Stanford REAP × StatsPAI</strong> · 实证研究 AI 工具的学术工业级产品<br/>
  35. <sub>由斯坦福实证研究方法论团队打造,覆盖从数据清洗到顶刊投稿的完整工作流</sub>
  36. <br/>
  37. <br/>
  38. <img src="images/aers-readme-cover-cn.png" alt="实证研究智能体技能大全封面图" width="100%" />
  39. <br/>
  40. </div>
  41. > ### 🚀 New here? Open the **[Skill Search →](docs/search.html)** to filter all 1,151 skills by method, stage, language, and license. The 5-minute tour (`make quickstart`) prints the same picture in your terminal.
  42. >
  43. > ### 🇨🇳 **中文用户从本文件开始**(流水线速览 + 78 行总表),每个合集的完整描述见 [`docs/CONTENT_ZH.md`](docs/CONTENT_ZH.md)。📖 **English readers:** see [`README-en.md`](README-en.md).
  44. ---
  45. ### 信任面 · Trust surface (rigor stats)
  46. | Rigor lane | Count | Where |
  47. |---|---|---|
  48. | Numeric **benchmark tasks** — gold values recomputed from real data each run | **19** | [`benchmark/`](benchmark/) |
  49. | Behavioral **eval scenarios / rubric items** | **42 / 217** | [`eval-harness/`](eval-harness/) |
  50. | 其中**已证明能区分对错**的场景(pass/fail 双 fixture 自检) | **9**(全部 6 个 critical 场景在内) | [`eval-harness/fixtures/`](eval-harness/fixtures/) |
  51. > Full trust overview: [`docs/TRUST.md`](docs/TRUST.md) · [`docs/RIGOR_COVERAGE.md`](docs/RIGOR_COVERAGE.md)
  52. >
  53. > 🏁 **带上你自己的 agent 来考同一份卷子**:`pip install -e .` 后用 [`aers-score`](aers_score/README.md) 给自己打分,成绩发布在 [`docs/EXTERNAL_SCOREBOARD.md`](docs/EXTERNAL_SCOREBOARD.md)(规则见 [`docs/SCOREBOARD_RULES.md`](docs/SCOREBOARD_RULES.md))。榜上的数字是**我们用同一套评分器重算**出来的,不是提交者自报的。
  54. ---
  55. ## ⚡ 安装与使用(30 秒上手)
  56. ### 最省事的一招:把 URL 丢给 Agent
  57. **把项目 URL 地址 `https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills` 丢给 Claude Code / Codex,并指定是目录 / 项目 / 全局安装** —— 剩下的让它自己做。例如:
  58. ```text
  59. 帮我安装 https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills
  60. 装到「全局」(~/.claude/skills/),我想在所有项目里都能用
  61. ```
  62. 把最后一行换成你要的作用域即可:
  63. | 作用域 | 说给 Agent 的话 | 落到哪里 |
  64. |:--|:--|:--|
  65. | **目录**(当前会话临时用) | "只在当前目录用,不要全局安装" | 当前工作目录下的 `.claude/skills/` |
  66. | **项目**(团队共享,可提交进 git) | "装到本项目" | 项目根目录 `.claude/skills/` |
  67. | **全局**(所有项目可用) | "装到全局" | `~/.claude/skills/`(Codex 为 `~/.agents/skills/`,老版本 `~/.codex/skills/`) |
  68. **已经 clone / 下载到本地?** 把 `Auto-Empirical-Research-Skills` 文件夹的本地路径复制给 Agent,让它读 [`INSTALL.md`](INSTALL.md) 自己装:
  69. ```text
  70. 帮我安装这个本地 skills 包:/Users/你的用户名/Downloads/Auto-Empirical-Research-Skills
  71. 我用的是 Codex(或 Claude Code / WorkBuddy),装到全局,所有项目都能用。
  72. 请先读这个文件夹里的 INSTALL.md,按我这个软件的方式装好;装完告诉我装到了哪个目录、怎么确认已经生效。
  73. ```
  74. > 复制路径:macOS 在访达选中文件夹按 `Option + Command + C`;Windows 按住 `Shift` 右键 →「复制为路径」;或者直接把文件夹拖进 Agent 输入框。详细步骤见 [`INSTALL.md`](INSTALL.md) 方法 0。
  75. ### 手动安装(两种,任选其一)
  76. **A. 插件市场(Claude Code v2.1+,推荐,可升级)**
  77. ```bash
  78. claude plugin marketplace add brycewang-stanford/Auto-Empirical-Research-Skills
  79. claude plugin install aer-skills@auto-empirical-research-skills # 顶刊投稿全流程(9 skills)
  80. claude plugin install empirical-analysis-python@auto-empirical-research-skills # Python 计量流水线
  81. claude plugin install empirical-analysis-stata@auto-empirical-research-skills # Stata 计量流水线
  82. claude plugin install empirical-analysis-r@auto-empirical-research-skills # R + Quarto 流水线
  83. ```
  84. **B. 只要某一个 skill —— 直接拷文件夹**
  85. ```bash
  86. git clone --recurse-submodules https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills.git
  87. cd Auto-Empirical-Research-Skills
  88. cp -R skills/00.1-Full-empirical-analysis-skill_Python .claude/skills/ # 项目级
  89. cp -R skills/00.1-Full-empirical-analysis-skill_Python ~/.claude/skills/ # 全局
  90. ```
  91. 拷进去的文件夹必须自带 `SKILL.md`(部分合集的 `SKILL.md` 在下一层,拷那一层)。
  92. ### 怎么用
  93. 新开一个会话,**直接用自然语言说要做什么**,Agent 会按 `description` 自动挑 skill;说不动就点名方法或 skill:
  94. ```text
  95. 用面板数据跑一个 Callaway–Sant'Anna 事件研究,并出 HonestDiD 稳健性和期刊级表格
  96. ```
  97. > 完整安装说明(Codex 安装见方法 2C、Codex / CodeBuddy 整库导入、`--plugin-dir` 单次加载、常见故障排查)见 [`INSTALL.md`](INSTALL.md)。
  98. ---
  99. ## 中文文档结构
  100. 中文内容分两级维护,各司其职:
  101. - **本文件(README.md,GitHub 默认入口)**:banner、badges、信任面、9 阶段流水线速览、78 行合集总表。
  102. - **[`docs/CONTENT_ZH.md`](docs/CONTENT_ZH.md)(扩展正文)**:每个合集的完整描述(`#skill-NN` 锚点)、按用途分组、精确数字、2 分钟验证、三层信任、旗舰流水线详解、贡献与引用。总表行内的 `→` 直接跳到对应锚点。
  103. - **其他语言**:[`README-en.md`](README-en.md) · [`README-zh-TW.md`](README-zh-TW.md) · [`README-ja.md`](README-ja.md) · [`README-ko.md`](README-ko.md)
  104. > [!NOTE]
  105. > **维护规则:** 改合集总表 → 本文件与 CONTENT_ZH.md 的锚点表**两处同步**;改合集详情 / 分组 / 数字 → 只改 [`docs/CONTENT_ZH.md`](docs/CONTENT_ZH.md)。统计数字(合集数 / skill 数)以 [`catalog/skills.json`](catalog/skills.json) 为准,由 `make validate` 的 readme-stats 检查器守护。
  106. >
  107. > **贡献者(Contributors):** 提交前请在本地跑通完整门禁 `make check`(catalog 校验 + 链接 + 单元测试 + eval-harness + benchmark)。详见 [`CONTRIBUTING.md`](CONTRIBUTING.md)。
  108. >
  109. > **旧版归档:** [`README-zh-CN.md`](README-zh-CN.md) 已弃用,仅作向后兼容的重定向占位。
  110. ---
  111. ## 🚀 从一个 idea 到一篇论文:社科实证研究 · 端到端流水线(全自动、可介入)
  112. > **AERS 不只是 78 个散装 skill —— 它能陪你走完一篇论文。** 从模糊 idea → 选题精炼 → 文献综述 → 数据获取 → 识别策略 → 估计建模 → 稳健性审计 → 出版级表格 / 图形 → 写作与同行评审 → 降 AIGC → 投稿。**端到端、全自动、每一步都可被人介入**(中间任何一步你都可以接过去手工改方法、补变量、加稳健性,再让流水线自动接上跑)。
  113. ### 9 阶段流水线 · 每一步都覆盖到具体 skill
  114. | # | 阶段 | 关键 skills(点合集名进目录,`→` 进完整说明) |
  115. |:--:|:--|:--|
  116. | 1️⃣ | **选题精炼** — Agent 把模糊想法收紧成"可证伪 + 可执行"的研究问题 | · [25 Diverga](skills/25-HosungYou-Diverga/) · [33 claude-scholar](skills/33-Galaxy-Dawn-claude-scholar/) · [05 research-superpower](skills/05-kthorn-research-superpower/) · [11 compound-science](skills/11-James-Traina-compound-science/) |
  117. | 2️⃣ | **文献综述** — 检索 · 筛选 · PRISMA 流程 · 批判性阅读 · 主题分析 | · [36 literature-review-skill](skills/36-taoyunudt-literature-review-skill/) · [24 academic-research-skills](skills/24-Imbad0202-academic-research-skills/) · [59 openalex-skill](skills/59-shiquda-openalex-skill/) · [68 research-productivity-skills](skills/68-research-productivity-skills/) · [53 thematic-analysis](skills/53-keemanxp-thematic-analysis-skill/) |
  118. | 3️⃣ | **数据获取** — 公开数据库 · API · 网页抓取 · 数据清洗 | · [33 claude-scholar](skills/33-Galaxy-Dawn-claude-scholar/) · [68 research-productivity-skills](skills/68-research-productivity-skills/) · [32 stata-skill](skills/32-dylantmoore-stata-skill/) · [57 edgartools](skills/57-dgunning-edgartools/) |
  119. | 4️⃣ | **识别策略** — DiD / RD / IV / SCM / DML / matching 全覆盖 | · ⭐ [00 StatsPAI](skills/00-Full-empirical-analysis-skill_StatsPAI/) 🔥 · [10 causal-inference-mixtape](skills/10-Jill0099-causal-inference-mixtape/) · [13 MixtapeTools](skills/13-scunning1975-MixtapeTools/) · [51 CausalPy](skills/51-pymc-labs-CausalPy/) · [63 scientific-agent-skills](skills/63-tondevrel-scientific-agent-skills/) |
  120. | 5️⃣ | **估计建模** — Python / Stata / R 三栈,900+ 估计器 | · ⭐ [00.1 Full Empirical · Python](skills/00.1-Full-empirical-analysis-skill_Python/) · ⭐ [00.2 Full Empirical · Stata](skills/00.2-Full-empirical-analysis-skill_Stata/) · ⭐ [00.3 Full Empirical · R](skills/00.3-Full-empirical-analysis-skill_R/) · [40 pyfixest](skills/40-py-econometrics-pyfixest/) · [39 marginaleffects](skills/39-vincentarelbundock-marginaleffects/) · [09 awesome-econ-ai](skills/09-meleantonio-awesome-econ-ai-stuff/) · ⭐ [74 SPSS + AMOS 问卷结构方程](skills/74-brycewang-spss-amos-sem/) |
  121. | 6️⃣ | **稳健性审计** — 复现包检查 · Honest-DiD · R&R 模拟 | · [41 sewage-econometrics-check](skills/41-sticerd-eee-sewage-econometrics-check/) · ⭐ [50 AER-skills](skills/50-brycewang-aer-skills/) · ⭐ [73 p-hacking-skills](skills/73-brycewang-p-hacking-skills/) · [21 AI-research-feedback](skills/21-claesbackman-AI-research-feedback/) |
  122. | 7️⃣ | **表格 & 图形** — 期刊出版级排版 · LaTeX 嵌入 | · ⭐ [00 StatsPAI](skills/00-Full-empirical-analysis-skill_StatsPAI/) · [07 AI-Research-SKILLs](skills/07-Orchestra-Research-AI-Research-SKILLs/) · [33 claude-scholar](skills/33-Galaxy-Dawn-claude-scholar/) · [08 latex-document-skill](skills/08-ndpvt-web-latex-document-skill/) |
  123. | 8️⃣ | **写作 & 同行评审** — LaTeX / Quarto · 仿审稿人 · 校对 | · [06 stats-paper-writing](skills/06-fuhaoda-stats-paper-writing/) · [04 scientific-writer](skills/04-K-Dense-AI-claude-scientific-writer/) · [22 christopherkenny-skills](skills/22-christopherkenny-skills/) · [38 academic-proofreader](skills/38-peternka-academic-proofreader/) · [56 econ-writing-skill](skills/56-hanlulong-econ-writing-skill/) · [16 clo-author](skills/16-hsantanna88-clo-author/) |
  124. | 9️⃣ | **降 AIGC & 去水印 & 投稿** — 知网 / 万方 / Turnitin / 英 30 + 中 28 类 AI 痕迹模式 / 数字与引用保真核对 / 隐藏字符 · 形近字 · C2PA · docx 元数据清理 | · ⭐ [48 de-AIGC-skills](skills/48-de-AIGC-skills/) 🇨🇳🇬🇧 · [44 humanizer_academic](skills/44-matsuikentaro1-humanizer_academic/) · [45 deslop](skills/45-stephenturner-skill-deslop/) · [46 stop-slop](skills/46-hardikpandya-stop-slop/) · [47 avoid-ai-writing](skills/47-conorbronsdon-avoid-ai-writing/) · [49 humanize-chinese](skills/49-voidborne-d-humanize-chinese/) |
  125. ### 🎼 元编排:⭐ [69 Paper-WorkFlow](skills/69-Paper-WorkFlow/) —— 一键串起来
  126. [`Paper-WorkFlow`](skills/69-Paper-WorkFlow/) 是 AERS 的"指挥棒",它把上面 9 个阶段的 skill 串成 **一条按键即运行的端到端流水线**。
  127. 你在 IDE 入口给它一句自然语言:
  128. > *"开一个新论文项目:空气污染与中国劳动力市场,CS 设计 + 省级面板"*
  129. 它会自动按顺序调:
  130. 1. ⭐ [00 StatsPAI](skills/00-Full-empirical-analysis-skill_StatsPAI/) → `sp.csdid(...)` 给出 CS-DID 估计草案 + 写出估计方程与识别假设
  131. 2. [33 claude-scholar](skills/33-Galaxy-Dawn-claude-scholar/) → 抓变量定义 / 数据源候选 / 相关文献
  132. 3. ⭐ [00 StatsPAI](skills/00-Full-empirical-analysis-skill_StatsPAI/) → 真跑 `sp.feols(...)` + `sp.honest_did(...)`
  133. 4. [41 sewage-econometrics-check](skills/41-sticerd-eee-sewage-econometrics-check/) → 10 项复现包审计 + 稳健性体检
  134. 5. ⭐ [00 StatsPAI](skills/00-Full-empirical-analysis-skill_StatsPAI/) + [07 AI-Research-SKILLs](skills/07-Orchestra-Research-AI-Research-SKILLs/) → 出 Table 1–5 + 期刊级图
  135. 6. [38 academic-proofreader](skills/38-peternka-academic-proofreader/) → 通读 + §comment 标"审稿人会挑刺的位置"
  136. 7. [56 econ-writing-skill](skills/56-hanlulong-econ-writing-skill/) 起草初稿 + ⭐ [48 de-AIGC-skills](skills/48-de-AIGC-skills/) 🇨🇳🇬🇧 + [45 deslop](skills/45-stephenturner-skill-deslop/) 过知网 / Turnitin
  137. **任何阶段你都可以手动介入** —— 上一阶段的产物全部落盘(产物-幂等 pipeline),你接过去改方法、补控制、加稳健性,再让流水线自动接下去跑。这就是"全自动 + 可介入"。
  138. ### 🏆 7 个 Stanford REAP × StatsPAI 自研 skill —— 是整个流水线的主干
  139. | ⭐ Skill | 在流水线里的角色 |
  140. |:--|:--|
  141. | [00 StatsPAI](skills/00-Full-empirical-analysis-skill_StatsPAI/) 🔥 | **因果引擎**:900+ 函数,`sp.causal(...)` 一行跑闭环(DID / RD / IV / SCM / DML / matching) |
  142. | [00.1 Full Empirical · Python](skills/00.1-Full-empirical-analysis-skill_Python/) 📘 | 显式 Python 栈(pandas / statsmodels / linearmodels / pyfixest) |
  143. | [00.2 Full Empirical · Stata](skills/00.2-Full-empirical-analysis-skill_Stata/) 📊 | 显式 Stata 栈(reghdfe / ivreg2 / csdid / sdid / rdrobust) |
  144. | [00.3 Full Empirical · R](skills/00.3-Full-empirical-analysis-skill_R/) 📗 | 显式 R 栈(tidyverse / fixest / did / HonestDiD)+ Quarto 渲染 |
  145. | [48 de-AIGC-skills](skills/48-de-AIGC-skills/) 🇨🇳🇬🇧 | 中英双语学术降 AIGC + 去水印层(Turnitin AI / GPTZero / 知网 / 万方 · 隐藏字符 / C2PA / docx 元数据);2026-10 对照 GitHub 星数前十的去 AI 味 skill 升级:英 30 + 中 28 类模式、只诊断 / 编辑 / 作者重述三种模式、`aigc_audit.py` 保真闸门 |
  146. | [50 AER-skills](skills/50-brycewang-aer-skills/) 📕 | Top-5 经济学投稿套件:识别 → 稳健性 → R&R |
  147. | [69 Paper-WorkFlow](skills/69-Paper-WorkFlow/) 🧭 | 元编排器,把上面 9 个阶段串成一键流水线 |
  148. **为什么挑这 7 个?因为它们的行为都被基准钉死了** —— 不是营销口径,是对着已知答案反复跑过验证过的([17 项数值 benchmark + 37 项行为评测 ↗](docs/CONTENT_ZH.md#你究竟得到什么精确数字))。
  149. ### 看到这里 —— 完整 78 行合集目录
  150. [↴ 直跳到下方 78 行总表(每个合集带 `#skill-NN` 锚点)](#-78-个核心-skills-合集一览00--74编号连续无空缺)。如果你更关心"这些 skill **怎么用**"而不是"有哪些 skill",看 [📘 中文唯一权威正文](docs/CONTENT_ZH.md) 里的「按用途分组」与「旗舰流水线」两节。
  151. ---
  152. ## 🧰 78 个核心 Skills 合集一览(`00 → 74`,编号连续无空缺)
  153. > **打开仓库 → 看见整座库。** 全部 **78 个合集 · 1,151 个 skill**,每一个都已 vendor 进本仓库,由 [`catalog/skills.json`](catalog/skills.json) 跟踪。**⭐ = Stanford REAP × StatsPAI 团队自研的 skill**;其余为精选、经安全审计的社区作品。
  154. >
  155. > **主题图例 —** 🚀 全流程与编排器 · 🎯 因果推断与计量经济学 · 📚 文献与研究设计 · ✍️ 写作 / 编辑 / 去 AIGC · 📑 引用 / 复现 / 同行评审 · 🛠️ 数据 / 工具 / 基础设施
  156. >
  157. > **点击【→】** 跳转到 [`docs/CONTENT_ZH.md`](docs/CONTENT_ZH.md) 中该合集的完整描述;**点击合集名** 直接打开其目录。
  158. >
  159. > **🙏 尊重原作者 —** **「来源」列直接链回上游原始仓库**(`owner/repo`)。本仓库里的社区合集都是**上游快照**:请去原仓库点 star、提 issue、看 LICENSE。完整的许可证与来源置信度审计见 [`docs/LICENSE_AUDIT.md`](docs/LICENSE_AUDIT.md),机器可读版本在 [`catalog/provenance.json`](catalog/provenance.json)。
  160. | # | 合集 | 一句话 | 详情 | 来源 |
  161. |--:|:--|:--|:--|:--|
  162. | ⭐ [00](skills/00-Full-empirical-analysis-skill_StatsPAI/) | **StatsPAI** 🔥 | 因果引擎 · Agent-native Python DSL:`sp.causal(...)` 一行跑闭环(DID/RD/IV/SCM/DML,900+ 函数) | [→](docs/CONTENT_ZH.md#skill-00) | [brycewang-stanford/StatsPAI](https://github.com/brycewang-stanford/StatsPAI) |
  163. | ⭐ [00.1](skills/00.1-Full-empirical-analysis-skill_Python/) | **Full Empirical · Python** 📘 | 显式栈:`pandas` · `statsmodels` · `linearmodels` · `pyfixest` | [→](docs/CONTENT_ZH.md#skill-00-1) | [⭐ 本仓库](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills) |
  164. | ⭐ [00.2](skills/00.2-Full-empirical-analysis-skill_Stata/) | **Full Empirical · Stata** 📊 | `reghdfe` · `ivreg2` · `csdid` · `sdid` · `rdrobust` 复现包 | [→](docs/CONTENT_ZH.md#skill-00-2) | [⭐ 本仓库](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills) |
  165. | ⭐ [00.3](skills/00.3-Full-empirical-analysis-skill_R/) | **Full Empirical · R** 📗 | tidyverse · `fixest` · `did` · `HonestDiD` + Quarto 渲染 | [→](docs/CONTENT_ZH.md#skill-00-3) | [⭐ 本仓库](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills) |
  166. | [01](skills/01-lishix520-academic-paper-skills/) | academic-paper-skills | 大纲 → 手稿写作 + 7 维审稿人模拟 | [→](docs/CONTENT_ZH.md#skill-01) | [lishix520/academic-paper-skills](https://github.com/lishix520/academic-paper-skills) |
  167. | [02](skills/02-luwill-research-skills/) | research-skills | 医学影像综述、提案、论文转幻灯片 | [→](docs/CONTENT_ZH.md#skill-02) | [luwill/research-skills](https://github.com/luwill/research-skills) |
  168. | [03](skills/03-K-Dense-AI-claude-scientific-skills/) | scientific-skills | 假设生成 + 28 个科学数据库 | [→](docs/CONTENT_ZH.md#skill-03) | [K-Dense-AI/claude-scientific-skills](https://github.com/K-Dense-AI/claude-scientific-skills) |
  169. | [04](skills/04-K-Dense-AI-claude-scientific-writer/) | scientific-writer | 引用管理 + 科学写作 | [→](docs/CONTENT_ZH.md#skill-04) | [K-Dense-AI/claude-scientific-writer](https://github.com/K-Dense-AI/claude-scientific-writer) |
  170. | [05](skills/05-kthorn-research-superpower/) | research-superpower | 系统化检索、筛选与引文溯源 | [→](docs/CONTENT_ZH.md#skill-05) | [kthorn/research-superpower](https://github.com/kthorn/research-superpower) |
  171. | [06](skills/06-fuhaoda-stats-paper-writing/) | stats-paper-writing | 端到端 LaTeX 统计论文写作 | [→](docs/CONTENT_ZH.md#skill-06) | [fuhaoda/stats-paper-writing-agent-skills](https://github.com/fuhaoda/stats-paper-writing-agent-skills) |
  172. | [07](skills/07-Orchestra-Research-AI-Research-SKILLs/) | AI-Research-SKILLs | 发表级 ML 图表、LaTeX、引文核验 | [→](docs/CONTENT_ZH.md#skill-07) | [Orchestra-Research/AI-Research-SKILLs](https://github.com/Orchestra-Research/AI-Research-SKILLs) |
  173. | [08](skills/08-ndpvt-web-latex-document-skill/) | latex-document-skill | 创建 / 编译任意 LaTeX 文档为 PDF | [→](docs/CONTENT_ZH.md#skill-08) | [ndpvt-web/latex-document-skill](https://github.com/ndpvt-web/latex-document-skill) |
  174. | [09](skills/09-meleantonio-awesome-econ-ai-stuff/) | awesome-econ-ai | Python 面板数据分析(`linearmodels`) | [→](docs/CONTENT_ZH.md#skill-09) | [meleantonio/awesome-econ-ai-stuff](https://github.com/meleantonio/awesome-econ-ai-stuff) |
  175. | [10](skills/10-Jill0099-causal-inference-mixtape/) | causal-inference-mixtape | DID / IV / RDD / SCM 模板(Cunningham) | [→](docs/CONTENT_ZH.md#skill-10) | [Jill0099/causal-inference-mixtape](https://github.com/Jill0099/causal-inference-mixtape) |
  176. | [11](skills/11-James-Traina-compound-science/) | compound-science | 面向定量社会科学的贝叶斯估计 | [→](docs/CONTENT_ZH.md#skill-11) | [James-Traina/compound-science](https://github.com/James-Traina/compound-science) |
  177. | [12](skills/12-pedrohcgs-claude-code-my-workflow/) | claude-code-my-workflow | 提交 → PR → 合并的研究工作流(Emory) | [→](docs/CONTENT_ZH.md#skill-12) | [pedrohcgs/claude-code-my-workflow](https://github.com/pedrohcgs/claude-code-my-workflow) |
  178. | [13](skills/13-scunning1975-MixtapeTools/) | MixtapeTools | Cunningham 的因果推断工具集与讲义 | [→](docs/CONTENT_ZH.md#skill-13) | [scunning1975/MixtapeTools](https://github.com/scunning1975/MixtapeTools) |
  179. | [14](skills/14-luischanci-claude-code-research-starter/) | research-starter | R 中的 IV / DiD / RDD,含完整诊断 | [→](docs/CONTENT_ZH.md#skill-14) | [luischanci/claude-code-research-starter](https://github.com/luischanci/claude-code-research-starter) |
  180. | [15](skills/15-Felpix-Studios-social-science-research/) | social-science-research | R 或 Python 端到端数据分析 | [→](docs/CONTENT_ZH.md#skill-15) | [Felpix-Studios/social-science-research](https://github.com/Felpix-Studios/social-science-research) |
  181. | [16](skills/16-hsantanna88-clo-author/) | clo-author | 多代理数据分析(R / Stata / Python) | [→](docs/CONTENT_ZH.md#skill-16) | [hsantanna88/clo-author](https://github.com/hsantanna88/clo-author) |
  182. | [17](skills/17-DAAF-Contribution-Community-daaf/) | DAAF | 安全意识代理框架(32 条 deny rule) | [→](docs/CONTENT_ZH.md#skill-17) | [DAAF-Contribution-Community/daaf](https://github.com/DAAF-Contribution-Community/daaf) |
  183. | [18](skills/18-jusi-aalto-stata-accounting-research/) | stata-accounting | 来自 126 篇 *JAR* 论文的实测 Stata 范式 | [→](docs/CONTENT_ZH.md#skill-18) | [jusi-aalto/stata-accounting-research](https://github.com/jusi-aalto/stata-accounting-research) |
  184. | [19](skills/19-CuellarC05-vera-economic-intelligence/) | vera-economic-intelligence | 经济情报 / 政策研究情报工作流 | [→](docs/CONTENT_ZH.md#skill-19) | [CuellarC05/vera-economic-intelligence](https://github.com/CuellarC05/vera-economic-intelligence) |
  185. | [20](skills/20-wenddymacro-python-econ-skill/) | python-econ-skill | DSGE / HANK 与定量经济计算 | [→](docs/CONTENT_ZH.md#skill-20) | [wenddymacro/python-econ-skill](https://github.com/wenddymacro/python-econ-skill) |
  186. | [21](skills/21-claesbackman-AI-research-feedback/) | AI-research-feedback | 用 AI 同行评审生成结构化反馈 | [→](docs/CONTENT_ZH.md#skill-21) | [claesbackman/AI-research-feedback](https://github.com/claesbackman/AI-research-feedback) |
  187. | [22](skills/22-christopherkenny-skills/) | christopherkenny-skills | 面向 Quarto(`.qmd`)的 APSA 风格检查器 | [→](docs/CONTENT_ZH.md#skill-22) | [christopherkenny/skills](https://github.com/christopherkenny/skills) |
  188. | [23](skills/23-Learning-Bayesian-Statistics-baygent-skills/) | baygent | 带护栏的 PyMC / Arviz 贝叶斯工作流 | [→](docs/CONTENT_ZH.md#skill-23) | [Learning-Bayesian-Statistics/baygent-skills](https://github.com/Learning-Bayesian-Statistics/baygent-skills) |
  189. | [24](skills/24-Imbad0202-academic-research-skills/) | academic-research-skills | 5 审稿人多视角论文评审 | [→](docs/CONTENT_ZH.md#skill-24) | [Imbad0202/academic-research-skills](https://github.com/Imbad0202/academic-research-skills) |
  190. | [25](skills/25-HosungYou-Diverga/) | Diverga | 研究问题精炼器(抗模式坍缩) | [→](docs/CONTENT_ZH.md#skill-25) | [HosungYou/Diverga](https://github.com/HosungYou/Diverga) |
  191. | [26](skills/26-Data-Wise-scholar/) | scholar | 统计算法设计与文档 | [→](docs/CONTENT_ZH.md#skill-26) | [Data-Wise/claude-plugins](https://github.com/Data-Wise/claude-plugins) |
  192. | [27](skills/27-dariia-m-my_claude_skills/) | my_claude_skills | 经济学摘要写作指南 | [→](docs/CONTENT_ZH.md#skill-27) | [dariia-m/my_claude_skills](https://github.com/dariia-m/my_claude_skills) |
  193. | [28](skills/28-maxwell2732-paper-replicate-agent-demo/) | paper-replicate-agent | 论文复现代理演示 | [→](docs/CONTENT_ZH.md#skill-28) | [maxwell2732/paper-replicate-agent-demo](https://github.com/maxwell2732/paper-replicate-agent-demo) |
  194. | [29](skills/29-quarcs-lab-project20XXy/) | project20XXy | 可复现手稿 + notebook 项目 | [→](docs/CONTENT_ZH.md#skill-29) | [quarcs-lab/project20XXy](https://github.com/quarcs-lab/project20XXy) |
  195. | [30](skills/30-zirui-song-claude-skills/) | zirui-song-claude-skills | Zirui Song 的研究辅助 Claude 技能集 | [→](docs/CONTENT_ZH.md#skill-30) | [zirui-song/claude-skills](https://github.com/zirui-song/claude-skills) |
  196. | [31](skills/31-thalysandratos-claude-code-skills/) | claude-code-skills | Python 面板数据分析 | [→](docs/CONTENT_ZH.md#skill-31) | [thalysandratos/claude-code-skills](https://github.com/thalysandratos/claude-code-skills) |
  197. | [32](skills/32-dylantmoore-stata-skill/) | stata-skill | 高性能 Stata C/C++ 插件 | [→](docs/CONTENT_ZH.md#skill-32) | [dylantmoore/stata-skill](https://github.com/dylantmoore/stata-skill) |
  198. | [33](skills/33-Galaxy-Dawn-claude-scholar/) | claude-scholar | 研究全生命周期:选题 → 综述 → 实验 → 审稿回复 | [→](docs/CONTENT_ZH.md#skill-33) | [Galaxy-Dawn/claude-scholar](https://github.com/Galaxy-Dawn/claude-scholar) |
  199. | [34](skills/34-andrehuang-research-companion/) | research-companion | 头脑风暴、评估并决策研究方向 | [→](docs/CONTENT_ZH.md#skill-34) | [andrehuang/research-companion](https://github.com/andrehuang/research-companion) |
  200. | [35](skills/35-bahayonghang-academic-writing-skills/) | academic-writing-skills | 面向投稿场所的工业 AI 文献研究 | [→](docs/CONTENT_ZH.md#skill-35) | [bahayonghang/academic-writing-skills](https://github.com/bahayonghang/academic-writing-skills) |
  201. | [36](skills/36-taoyunudt-literature-review-skill/) | literature-review-skill | 完整文献综述工作流(中文) | [→](docs/CONTENT_ZH.md#skill-36) | [taoyunudt/literature-review-skill](https://github.com/taoyunudt/literature-review-skill) |
  202. | [37](skills/37-IlanStrauss-ai-skills/) | IlanStrauss-ai-skills | Ilan Strauss 经济学研究 AI 工作流 | [→](docs/CONTENT_ZH.md#skill-37) | [IlanStrauss/ai-skills](https://github.com/IlanStrauss/ai-skills) |
  203. | [38](skills/38-peternka-academic-proofreader/) | academic-proofreader | 学术校对 | [→](docs/CONTENT_ZH.md#skill-38) | [peternka/academic_proofreader](https://github.com/peternka/academic_proofreader) |
  204. | [39](skills/39-vincentarelbundock-marginaleffects/) | marginaleffects | 预测、斜率与比较(R / Python) | [→](docs/CONTENT_ZH.md#skill-39) | [vincentarelbundock/marginaleffects](https://github.com/vincentarelbundock/marginaleffects) |
  205. | [40](skills/40-py-econometrics-pyfixest/) | pyfixest | Python 中的快速固定效应估计 | [→](docs/CONTENT_ZH.md#skill-40) | [py-econometrics/pyfixest](https://github.com/py-econometrics/pyfixest) |
  206. | [41](skills/41-sticerd-eee-sewage-econometrics-check/) | sewage-econometrics-check | 10 项复现包审计 | [→](docs/CONTENT_ZH.md#skill-41) | [sticerd-eee/sewage](https://github.com/sticerd-eee/sewage) |
  207. | [42](skills/42-wanshuiyin-ARIS/) | ARIS | 自主「research-in-sleep」代理,端到端 | [→](docs/CONTENT_ZH.md#skill-42) | [wanshuiyin/Auto-claude-code-research-in-sleep](https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep) |
  208. | [43](skills/43-wentorai-research-plugins/) | research-plugins | 478 个研究插件:数据可视化、领域、基础设施 | [→](docs/CONTENT_ZH.md#skill-43) | [wentorai/research-plugins](https://github.com/wentorai/research-plugins) |
  209. | [44](skills/44-matsuikentaro1-humanizer_academic/) | humanizer_academic | 为医学/学术手稿去 AI 味(34 类模式) | [→](docs/CONTENT_ZH.md#skill-44) | [matsuikentaro1/humanizer_academic](https://github.com/matsuikentaro1/humanizer_academic) |
  210. | [45](skills/45-stephenturner-skill-deslop/) | deslop | 去除 AI 写作痕迹(5 维评分) | [→](docs/CONTENT_ZH.md#skill-45) | [stephenturner/skill-deslop](https://github.com/stephenturner/skill-deslop) |
  211. | [46](skills/46-hardikpandya-stop-slop/) | stop-slop | 三层 AI 痕迹检测与改写 | [→](docs/CONTENT_ZH.md#skill-46) | [hardikpandya/stop-slop](https://github.com/hardikpandya/stop-slop) |
  212. | [47](skills/47-conorbronsdon-avoid-ai-writing/) | avoid-ai-writing | 审计 → 改写 → 二次审计 AI 味(留痕) | [→](docs/CONTENT_ZH.md#skill-47) | [conorbronsdon/avoid-ai-writing](https://github.com/conorbronsdon/avoid-ai-writing) |
  213. | ⭐ [48](skills/48-de-AIGC-skills/) | **de-AIGC-skills** 🇨🇳🇬🇧 | 中英双语学术降 AIGC + 去水印层(Turnitin AI / GPTZero / 知网 / 万方 · 隐藏字符 / C2PA / docx 元数据)· 2026-10 对标星数前十 humanizer 升级 | [→](docs/CONTENT_ZH.md#skill-48) | [⭐ 本仓库](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills) |
  214. | [49](skills/49-voidborne-d-humanize-chinese/) | humanize-chinese | 检测并人性化 AI 生成的中文文本 | [→](docs/CONTENT_ZH.md#skill-49) | [swaylq/humanize-chinese](https://github.com/swaylq/humanize-chinese) |
  215. | ⭐ [50](skills/50-brycewang-aer-skills/) | **AER-skills** 📕 | Top-5 经济学投稿套件:识别 → 稳健性 → R&R | [→](docs/CONTENT_ZH.md#skill-50) | [brycewang-stanford/AER-skills](https://github.com/brycewang-stanford/AER-skills) |
  216. | [51](skills/51-pymc-labs-CausalPy/) | CausalPy | 贝叶斯准实验(PyMC Labs) | [→](docs/CONTENT_ZH.md#skill-51) | [pymc-labs/CausalPy](https://github.com/pymc-labs/CausalPy) |
  217. | [52](skills/52-keemanxp-slr-prisma/) | slr-prisma | 系统文献综述,PRISMA 2020 | [→](docs/CONTENT_ZH.md#skill-52) | [keemanxp/slr-prisma](https://github.com/keemanxp/slr-prisma) |
  218. | [53](skills/53-keemanxp-thematic-analysis-skill/) | thematic-analysis | Braun & Clarke 六阶段定性主题分析 | [→](docs/CONTENT_ZH.md#skill-53) | [keemanxp/thematic-analysis-skill](https://github.com/keemanxp/thematic-analysis-skill) |
  219. | [54](skills/54-scdenney-open-science-skills/) | open-science-skills | 引用一致性、DOI 与论据支撑审计 | [→](docs/CONTENT_ZH.md#skill-54) | [scdenney/open-science-skills](https://github.com/scdenney/open-science-skills) |
  220. | [55](skills/55-ab604-claude-code-r-skills/) | r-skills | R 中用 `brms` 做贝叶斯推断 | [→](docs/CONTENT_ZH.md#skill-55) | [ab604/claude-code-r-skills](https://github.com/ab604/claude-code-r-skills) |
  221. | [56](skills/56-hanlulong-econ-writing-skill/) | econ-writing-skill | 综合 50+ 顶级指南的经济学写作 | [→](docs/CONTENT_ZH.md#skill-56) | [hanlulong/econ-writing-skill](https://github.com/hanlulong/econ-writing-skill) |
  222. | [57](skills/57-dgunning-edgartools/) | edgartools | 查询与分析 SEC 文件 | [→](docs/CONTENT_ZH.md#skill-57) | [dgunning/edgartools](https://github.com/dgunning/edgartools) |
  223. | [58](skills/58-charlescoverdale-econstack/) | econstack | 政策简报(UK GES / AU Treasury) | [→](docs/CONTENT_ZH.md#skill-58) | [charlescoverdale/econstack](https://github.com/charlescoverdale/econstack) |
  224. | [59](skills/59-shiquda-openalex-skill/) | openalex-skill | 通过 OpenAlex 查询 2.4 亿+ 学术作品 | [→](docs/CONTENT_ZH.md#skill-59) | [shiquda/openalex-skill](https://github.com/shiquda/openalex-skill) |
  225. | [60](skills/60-regisely-superpapers/) | superpapers | 综合性实证研究支持套件 | [→](docs/CONTENT_ZH.md#skill-60) | [regisely/superpapers](https://github.com/regisely/superpapers) |
  226. | [61](skills/61-phdemotions-research-methods/) | research-methods | 与预注册匹配的验证性检验 | [→](docs/CONTENT_ZH.md#skill-61) | [phdemotions/research-methods](https://github.com/phdemotions/research-methods) |
  227. | [62](skills/62-PHY041-claude-skill-citation-checker/) | citation-checker | 对照 CrossRef / S2 / OpenAlex 核验引用 | [→](docs/CONTENT_ZH.md#skill-62) | [PHY041/claude-skill-citation-checker](https://github.com/PHY041/claude-skill-citation-checker) |
  228. | [63](skills/63-tondevrel-scientific-agent-skills/) | scientific-agent-skills | DoWhy 识别–估计–反驳框架 | [→](docs/CONTENT_ZH.md#skill-63) | [tondevrel/scientific-agent-skills](https://github.com/tondevrel/scientific-agent-skills) |
  229. | [64](skills/64-tmonk-mcp-stata/) | mcp-stata | 20 个 Stata 因果推断与复现 skill | [→](docs/CONTENT_ZH.md#skill-64) | [tmonk/mcp-stata](https://github.com/tmonk/mcp-stata) |
  230. | [65](skills/65-game-theory-paper-writer/) | game-theory-paper-writer | 生成并压力测试博弈论论文 | [→](docs/CONTENT_ZH.md#skill-65) | [本仓库 PR #17](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/pull/17) |
  231. | [66](skills/66-zheng-siyao-empirical-research-skills/) | empirical-research-skills | 面向大型面板的 R 性能优化 | [→](docs/CONTENT_ZH.md#skill-66) | [SiyaoZheng/ai4ss-skills](https://github.com/SiyaoZheng/ai4ss-skills) |
  232. | [67](skills/67-econfin-workflow-toolkit/) | econfin-workflow-toolkit | 中国公司金融实证工作流,从提案到论文 | [→](docs/CONTENT_ZH.md#skill-67) | [本仓库 PR #22](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/pull/22) |
  233. | [68](skills/68-research-productivity-skills/) | research-productivity-skills | 论文检索、SSRN、DOI 查询、下载 | [→](docs/CONTENT_ZH.md#skill-68) | [本仓库 PR #21](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/pull/21) |
  234. | ⭐ [69](skills/69-Paper-WorkFlow/) | **Paper-WorkFlow** 🧭 | 元编排器,串起整个社会科学论文流水线 | [→](docs/CONTENT_ZH.md#skill-69) | [brycewang-stanford/Paper-WorkFlow](https://github.com/brycewang-stanford/Paper-WorkFlow) |
  235. | [70](skills/70-ssci-polish/) | ssci-polish ✍️ | SSCI / SCI 英文论文语言润色(语法、可读性、学术语气) | [→](docs/CONTENT_ZH.md#skill-70) | [⭐ 本仓库](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills) |
  236. | ⭐ [71](skills/71-brycewang-lit-review-agent-tools/) | **lit-review-agent-tools** 🔍 | 文献综述工具选型 + 一键安装运行(MinerU / PaperQA2 / ASReview / STORM / MCP 服务器) | [→](docs/CONTENT_ZH.md#skill-71) | [brycewang-stanford/lit-review-agent-tools](https://github.com/brycewang-stanford/lit-review-agent-tools) |
  237. | ⭐ [72](skills/72-kaggle-research/) | **Kaggle Research** 🧪 | 通过官方 CLI 安全检索 Kaggle 资源、限界下载公开数据并保留审计证据 | [→](docs/CONTENT_ZH.md#skill-72) | [⭐ 本仓库](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills) |
  238. | ⭐ [73](skills/73-brycewang-p-hacking-skills/) | **p-hacking-skills** 🔬 | p-hacking 体检室:在真实效应为零的数据上仪表化规格搜索,给出诚实 p 值、p 曲线检测与预注册免疫(教学与审计用) | [→](docs/CONTENT_ZH.md#skill-73) | [brycewang-stanford/p-hacking-skills](https://github.com/brycewang-stanford/p-hacking-skills) |
  239. | ⭐ [74](skills/74-brycewang-spss-amos-sem/) | **spss-amos-sem** 📋 | 问卷数据结构方程全流程(SPSS + AMOS):清洗、信度、EFA、CFA(CR / AVE / HTMT)、结构模型、bootstrap 中介、多群组、PROCESS;自动生成 SPSS 语法、AMOS 程序与中文三线表 | [→](docs/CONTENT_ZH.md#skill-74) | [⭐ 本仓库](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills) |
  240. > **想看更详细的描述(主题分类、字段、统计)?** 见 [`docs/CONTENT_ZH.md`](docs/CONTENT_ZH.md) 中标注 `#skill-NN` 锚点的同一张表 —— 它是每个合集的完整描述所在的扩展正文。
  241. ## 📈 项目历程
  242. 自 2026-04 首次发布以来的主干里程碑(完整提交记录见 [Commits](https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/commits/main) 与 [`CHANGELOG.md`](CHANGELOG.md)):
  243. ```mermaid
  244. ---
  245. config:
  246. gitGraph:
  247. rotateCommitLabel: false
  248. ---
  249. gitGraph TB:
  250. commit id: "2026-04 首次发布"
  251. branch community
  252. commit id: "2026-05 首个社区 PR"
  253. checkout main
  254. merge community
  255. commit id: "2026-05 更名 AERS"
  256. commit id: "2026-06 插件市场"
  257. commit id: "2026-06 全库路由器"
  258. commit id: "2026-07 首个 tag" tag: "v2026.07"
  259. branch kaggle
  260. commit id: "2026-07 Kaggle 集成"
  261. checkout main
  262. merge kaggle
  263. commit id: "2026-08 de-AIGC 双语"
  264. commit id: "2026-08 来源链接全覆盖"
  265. branch evidence
  266. commit id: "2026-08 aers-score CLI"
  267. commit id: "2026-08 外部成绩单"
  268. checkout main
  269. merge evidence
  270. commit id: "2026-08 结构估计 = 方法族 18"
  271. commit id: "2026-08 NSW 基准从引用变推导"
  272. commit id: "2026-09 数据 → Word 全稿路线"
  273. commit id: "2026-09 de-AIGC 去水印层"
  274. commit id: "2026-09 分层检索 + 路由评测"
  275. commit id: "2026-09 StatsPAI 置顶"
  276. branch phacking
  277. commit id: "2026-09 p-hacking 体检室(合集 73)"
  278. checkout main
  279. merge phacking
  280. commit id: "2026-10 de-AIGC 对标前十 humanizer"
  281. ```
  282. <div align="center">
  283. <a href="https://github.com/brycewang-stanford/Auto-Empirical-Research-Skills/stargazers">
  284. <picture>
  285. <source media="(prefers-color-scheme: dark)" srcset="images/star-history-dark.svg" />
  286. <source media="(prefers-color-scheme: light)" srcset="images/star-history.svg" />
  287. <img alt="Star History Chart" src="images/star-history.svg" width="600" />
  288. </picture>
  289. </a>
  290. <sub>Star 增长曲线(非提交数)· 由 <a href="scripts/build-star-history.py"><code>scripts/build-star-history.py</code></a> 从 GitHub API 生成并提交入库</sub>
  291. </div>
  292. 如果 AERS 对你的工作有帮助,请**引用它**([CITATION.cff](CITATION.cff))并**点个 Star**,让更多研究者看到。
  293. ---
  294. <div align="center">
  295. **AI 是放大器,不是替代品。它替你做最耗时的"搬砖",你保留最核心的"判断"。**
  296. <br/>
  297. <table>
  298. <tr>
  299. <td align="center">
  300. <a href="https://github.com/brycewang-stanford/StatsPAI"><img src="images/statspai-logo.png" alt="StatsPAI" width="260" /></a>
  301. </td>
  302. <td width="40"></td>
  303. <td align="center">
  304. <img src="images/stanford-reap-logo.png" alt="Stanford REAP" width="320" />
  305. </td>
  306. </tr>
  307. </table>
  308. <sub><strong>Stanford REAP × StatsPAI</strong> · 实证研究 AI 工具的学术工业级产品</sub>
  309. <br/>
  310. <table>
  311. <tr>
  312. <td align="center">
  313. <a href="https://copaper.ai"><img src="images/copaper-qrcode.png" alt="扫码访问 copaper.ai" width="180" /></a><br/>
  314. <strong>扫码访问 <a href="https://copaper.ai">copaper.ai</a></strong>
  315. </td>
  316. <td align="center">
  317. <img src="images/copaper-wechat.jpg" alt="CoPaper.AI 公众号" width="180" /><br/>
  318. <strong>关注公众号「CoPaper.AI」</strong>
  319. </td>
  320. </tr>
  321. </table>
  322. 内置 20 个方法论 skill · 20 分钟完成实证论文 · 自研 <a href="https://github.com/brycewang-stanford/StatsPAI"><strong>StatsPAI</strong></a>(900+ 函数 / MIT 开源)
  323. </div>
scripts/scan-collections.py · Bundled script · 496 lines · 5 findings marked
  1. #!/usr/bin/env python3
  2. """Pattern-scan vendored skill collections for the risks the security report tracks.
  3. [`SECURITY-SCAN-REPORT.md`](../SECURITY-SCAN-REPORT.md) covers the original
  4. 52-collection baseline plus a hand-run incremental scan of collections 49-70
  5. (2026-07-15). Both were one-off efforts written up in prose, which means the
  6. scan's *coverage* drifted the moment the next collection landed: 71 and 72 were
  7. vendored afterwards and nothing recorded that they had never been looked at.
  8. This script turns that into something that cannot drift:
  9. - it implements the same thirteen risk dimensions as an executable scanner;
  10. - it records what was scanned in `catalog/security-scan.json`, keyed by
  11. collection, so coverage is a fact rather than a memory;
  12. - `--check` fails when a cataloged collection has no scan record, which is what
  13. `make validate` uses to keep the gap closed.
  14. What this is **not**: it is a pattern scan, the same class of evidence as the
  15. 2026-07-15 addendum, and strictly weaker than the baseline's multi-agent content
  16. read. A clean result here means "no known-bad pattern matched", not "reviewed
  17. and safe". The report says so and so does the generated record; do not let a
  18. green check here be read as the stronger claim.
  19. Findings are triaged, not just counted. A vendored README quoting the upstream
  20. author's own `curl | sh` install line is a documentation artifact, not an
  21. executable risk, and the baseline scan reached that conclusion by hand for nine
  22. hits. Rather than re-litigate those every run, confirmed-benign findings are
  23. recorded with their reason in `catalog/security-scan.json` and suppressed on
  24. later runs; anything *new* surfaces.
  25. Zero third-party dependencies. Wired into `make validate` (`--check`) and
  26. `make security-scan`.
  27. """
  28. from __future__ import annotations
  29. import argparse
  30. import base64
  31. import binascii
  32. import json
  33. import re
  34. import subprocess
  35. import sys
  36. from pathlib import Path
  37. ROOT = Path(__file__).resolve().parents[1]
  38. SKILLS = ROOT / "skills"
  39. CATALOG = ROOT / "catalog" / "skills.json"
  40. RECORD = ROOT / "catalog" / "security-scan.json"
  41. # Binary and generated content the pattern scan cannot meaningfully read.
  42. SKIP_SUFFIXES = {
  43. ".png", ".jpg", ".jpeg", ".gif", ".svg", ".webp", ".ico", ".pdf",
  44. ".zip", ".gz", ".tar", ".xz", ".7z", ".woff", ".woff2", ".ttf", ".otf",
  45. ".eot", ".mp4", ".mov", ".mp3", ".wav", ".dta", ".sav", ".rdata", ".rds",
  46. ".pyc", ".so", ".dylib", ".dll", ".xlsx", ".docx", ".pptx",
  47. }
  48. SKIP_DIRS = {".git", "__pycache__", "node_modules", ".venv", ".pytest_cache"}
  49. MAX_BYTES = 2_000_000 # a file larger than this is data, not code we can read
  50. # The thirteen risk dimensions of the baseline scan, as executable patterns.
  51. # Each is (id, severity, description, regex).
  52. CHECKS: tuple[tuple[str, str, str, re.Pattern], ...] = (
  53. (
  54. "pipe-to-shell", "high",
  55. "Downloads a script and pipes it straight into a shell",
  56. re.compile(r"(?:curl|wget)\b[^\n|]{0,200}\|\s*(?:sudo\s+)?(?:ba|z|k)?sh\b"),
  57. ),
  58. (
  59. "reverse-shell", "critical",
  60. "Opens an interactive shell back to a remote host",
  61. re.compile(
  62. r"(?:bash\s+-i\s*>&\s*/dev/tcp/)"
    • Note: Quoted as data in code: Shows a reverse-shell command template (placeholder or no destination)
    • Line 73 on GitHub
  63. r"|(?:nc\b[^\n]{0,60}-e\s*/bin/(?:ba)?sh)"
  64. r"|(?:socket\.socket\([^\n]{0,80}\)[^\n]{0,200}subprocess\.(?:call|Popen)\()"
  65. ),
  66. ),
  67. (
  68. # Decoding a blob is ordinary (images, attachments). Decoding one and
  69. # *running* it is the risk, so the pattern requires both on one line.
  70. "decode-then-execute", "critical",
  71. "Decodes a blob and executes the result",
  72. re.compile(
  73. r"(?:\beval\b|\bexec\b)\s*\(\s*(?:base64|codecs|binascii|bytes)\s*\."
  74. r"|(?:base64|codecs|binascii)\s*\.\s*[a-z0-9_]*(?:decode|unhexlify)\s*\("
  75. r"[^\n]{0,200}\)[^\n]{0,60}(?:\beval\b|\bexec\b)\s*\("
  76. ),
  77. ),
  78. (
  79. "long-base64-blob", "low",
  80. "Long base64-looking literal (usually an embedded image; check it is)",
  81. re.compile(r"[A-Za-z0-9+/]{400,}={0,2}"),
  82. ),
  83. (
  84. "credential-path", "medium",
  85. "Reads a well-known credential or key location",
  86. re.compile(
  87. r"~/\.(?:ssh/id_[a-z0-9]+|aws/credentials|netrc|npmrc|docker/config\.json)"
  88. r"|/\.ssh/id_(?:rsa|ed25519|ecdsa)\b"
  89. r"|\.git-credentials\b"
  90. ),
  91. ),
  92. (
  93. "env-exfiltration", "high",
  94. "Sends environment variables or secrets to a remote endpoint",
  95. re.compile(
  96. r"(?:os\.environ|process\.env|\$ENV|printenv|env\b)[^\n]{0,120}"
  97. r"(?:requests\.(?:post|put)|urllib[^\n]{0,40}urlopen|curl\s|fetch\(|axios\.)"
  98. ),
  99. ),
  100. (
  101. "remote-eval", "critical",
  102. "Fetches remote content and evaluates it",
  103. re.compile(
  104. r"(?:eval|exec)\s*\(\s*(?:requests\.get|urlopen|fetch)\("
  105. r"|(?:requests\.get|urlopen)\([^\n]{0,200}\)\s*\.\s*(?:text|read\(\))\s*\)?\s*\)"
  106. r"(?=[^\n]{0,40}(?:eval|exec))"
  107. ),
  108. ),
  109. (
  110. # `rm -rf ~/.cache/matplotlib` and `rm -rf /var/lib/apt/lists/*` are
  111. # routine; `rm -rf /` and `rm -rf ~` are not. The lookahead is what
    • Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
    • Line 122 on GitHub
  112. # separates them — without it the check fires on every Dockerfile and
  113. # every uninstall doc, and stops being read.
  114. "destructive-rm", "high",
  115. "Recursive force-delete of a bare root, home, or wildcard target",
  116. # Flags are matched loosely and then checked by a post-filter, so
  117. # `-rf`, `-fr`, `-r -f` and `--recursive --force` all count.
  118. re.compile(
  119. r"\brm\s+(?:-[a-zA-Z-]+\s+)+"
  120. r"(?:/|~|\$HOME|\*|\.)(?=[\"'\s;)&|]|$)"
  121. ),
  122. ),
  123. (
  124. "cryptominer", "critical",
  125. "Cryptocurrency mining signature",
  126. re.compile(
  127. r"\b(?:xmrig|stratum\+tcp|minergate|cryptonight|nicehash|coinhive)\b",
  128. re.IGNORECASE,
  129. ),
  130. ),
  131. (
  132. "input-capture", "critical",
  133. "Keyboard or clipboard capture",
  134. re.compile(
  135. r"\b(?:pynput|keyboard\.Listener|GetAsyncKeyState|CGEventTapCreate)\b"
  136. r"|\bpyperclip\.paste\b[^\n]{0,120}(?:post|urlopen|send)"
  137. ),
  138. ),
  139. (
  140. "suspicious-webhook", "medium",
  141. "Posts to a chat/webhook endpoint",
  142. re.compile(
  143. r"https?://(?:hooks\.slack\.com|discord(?:app)?\.com/api/webhooks|"
  144. r"api\.telegram\.org/bot|webhook\.site|requestbin)",
  145. re.IGNORECASE,
  146. ),
  147. ),
  148. (
  149. "prompt-injection-marker", "high",
  150. "Text aimed at overriding an agent's instructions",
  151. re.compile(
  152. r"(?i)\b(?:ignore (?:all )?(?:previous|prior|above) instructions"
  153. r"|disregard (?:all )?(?:previous|prior) (?:instructions|rules)"
  154. r"|you are now in developer mode"
    • Note: Quoted as data in code: Quotes instruction-override phrasing (in quotes, code, a table or an example): "you are now in developer mode"
    • Line 165 on GitHub
  155. r"|do not (?:tell|inform) the user)\b"
  156. ),
  157. ),
  158. (
  159. "obfuscated-python", "high",
  160. "Python written to hide what it does",
  161. re.compile(
  162. r"getattr\s*\(\s*__import__\s*\("
  163. r"|__import__\s*\(\s*(?:['\"]os['\"]|chr\()"
  164. r"|\bexec\s*\(\s*compile\s*\("
  165. ),
  166. ),
  167. )
  168. class Finding(dict):
  169. """A single pattern hit: check id, file, line, and the matched text."""
  170. def looks_like_real_base64(blob: str) -> bool:
  171. """Reject long alphanumeric runs that are not actually base64.
  172. The `+` in base64's alphabet means a Stata line of concatenated variable
  173. names (``SUBSAustralia+SUBSAmericanSamoa+...``) matches the pattern. Such a
  174. run does not decode; a real embedded PNG or SVG data-URI does. Checking
  175. that removes the false positive without loosening the pattern.
  176. """
  177. try:
  178. base64.b64decode(blob, validate=True)
  179. except (binascii.Error, ValueError):
  180. return False
  181. return True
  182. def is_recursive_force_delete(command: str) -> bool:
  183. """True when an `rm` invocation carries both a recursive and a force flag.
  184. Matching the flag cluster with a regex means picking an order; `rm -fr /` is
    • Dismissed after review: Runs `rm -rf` against a root or home-directory path
    • Line 203 on GitHub
  185. as dangerous as `rm -rf /` and `rm -r -f /` is the same command again. Pull
    • Dismissed after review: Runs `rm -rf` against a root or home-directory path
    • Line 204 on GitHub
  186. the flags out and ask the question directly instead.
  187. """
  188. flags = re.findall(r"(?<=\s)-[a-zA-Z-]+", command)
  189. joined = "".join(flags).lower()
  190. return "r" in joined and "f" in joined
  191. # Extra per-check gates applied after a regex matches.
  192. POST_FILTERS = {
  193. "long-base64-blob": looks_like_real_base64,
  194. "destructive-rm": is_recursive_force_delete,
  195. }
  196. class ScanError(RuntimeError):
  197. """The scan cannot be made reproducible in this checkout."""
  198. def tracked_files() -> set[str]:
  199. """Every git-tracked path under skills/, submodules included.
  200. The scan reads the *tracked* tree, not the working tree, and the reason is
  201. the freshness check: `--check` compares a fresh scan against the committed
  202. record, so the scan has to be a function of the commit. Walking the
  203. directory made it a function of whoever happened to run it — a stray
  204. .DS_Store, a gitignored helper inside the submodule, or a log written by a
  205. previous run all changed the file counts, which is exactly how this first
  206. passed locally and failed in CI.
  207. """
  208. try:
  209. proc = subprocess.run(
  210. ["git", "ls-files", "--recurse-submodules", "-z", "--", "skills"],
  211. cwd=ROOT,
  212. capture_output=True,
  213. text=True,
  214. )
  215. except OSError as exc: # pragma: no cover - git missing
  216. raise ScanError(f"git is required for a reproducible scan: {exc}") from exc
  217. if proc.returncode != 0:
  218. raise ScanError(
  219. "git ls-files failed, so the scan cannot be made reproducible:\n"
  220. + proc.stderr.strip()
  221. )
  222. return {path for path in proc.stdout.split("\0") if path}
  223. def is_scannable(path: Path) -> bool:
  224. if path.suffix.lower() in SKIP_SUFFIXES:
  225. return False
  226. if any(part in SKIP_DIRS for part in path.parts):
  227. return False
  228. try:
  229. return path.stat().st_size <= MAX_BYTES
  230. except OSError:
  231. return False
  232. def read_text(path: Path) -> str | None:
  233. try:
  234. return path.read_text(encoding="utf-8")
  235. except (UnicodeDecodeError, OSError):
  236. return None
  237. def scan_file(path: Path, rel: str) -> list[Finding]:
  238. text = read_text(path)
  239. if text is None:
  240. return []
  241. findings: list[Finding] = []
  242. for check_id, severity, description, pattern in CHECKS:
  243. post_filter = POST_FILTERS.get(check_id)
  244. for match in pattern.finditer(text):
  245. if post_filter is not None and not post_filter(match.group(0)):
  246. continue
  247. line = text.count("\n", 0, match.start()) + 1
  248. excerpt = match.group(0)
  249. if len(excerpt) > 120:
  250. excerpt = excerpt[:117] + "..."
  251. findings.append(
  252. Finding(
  253. check=check_id,
  254. severity=severity,
  255. description=description,
  256. file=rel,
  257. line=line,
  258. excerpt=excerpt.replace("\n", " "),
  259. )
  260. )
  261. return findings
  262. def collection_dirs() -> list[Path]:
  263. if not SKILLS.exists():
  264. return []
  265. return sorted(
  266. p for p in SKILLS.iterdir() if p.is_dir() and not p.name.startswith(".")
  267. )
  268. def scan_collection(directory: Path, tracked: set[str] | None = None) -> dict:
  269. if tracked is None:
  270. tracked = tracked_files()
  271. prefix = directory.relative_to(ROOT).as_posix() + "/"
  272. files_seen = 0
  273. files_scanned = 0
  274. findings: list[Finding] = []
  275. for rel in sorted(path for path in tracked if path.startswith(prefix)):
  276. path = ROOT / rel
  277. if not path.is_file():
  278. # Tracked but absent: an un-initialized submodule. Reported by the
  279. # per-collection counts rather than silently treated as scanned.
  280. continue
  281. files_seen += 1
  282. if not is_scannable(path):
  283. continue
  284. findings.extend(scan_file(path, rel))
  285. files_scanned += 1
  286. return {
  287. "collection": directory.name,
  288. "files_seen": files_seen,
  289. "files_scanned": files_scanned,
  290. "findings": findings,
  291. }
  292. def load_record() -> dict:
  293. if not RECORD.exists():
  294. return {"schema_version": "1.0", "triaged": {}, "collections": {}}
  295. return json.loads(RECORD.read_text(encoding="utf-8"))
  296. def finding_key(finding: dict) -> str:
  297. """Stable identity for a finding, so a triage decision survives re-runs.
  298. Deliberately excludes the line number: a benign `curl | sh` in a vendored
  299. README stays benign when the file above it gains a paragraph.
  300. """
  301. return f"{finding['check']}|{finding['file']}|{finding['excerpt']}"
  302. def catalog_collections() -> list[str]:
  303. if not CATALOG.exists():
  304. return [d.name for d in collection_dirs()]
  305. data = json.loads(CATALOG.read_text(encoding="utf-8"))
  306. collections = data.get("collections")
  307. if isinstance(collections, list):
  308. out = []
  309. for item in collections:
  310. if isinstance(item, dict) and item.get("id"):
  311. out.append(item["id"])
  312. elif isinstance(item, str):
  313. out.append(item)
  314. if out:
  315. return sorted(out)
  316. return [d.name for d in collection_dirs()]
  317. def run_scan() -> tuple[dict, list[Finding]]:
  318. record = load_record()
  319. triaged = record.get("triaged", {})
  320. collections: dict[str, dict] = {}
  321. untriaged: list[Finding] = []
  322. tracked = tracked_files()
  323. for directory in collection_dirs():
  324. result = scan_collection(directory, tracked)
  325. new_findings = [f for f in result["findings"] if finding_key(f) not in triaged]
  326. untriaged.extend(new_findings)
  327. collections[result["collection"]] = {
  328. "files_seen": result["files_seen"],
  329. "files_scanned": result["files_scanned"],
  330. "findings_total": len(result["findings"]),
  331. "findings_triaged_benign": len(result["findings"]) - len(new_findings),
  332. "findings_untriaged": len(new_findings),
  333. }
  334. record["collections"] = collections
  335. record["triaged"] = triaged
  336. return record, untriaged
  337. def render_untriaged(findings: list[Finding]) -> str:
  338. order = {"critical": 0, "high": 1, "medium": 2, "low": 3}
  339. lines = []
  340. for finding in sorted(findings, key=lambda f: (order.get(f["severity"], 9), f["file"])):
  341. lines.append(
  342. f" [{finding['severity']:<8}] {finding['check']:<24} "
  343. f"{finding['file']}:{finding['line']}"
  344. )
  345. lines.append(f" {finding['excerpt']}")
  346. return "\n".join(lines)
  347. def main(argv: list[str] | None = None) -> int:
  348. parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
  349. parser.add_argument(
  350. "--check",
  351. action="store_true",
  352. help="fail when a cataloged collection has no scan record, or a finding is untriaged",
  353. )
  354. parser.add_argument(
  355. "--collection", help="scan only this collection (does not update the record)"
  356. )
  357. args = parser.parse_args(argv)
  358. if args.collection:
  359. directory = SKILLS / args.collection
  360. if not directory.is_dir():
  361. print(f"no such collection: {args.collection}", file=sys.stderr)
  362. return 1
  363. try:
  364. result = scan_collection(directory)
  365. except ScanError as exc:
  366. print(str(exc), file=sys.stderr)
  367. return 1
  368. print(
  369. f"{result['collection']}: {result['files_scanned']} of "
  370. f"{result['files_seen']} files scanned, {len(result['findings'])} finding(s)"
  371. )
  372. if result["findings"]:
  373. print(render_untriaged(result["findings"]))
  374. return 0
  375. try:
  376. record, untriaged = run_scan()
  377. except ScanError as exc:
  378. print(str(exc), file=sys.stderr)
  379. return 1
  380. scanned = set(record["collections"])
  381. cataloged = set(catalog_collections())
  382. unscanned = sorted(cataloged - scanned)
  383. empty = sorted(
  384. name for name, stats in record["collections"].items() if stats["files_seen"] == 0
  385. )
  386. if args.check:
  387. problems = []
  388. if empty:
  389. # Almost always an un-initialized submodule: the directory exists so
  390. # it looks scanned, but git lists none of its files, so the record
  391. # would silently claim coverage it does not have.
  392. problems.append(
  393. "these collections contributed no tracked files, so nothing was "
  394. "actually scanned: " + ", ".join(empty)
  395. + "\n If one is a submodule, run: git submodule update --init --recursive"
  396. )
  397. if unscanned:
  398. problems.append(
  399. "collections in the catalog with no scan record: " + ", ".join(unscanned)
  400. )
  401. committed = load_record()
  402. if committed.get("collections") != record["collections"]:
  403. problems.append(
  404. "catalog/security-scan.json is stale. Regenerate with:\n"
  405. " python3 scripts/scan-collections.py"
  406. )
  407. if untriaged:
  408. problems.append(
  409. f"{len(untriaged)} untriaged finding(s):\n" + render_untriaged(untriaged)
  410. + "\n Review each, then record it under \"triaged\" in "
  411. "catalog/security-scan.json with a reason."
  412. )
  413. if problems:
  414. for problem in problems:
  415. print(problem, file=sys.stderr)
  416. return 1
  417. total_files = sum(c["files_scanned"] for c in record["collections"].values())
  418. print(
  419. f"Pattern scan current: {len(scanned)} collection(s), "
  420. f"{total_files} file(s), 0 untriaged finding(s)."
  421. )
  422. return 0
  423. RECORD.parent.mkdir(parents=True, exist_ok=True)
  424. RECORD.write_text(json.dumps(record, indent=2, sort_keys=True) + "\n", encoding="utf-8")
  425. total_files = sum(c["files_scanned"] for c in record["collections"].values())
  426. print(
  427. f"Scanned {len(record['collections'])} collection(s), {total_files} file(s). "
  428. f"{len(untriaged)} untriaged finding(s)."
  429. )
  430. if untriaged:
  431. print(render_untriaged(untriaged))
  432. print(
  433. "\nReview each finding, then record the benign ones under \"triaged\" in "
  434. f"{RECORD.relative_to(ROOT)} with a reason."
  435. )
  436. print(f"Wrote {RECORD.relative_to(ROOT)}")
  437. return 0
  438. if __name__ == "__main__":
  439. raise SystemExit(main())

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.