Skip to content
Back to search

ai-research-skills

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/orchestra-research-ai-research-skills/badge.svg)](/listings/orchestra-research-ai-research-skills)

HTML

<a href="/listings/orchestra-research-ai-research-skills"><img src="/api/listings/orchestra-research-ai-research-skills/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

**For humans** — interactive installer with one command:

npx @orchestra-research/ai-research-skills

**For AI agents** — point your agent to the welcome doc and it handles the rest:

Read https://www.orchestra-research.com/ai-research-skills/welcome.md and follow the instructions to install and use AI Research Skills.

This installs all 98 skills, loads the **autoresearch** orchestration layer, and starts autonomous research.

<details> <summary><b>What the installer does</b></summary>

- **Auto-detects** your installed coding agents (Claude Code, Hermes Agent, OpenCode, Qoder, Cursor, Gemini CLI, etc.) - **Installs** skills to `~/.orchestra/skills/` with symlinks to each agent (falls back to copy on Windows) - **Offers** everything, quickstart bundle, by category, or individual skills - **Updates** installed skills with latest versions - **Uninstalls** all or selected skills

</details>

<details> <summary><b>CLI Commands</b></summary>

# Interactive installer (recommended)
npx @orchestra-research/ai-research-skills

# Direct commands
npx @orchestra-research/ai-research-skills list      # View installed skills
npx @orchestra-research/ai-research-skills update    # Update installed skills

</details>

<details> <summary><b>Claude Code Marketplace (Alternative)</b></summary>

Install skill categories directly using the **Claude Code CLI**:

# Add the marketplace
/plugin marketplace add orchestra-research/AI-research-SKILLs

# Install by category (23 categories available)
/plugin install fine-tuning@ai-research-skills        # Axolotl, LLaMA-Factory, PEFT, Unsloth
/plugin install post-training@ai-research-skills      # TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge
/plugin install inference-serving@ai-research-skills  # vLLM, TensorRT-LLM, llama.cpp, SGLang
/plugin install distributed-training@ai-research-skills
/plugin install optimization@ai-research-skills

</details>

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 293 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "ai-research-skills",
  3. "owner": {
  4. "name": "Orchestra Research",
  5. "email": "zechen@orchestra-research.com"
  6. },
  7. "metadata": {
  8. "description": "Comprehensive library of 98 AI research engineering skills enabling autonomous AI research from hypothesis to experimental verification",
  9. "version": "1.2.0"
  10. },
  11. "plugins": [
  12. {
  13. "name": "model-architecture",
  14. "description": "LLM architectures and implementations including LitGPT, Mamba, NanoGPT, RWKV, and TorchTitan. Use when implementing, training, or understanding transformer and alternative architectures.",
  15. "source": "./",
  16. "strict": false,
  17. "skills": [
  18. "./01-model-architecture/litgpt",
  19. "./01-model-architecture/mamba",
  20. "./01-model-architecture/nanogpt",
  21. "./01-model-architecture/rwkv",
  22. "./01-model-architecture/torchtitan"
  23. ]
  24. },
  25. {
  26. "name": "tokenization",
  27. "description": "Text tokenization for LLMs including HuggingFace Tokenizers and SentencePiece. Use when training custom tokenizers or handling multilingual text.",
  28. "source": "./",
  29. "strict": false,
  30. "skills": [
  31. "./02-tokenization/huggingface-tokenizers",
  32. "./02-tokenization/sentencepiece"
  33. ]
  34. },
  35. {
  36. "name": "fine-tuning",
  37. "description": "LLM fine-tuning frameworks including Axolotl, LLaMA-Factory, PEFT, and Unsloth. Use when fine-tuning models with LoRA, QLoRA, or full fine-tuning.",
  38. "source": "./",
  39. "strict": false,
  40. "skills": [
  41. "./03-fine-tuning/axolotl",
  42. "./03-fine-tuning/llama-factory",
  43. "./03-fine-tuning/peft",
  44. "./03-fine-tuning/unsloth"
  45. ]
  46. },
  47. {
  48. "name": "mechanistic-interpretability",
  49. "description": "Neural network interpretability tools including TransformerLens, SAELens, NNSight, and pyvene. Use when analyzing model internals, finding circuits, or understanding how models compute.",
  50. "source": "./",
  51. "strict": false,
  52. "skills": [
  53. "./04-mechanistic-interpretability/nnsight",
  54. "./04-mechanistic-interpretability/pyvene",
  55. "./04-mechanistic-interpretability/saelens",
  56. "./04-mechanistic-interpretability/transformer-lens"
  57. ]
  58. },
  59. {
  60. "name": "data-processing",
  61. "description": "Data curation and processing at scale including NeMo Curator and Ray Data. Use when preparing training datasets or processing large-scale data.",
  62. "source": "./",
  63. "strict": false,
  64. "skills": [
  65. "./05-data-processing/nemo-curator",
  66. "./05-data-processing/ray-data"
  67. ]
  68. },
  69. {
  70. "name": "post-training",
  71. "description": "RLHF and preference alignment including TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, and torchforge. Use when aligning models with human preferences, training reward models, or large-scale RL training.",
  72. "source": "./",
  73. "strict": false,
  74. "skills": [
  75. "./06-post-training/grpo-rl-training",
  76. "./06-post-training/miles",
  77. "./06-post-training/openrlhf",
  78. "./06-post-training/simpo",
  79. "./06-post-training/slime",
  80. "./06-post-training/torchforge",
  81. "./06-post-training/trl-fine-tuning",
  82. "./06-post-training/verl"
  83. ]
  84. },
  85. {
  86. "name": "safety-alignment",
  87. "description": "AI safety and content moderation including Constitutional AI, LlamaGuard, NeMo Guardrails, and Prompt Guard. Use when implementing safety filters, content moderation, or prompt injection detection.",
  88. "source": "./",
  89. "strict": false,
  90. "skills": [
  91. "./07-safety-alignment/constitutional-ai",
  92. "./07-safety-alignment/llamaguard",
  93. "./07-safety-alignment/nemo-guardrails",
  94. "./07-safety-alignment/prompt-guard"
  95. ]
  96. },
  97. {
  98. "name": "distributed-training",
  99. "description": "Multi-GPU and multi-node training including DeepSpeed, PyTorch FSDP, Accelerate, Megatron-Core, PyTorch Lightning, and Ray Train. Use when training large models across GPUs.",
  100. "source": "./",
  101. "strict": false,
  102. "skills": [
  103. "./08-distributed-training/accelerate",
  104. "./08-distributed-training/deepspeed",
  105. "./08-distributed-training/megatron-core",
  106. "./08-distributed-training/pytorch-fsdp2",
  107. "./08-distributed-training/pytorch-lightning",
  108. "./08-distributed-training/ray-train"
  109. ]
  110. },
  111. {
  112. "name": "infrastructure",
  113. "description": "GPU cloud and compute orchestration including Modal, Lambda Labs, and SkyPilot. Use when deploying training jobs or managing GPU resources.",
  114. "source": "./",
  115. "strict": false,
  116. "skills": [
  117. "./09-infrastructure/lambda-labs",
  118. "./09-infrastructure/modal",
  119. "./09-infrastructure/skypilot"
  120. ]
  121. },
  122. {
  123. "name": "optimization",
  124. "description": "Model optimization and quantization including Flash Attention, bitsandbytes, GPTQ, AWQ, GGUF, and HQQ. Use when reducing memory, accelerating inference, or quantizing models.",
  125. "source": "./",
  126. "strict": false,
  127. "skills": [
  128. "./10-optimization/awq",
  129. "./10-optimization/bitsandbytes",
  130. "./10-optimization/flash-attention",
  131. "./10-optimization/gguf",
  132. "./10-optimization/gptq",
  133. "./10-optimization/hqq",
  134. "./10-optimization/ml-training-recipes"
  135. ]
  136. },
  137. {
  138. "name": "evaluation",
  139. "description": "LLM benchmarking and evaluation including lm-evaluation-harness, BigCode Evaluation Harness, and NeMo Evaluator. Use when benchmarking models or measuring performance.",
  140. "source": "./",
  141. "strict": false,
  142. "skills": [
  143. "./11-evaluation/bigcode-evaluation-harness",
  144. "./11-evaluation/lm-evaluation-harness",
  145. "./11-evaluation/nemo-evaluator"
  146. ]
  147. },
  148. {
  149. "name": "inference-serving",
  150. "description": "Production LLM inference including vLLM, TensorRT-LLM, llama.cpp, and SGLang. Use when deploying models for production inference.",
  151. "source": "./",
  152. "strict": false,
  153. "skills": [
  154. "./12-inference-serving/llama-cpp",
  155. "./12-inference-serving/sglang",
  156. "./12-inference-serving/tensorrt-llm",
  157. "./12-inference-serving/vllm"
  158. ]
  159. },
  160. {
  161. "name": "mlops",
  162. "description": "ML experiment tracking and lifecycle including Weights & Biases, MLflow, and TensorBoard. Use when tracking experiments or managing models.",
  163. "source": "./",
  164. "strict": false,
  165. "skills": [
  166. "./13-mlops/mlflow",
  167. "./13-mlops/tensorboard",
  168. "./13-mlops/weights-and-biases"
  169. ]
  170. },
  171. {
  172. "name": "agents",
  173. "description": "LLM agent frameworks including LangChain, LlamaIndex, CrewAI, and AutoGPT. Use when building chatbots, autonomous agents, or tool-using systems.",
  174. "source": "./",
  175. "strict": false,
  176. "skills": [
  177. "./14-agents/autogpt",
  178. "./14-agents/crewai",
  179. "./14-agents/langchain",
  180. "./14-agents/llamaindex"
  181. ]
  182. },
  183. {
  184. "name": "rag",
  185. "description": "Retrieval-Augmented Generation including Chroma, FAISS, Pinecone, Qdrant, and Sentence Transformers. Use when building semantic search or document retrieval systems.",
  186. "source": "./",
  187. "strict": false,
  188. "skills": [
  189. "./15-rag/chroma",
  190. "./15-rag/faiss",
  191. "./15-rag/pinecone",
  192. "./15-rag/qdrant",
  193. "./15-rag/sentence-transformers"
  194. ]
  195. },
  196. {
  197. "name": "prompt-engineering",
  198. "description": "Structured LLM outputs including DSPy, Instructor, Guidance, and Outlines. Use when extracting structured data or constraining LLM outputs.",
  199. "source": "./",
  200. "strict": false,
  201. "skills": [
  202. "./16-prompt-engineering/dspy",
  203. "./16-prompt-engineering/guidance",
  204. "./16-prompt-engineering/instructor",
  205. "./16-prompt-engineering/outlines"
  206. ]
  207. },
  208. {
  209. "name": "observability",
  210. "description": "LLM application monitoring including LangSmith and Phoenix. Use when debugging LLM apps or monitoring production systems.",
  211. "source": "./",
  212. "strict": false,
  213. "skills": [
  214. "./17-observability/langsmith",
  215. "./17-observability/phoenix"
  216. ]
  217. },
  218. {
  219. "name": "multimodal",
  220. "description": "Vision, audio, and multimodal models including CLIP, Whisper, LLaVA, BLIP-2, Segment Anything, Stable Diffusion, AudioCraft, Cosmos Policy, OpenPI, and OpenVLA-OFT. Use when working with images, audio, multimodal tasks, or vision-language-action robot policies.",
  221. "source": "./",
  222. "strict": false,
  223. "skills": [
  224. "./18-multimodal/audiocraft",
  225. "./18-multimodal/blip-2",
  226. "./18-multimodal/clip",
  227. "./18-multimodal/cosmos-policy",
  228. "./18-multimodal/llava",
  229. "./18-multimodal/openpi",
  230. "./18-multimodal/openvla-oft",
  231. "./18-multimodal/segment-anything",
  232. "./18-multimodal/stable-diffusion",
  233. "./18-multimodal/whisper"
  234. ]
  235. },
  236. {
  237. "name": "emerging-techniques",
  238. "description": "Advanced ML techniques including MoE Training, Model Merging, Long Context, Speculative Decoding, Knowledge Distillation, and Model Pruning. Use when implementing cutting-edge optimization or architecture techniques.",
  239. "source": "./",
  240. "strict": false,
  241. "skills": [
  242. "./19-emerging-techniques/knowledge-distillation",
  243. "./19-emerging-techniques/long-context",
  244. "./19-emerging-techniques/model-merging",
  245. "./19-emerging-techniques/model-pruning",
  246. "./19-emerging-techniques/moe-training",
  247. "./19-emerging-techniques/speculative-decoding"
  248. ]
  249. },
  250. {
  251. "name": "autoresearch",
  252. "description": "Autonomous research orchestration using a two-loop architecture. Manages the full research lifecycle from literature survey to paper writing, routing to domain-specific skills for execution. Use when starting a research project, running autonomous experiments, or managing multi-hypothesis research.",
  253. "source": "./",
  254. "strict": false,
  255. "skills": [
  256. "./0-autoresearch-skill"
  257. ]
  258. },
  259. {
  260. "name": "ml-paper-writing",
  261. "description": "Write publication-ready ML/AI/Systems papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, OSDI, NSDI, ASPLOS, SOSP. Includes LaTeX templates, citation verification, reviewer guidelines, publication-quality figure generation, systems paper structural blueprints, and conference presentation slides.",
  262. "source": "./",
  263. "strict": false,
  264. "skills": [
  265. "./20-ml-paper-writing/ml-paper-writing",
  266. "./20-ml-paper-writing/academic-plotting",
  267. "./20-ml-paper-writing/systems-paper-writing",
  268. "./20-ml-paper-writing/presenting-conference-talks"
  269. ]
  270. },
  271. {
  272. "name": "ideation",
  273. "description": "Research ideation frameworks including structured brainstorming and creative thinking. Use when exploring new research directions, generating novel ideas, or seeking fresh angles on existing work.",
  274. "source": "./",
  275. "strict": false,
  276. "skills": [
  277. "./21-research-ideation/brainstorming-research-ideas",
  278. "./21-research-ideation/creative-thinking-for-research"
  279. ]
  280. },
  281. {
  282. "name": "agent-native-research-artifact",
  283. "description": "Agent-Native Research Artifact (ARA) tooling: compile any research input (paper, repo, notes) into a structured artifact, record session provenance as a post-task epilogue, and run Seal Level 2 epistemic review. Use when ingesting research into a falsifiable, agent-traversable artifact, capturing how a research project actually evolved, or auditing an ARA for evidence-claim alignment.",
  284. "source": "./",
  285. "strict": false,
  286. "skills": [
  287. "./22-agent-native-research-artifact/compiler",
  288. "./22-agent-native-research-artifact/research-manager",
  289. "./22-agent-native-research-artifact/rigor-reviewer"
  290. ]
  291. }
  292. ]
  293. }
README.md · README · 745 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # AI Research `Skills` Library
  2. > **The most comprehensive open-source skills library enabling AI agents to autonomously conduct AI research — from idea to paper**
  3. <p align="center">
  4. <img src="docs/assets/promo.gif" alt="AI Research Skills Demo" width="700">
  5. </p>
  6. <p align="center">
  7. <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
  8. <a href="https://www.npmjs.com/package/@orchestra-research/ai-research-skills"><img src="https://img.shields.io/npm/v/@orchestra-research/ai-research-skills.svg" alt="npm version"></a>
  9. <a href="https://www.orchestra-research.com/perspectives/ai-research-skills"><img src="https://img.shields.io/badge/Blog-Read%20More-orange.svg" alt="Blog Post"></a>
  10. <a href="https://join.slack.com/t/orchestrarese-efu1990/shared_invite/zt-3iu6gr8io-zJvpkZTPToEviQ9KFZvNSg"><img src="https://img.shields.io/badge/Slack-Join%20Community-4A154B.svg?logo=slack" alt="Slack"></a>
  11. <a href="https://x.com/orch_research"><img src="https://img.shields.io/badge/Twitter-Follow-1DA1F2.svg?logo=x" alt="Twitter"></a>
  12. <a href="https://www.linkedin.com/company/orchestra-research/"><img src="https://img.shields.io/badge/LinkedIn-Follow-0A66C2.svg?logo=linkedin" alt="LinkedIn"></a>
  13. </p>
  14. <div align="center">
  15. ### **98 Skills Powering AI Research in 2026**
  16. </div>
  17. <details>
  18. <summary><b>View All 23 Categories</b></summary>
  19. <div align="center">
  20. | | | |
  21. |:---:|:---:|:---:|
  22. | **Autoresearch** (1) | **Ideation** (2) | **ML Paper Writing** (2) |
  23. | **Model Architecture** (5) | **Fine-Tuning** (4) | **Post-Training** (8) |
  24. | **Distributed Training** (6) | **Optimization** (6) | **Inference** (4) |
  25. | **Tokenization** (2) | **Data Processing** (2) | **Evaluation** (3) |
  26. | **Safety & Alignment** (4) | **Agents** (4) | **RAG** (5) |
  27. | **Multimodal** (7) | **Prompt Engineering** (4) | **MLOps** (3) |
  28. | **Observability** (2) | **Infrastructure** (3) | **Mech Interp** (4) |
  29. | **Emerging Techniques** (6) | **Agent-Native Research Artifact** (3) | |
  30. </div>
  31. </details>
  32. ---
  33. ## Table of Contents
  34. - [Our Mission](#our-mission)
  35. - [Path Towards AI Research Agent](#path-towards-ai-research-agent)
  36. - [Available AI Research Engineering Skills](#available-ai-research-engineering-skills)
  37. - [Demos](#demos)
  38. - [Skill Structure](#skill-structure)
  39. - [Roadmap](#roadmap)
  40. - [Repository Structure](#repository-structure)
  41. - [Use Cases](#use-cases)
  42. - [Contributors](#contributors)
  43. - [Citation](#citation)
  44. - [Community](#community)
  45. ## Our Mission
  46. We enable AI agents to **autonomously conduct AI research** — from literature survey and idea generation through experiment execution to paper writing. The library provides both the **research orchestration layer** (autoresearch, ideation, paper writing) and the **engineering skills** (training, evaluation, deployment) needed at each stage.
  47. <p align="center">
  48. <img src="docs/skills.png" alt="AI Research Agent System" width="50%">
  49. <br>
  50. <em>System diagram of an AI research agent</em>
  51. </p>
  52. ## Path Towards AI Research Agent
  53. Modern AI research requires mastering dozens of specialized tools and frameworks.
  54. AI Researchers spend more time debugging infrastructure than testing hypotheses — slowing the pace of scientific discovery.
  55. We provide a comprehensive skills library that enables AI agents to autonomously conduct the full research lifecycle — from brainstorming ideas to writing the paper.
  56. - Autonomous Research - The **autoresearch** skill orchestrates the entire research workflow using a two-loop architecture, routing to domain skills as needed
  57. - Specialized Expertise - Each domain skill provides deep, production-ready knowledge of a specific framework (Megatron-LM, vLLM, TRL, etc.)
  58. - End-to-End Coverage - 98 skills spanning the full AI research lifecycle, from ideation and literature survey to experiments and paper writing
  59. - Research-Grade Quality - Documentation sourced from official repos, real GitHub issues, and battle-tested production workflows
  60. ## Available AI Research Engineering Skills
  61. **Quality over quantity**: Each skill provides comprehensive, expert-level guidance with real code examples, troubleshooting guides, and production-ready workflows.
  62. ### 📦 Quick Install (Recommended)
  63. **For humans** — interactive installer with one command:
  64. ```bash
  65. npx @orchestra-research/ai-research-skills
  66. ```
  67. **For AI agents** — point your agent to the welcome doc and it handles the rest:
  68. ```
  69. Read https://www.orchestra-research.com/ai-research-skills/welcome.md and follow the instructions to install and use AI Research Skills.
  70. ```
  71. This installs all 98 skills, loads the **autoresearch** orchestration layer, and starts autonomous research.
  72. <details>
  73. <summary><b>What the installer does</b></summary>
  74. - **Auto-detects** your installed coding agents (Claude Code, Hermes Agent, OpenCode, Qoder, Cursor, Gemini CLI, etc.)
  75. - **Installs** skills to `~/.orchestra/skills/` with symlinks to each agent (falls back to copy on Windows)
  76. - **Offers** everything, quickstart bundle, by category, or individual skills
  77. - **Updates** installed skills with latest versions
  78. - **Uninstalls** all or selected skills
  79. </details>
  80. <details>
  81. <summary><b>CLI Commands</b></summary>
  82. ```bash
  83. # Interactive installer (recommended)
  84. npx @orchestra-research/ai-research-skills
  85. # Direct commands
  86. npx @orchestra-research/ai-research-skills list # View installed skills
  87. npx @orchestra-research/ai-research-skills update # Update installed skills
  88. ```
  89. </details>
  90. <details>
  91. <summary><b>Claude Code Marketplace (Alternative)</b></summary>
  92. Install skill categories directly using the **Claude Code CLI**:
  93. ```bash
  94. # Add the marketplace
  95. /plugin marketplace add orchestra-research/AI-research-SKILLs
  96. # Install by category (23 categories available)
  97. /plugin install fine-tuning@ai-research-skills # Axolotl, LLaMA-Factory, PEFT, Unsloth
  98. /plugin install post-training@ai-research-skills # TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge
  99. /plugin install inference-serving@ai-research-skills # vLLM, TensorRT-LLM, llama.cpp, SGLang
  100. /plugin install distributed-training@ai-research-skills
  101. /plugin install optimization@ai-research-skills
  102. ```
  103. </details>
  104. ### All 23 Categories (98 Skills)
  105. | Category | Skills | Included |
  106. |----------|--------|----------|
  107. | **Autoresearch** | **1** | **Autonomous research orchestration — central layer that manages the full lifecycle and routes to all other skills** |
  108. | Ideation | 2 | Research Brainstorming, Creative Thinking |
  109. | ML Paper Writing | 2 | ML Paper Writing (LaTeX templates, citation verification), Academic Plotting |
  110. | Model Architecture | 5 | LitGPT, Mamba, NanoGPT, RWKV, TorchTitan |
  111. | Tokenization | 2 | HuggingFace Tokenizers, SentencePiece |
  112. | Fine-Tuning | 4 | Axolotl, LLaMA-Factory, PEFT, Unsloth |
  113. | Mech Interp | 4 | TransformerLens, SAELens, pyvene, nnsight |
  114. | Data Processing | 2 | NeMo Curator, Ray Data |
  115. | Post-Training | 8 | TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge |
  116. | Safety | 4 | Constitutional AI, LlamaGuard, NeMo Guardrails, Prompt Guard |
  117. | Distributed | 6 | DeepSpeed, FSDP, Accelerate, Megatron-Core, Lightning, Ray Train |
  118. | Infrastructure | 3 | Modal, Lambda Labs, SkyPilot |
  119. | Optimization | 6 | Flash Attention, bitsandbytes, GPTQ, AWQ, HQQ, GGUF |
  120. | Evaluation | 3 | lm-eval-harness, BigCode, NeMo Evaluator |
  121. | Inference | 4 | vLLM, TensorRT-LLM, llama.cpp, SGLang |
  122. | MLOps | 3 | W&B, MLflow, TensorBoard |
  123. | Agents | 4 | LangChain, LlamaIndex, CrewAI, AutoGPT |
  124. | RAG | 5 | Chroma, FAISS, Pinecone, Qdrant, Sentence Transformers |
  125. | Prompt Eng | 4 | DSPy, Instructor, Guidance, Outlines |
  126. | Observability | 2 | LangSmith, Phoenix |
  127. | Multimodal | 7 | CLIP, Whisper, LLaVA, BLIP-2, SAM, Stable Diffusion, AudioCraft |
  128. | Emerging | 6 | MoE, Model Merging, Long Context, Speculative Decoding, Distillation, Pruning |
  129. | Agent-Native Research Artifact | 3 | ARA Compiler, Research Manager, Rigor Reviewer |
  130. <details>
  131. <summary><b>View All 98 Skills in Details</b></summary>
  132. ### 🔬 Autoresearch (1 skill) — Central Orchestration Layer
  133. - **[Autoresearch](0-autoresearch-skill/)** - Autonomous research orchestration using a two-loop architecture (inner optimization + outer synthesis). Manages the full lifecycle from literature survey to paper writing, routing to all domain-specific skills. Supports Claude Code /loop and OpenClaw heartbeat for continuous operation (390 lines + 3 refs)
  134. ### 🏗️ Model Architecture (5 skills)
  135. - **[LitGPT](01-model-architecture/litgpt/)** - Lightning AI's 20+ clean LLM implementations with production training recipes (462 lines + 4 refs)
  136. - **[Mamba](01-model-architecture/mamba/)** - State-space models with O(n) complexity, 5× faster than Transformers (253 lines + 3 refs)
  137. - **[RWKV](01-model-architecture/rwkv/)** - RNN+Transformer hybrid, infinite context, Linux Foundation project (253 lines + 3 refs)
  138. - **[NanoGPT](01-model-architecture/nanogpt/)** - Educational GPT in ~300 lines by Karpathy (283 lines + 3 refs)
  139. - **[TorchTitan](01-model-architecture/torchtitan/)** - PyTorch-native distributed training for Llama 3.1 with 4D parallelism
  140. ### 🔤 Tokenization (2 skills)
  141. - **[HuggingFace Tokenizers](02-tokenization/huggingface-tokenizers/)** - Rust-based, <20s/GB, BPE/WordPiece/Unigram algorithms (486 lines + 4 refs)
  142. - **[SentencePiece](02-tokenization/sentencepiece/)** - Language-independent, 50k sentences/sec, used by T5/ALBERT (228 lines + 2 refs)
  143. ### 🎯 Fine-Tuning (4 skills)
  144. - **[Axolotl](03-fine-tuning/axolotl/)** - YAML-based fine-tuning with 100+ models (156 lines + 4 refs)
  145. - **[LLaMA-Factory](03-fine-tuning/llama-factory/)** - WebUI no-code fine-tuning (78 lines + 5 refs)
  146. - **[Unsloth](03-fine-tuning/unsloth/)** - 2x faster QLoRA fine-tuning (75 lines + 4 refs)
  147. - **[PEFT](03-fine-tuning/peft/)** - Parameter-efficient fine-tuning with LoRA, QLoRA, DoRA, 25+ methods (431 lines + 2 refs)
  148. ### 🔬 Mechanistic Interpretability (4 skills)
  149. - **[TransformerLens](04-mechanistic-interpretability/transformer-lens/)** - Neel Nanda's library for mech interp with HookPoints, activation caching (346 lines + 3 refs)
  150. - **[SAELens](04-mechanistic-interpretability/saelens/)** - Sparse Autoencoder training and analysis for feature discovery (386 lines + 3 refs)
  151. - **[pyvene](04-mechanistic-interpretability/pyvene/)** - Stanford's causal intervention library with declarative configs (473 lines + 3 refs)
  152. - **[nnsight](04-mechanistic-interpretability/nnsight/)** - Remote interpretability via NDIF, run experiments on 70B+ models (436 lines + 3 refs)
  153. ### 📊 Data Processing (2 skills)
  154. - **[Ray Data](05-data-processing/ray-data/)** - Distributed ML data processing, streaming execution, GPU support (318 lines + 2 refs)
  155. - **[NeMo Curator](05-data-processing/nemo-curator/)** - GPU-accelerated data curation, 16× faster deduplication (375 lines + 2 refs)
  156. ### 🎓 Post-Training (8 skills)
  157. - **[TRL Fine-Tuning](06-post-training/trl-fine-tuning/)** - Transformer Reinforcement Learning (447 lines + 4 refs)
  158. - **[GRPO-RL-Training](06-post-training/grpo-rl-training/)** (TRL) - Group Relative Policy Optimization with TRL (569 lines, **gold standard**)
  159. - **[OpenRLHF](06-post-training/openrlhf/)** - Full RLHF pipeline with Ray + vLLM (241 lines + 4 refs)
  160. - **[SimPO](06-post-training/simpo/)** - Simple Preference Optimization, no reference model needed (211 lines + 3 refs)
  161. - **[verl](06-post-training/verl/)** - ByteDance's HybridFlow RL framework, FSDP/Megatron + vLLM/SGLang backends (389 lines + 2 refs)
  162. - **[slime](06-post-training/slime/)** - THUDM's Megatron+SGLang framework powering GLM-4.x models (464 lines + 2 refs)
  163. - **[miles](06-post-training/miles/)** - Enterprise fork of slime with FP8, INT4, speculative RL for MoE training (315 lines + 2 refs)
  164. - **[torchforge](06-post-training/torchforge/)** - Meta's PyTorch-native RL with Monarch+TorchTitan+vLLM (380 lines + 2 refs)
  165. ### 🛡️ Safety & Alignment (4 skills)
  166. - **[Constitutional AI](07-safety-alignment/constitutional-ai/)** - AI-driven self-improvement via principles (282 lines)
  167. - **[LlamaGuard](07-safety-alignment/llamaguard/)** - Safety classifier for LLM inputs/outputs (329 lines)
  168. - **[NeMo Guardrails](07-safety-alignment/nemo-guardrails/)** - Programmable guardrails with Colang (289 lines)
  169. - **[Prompt Guard](07-safety-alignment/prompt-guard/)** - Meta's 86M prompt injection & jailbreak detector, 99%+ TPR, <2ms GPU (313 lines)
  170. ### ⚡ Distributed Training (6 skills)
  171. - **[Megatron-Core](08-distributed-training/megatron-core/)** - NVIDIA's framework for training 2B-462B param models with 47% MFU on H100 (359 lines + 4 refs)
  172. - **[DeepSpeed](08-distributed-training/deepspeed/)** - Microsoft's ZeRO optimization (137 lines + 9 refs)
  173. - **[PyTorch FSDP2](08-distributed-training/pytorch-fsdp2/)** - Fully Sharded Data Parallel v2 with `fully_shard` and DTensor (231 lines + 12 refs)
  174. - **[Accelerate](08-distributed-training/accelerate/)** - HuggingFace's 4-line distributed training API (324 lines + 3 refs)
  175. - **[PyTorch Lightning](08-distributed-training/pytorch-lightning/)** - High-level training framework with Trainer class (339 lines + 3 refs)
  176. - **[Ray Train](08-distributed-training/ray-train/)** - Multi-node orchestration and hyperparameter tuning (399 lines + 1 ref)
  177. ### 🚀 Optimization (6 skills)
  178. - **[Flash Attention](10-optimization/flash-attention/)** - 2-4x faster attention with memory efficiency (359 lines + 2 refs)
  179. - **[bitsandbytes](10-optimization/bitsandbytes/)** - 8-bit/4-bit quantization for 50-75% memory reduction (403 lines + 3 refs)
  180. - **[GPTQ](10-optimization/gptq/)** - 4-bit post-training quantization, 4× memory reduction, <2% accuracy loss (443 lines + 3 refs)
  181. - **[AWQ](10-optimization/awq/)** - Activation-aware weight quantization, 4-bit with minimal accuracy loss (310 lines + 2 refs)
  182. - **[HQQ](10-optimization/hqq/)** - Half-Quadratic Quantization, no calibration data needed, multi-backend (370 lines + 2 refs)
  183. - **[GGUF](10-optimization/gguf/)** - llama.cpp quantization format, K-quant methods, CPU/Metal inference (380 lines + 2 refs)
  184. ### 📊 Evaluation (3 skills)
  185. - **[lm-evaluation-harness](11-evaluation/lm-evaluation-harness/)** - EleutherAI's standard for benchmarking LLMs across 60+ tasks (482 lines + 4 refs)
  186. - **[BigCode Evaluation Harness](11-evaluation/bigcode-evaluation-harness/)** - Code model benchmarking with HumanEval, MBPP, MultiPL-E, pass@k metrics (406 lines + 3 refs)
  187. - **[NeMo Evaluator](11-evaluation/nemo-evaluator/)** - NVIDIA's enterprise platform for 100+ benchmarks across 18+ harnesses with multi-backend execution (454 lines + 4 refs)
  188. ### ☁️ Infrastructure (3 skills)
  189. - **[Modal](09-infrastructure/modal/)** - Serverless GPU cloud with Python-native API, T4-H200 on-demand (342 lines + 2 refs)
  190. - **[SkyPilot](09-infrastructure/skypilot/)** - Multi-cloud orchestration across 20+ providers with spot recovery (390 lines + 2 refs)
  191. - **[Lambda Labs](09-infrastructure/lambda-labs/)** - Reserved/on-demand GPU cloud with H100/A100, persistent filesystems (390 lines + 2 refs)
  192. ### 🔥 Inference & Serving (4 skills)
  193. - **[vLLM](12-inference-serving/vllm/)** - High-throughput LLM serving with PagedAttention (356 lines + 4 refs, **production-ready**)
  194. - **[TensorRT-LLM](12-inference-serving/tensorrt-llm/)** - NVIDIA's fastest inference, 24k tok/s, FP8/INT4 quantization (180 lines + 3 refs)
  195. - **[llama.cpp](12-inference-serving/llama-cpp/)** - CPU/Apple Silicon inference, GGUF quantization (251 lines + 3 refs)
  196. - **[SGLang](12-inference-serving/sglang/)** - Structured generation with RadixAttention, 5-10× faster for agents (435 lines + 3 refs)
  197. ### 🤖 Agents (4 skills)
  198. - **[LangChain](14-agents/langchain/)** - Most popular agent framework, 500+ integrations, ReAct pattern (658 lines + 3 refs, **production-ready**)
  199. - **[LlamaIndex](14-agents/llamaindex/)** - Data framework for LLM apps, 300+ connectors, RAG-focused (535 lines + 3 refs)
  200. - **[CrewAI](14-agents/crewai/)** - Multi-agent orchestration, role-based collaboration, autonomous workflows (498 lines + 3 refs)
  201. - **[AutoGPT](14-agents/autogpt/)** - Autonomous AI agent platform, visual workflow builder, continuous execution (400 lines + 2 refs)
  202. ### 🔍 RAG (5 skills)
  203. - **[Chroma](15-rag/chroma/)** - Open-source embedding database, local/cloud, 24k stars (385 lines + 1 ref)
  204. - **[FAISS](15-rag/faiss/)** - Facebook's similarity search, billion-scale, GPU acceleration (295 lines)
  205. - **[Sentence Transformers](15-rag/sentence-transformers/)** - 5000+ embedding models, multilingual, 15k stars (370 lines)
  206. - **[Pinecone](15-rag/pinecone/)** - Managed vector database, auto-scaling, <100ms latency (410 lines)
  207. - **[Qdrant](15-rag/qdrant/)** - High-performance vector search, Rust-powered, hybrid search with filtering (493 lines + 2 refs)
  208. ### 🎨 Multimodal (7 skills)
  209. - **[CLIP](18-multimodal/clip/)** - OpenAI's vision-language model, zero-shot classification, 25k stars (320 lines)
  210. - **[Whisper](18-multimodal/whisper/)** - Robust speech recognition, 99 languages, 73k stars (395 lines)
  211. - **[LLaVA](18-multimodal/llava/)** - Vision-language assistant, image chat, GPT-4V level (360 lines)
  212. - **[Stable Diffusion](18-multimodal/stable-diffusion/)** - Text-to-image generation via HuggingFace Diffusers, SDXL, ControlNet (380 lines + 2 refs)
  213. - **[Segment Anything](18-multimodal/segment-anything/)** - Meta's SAM for zero-shot image segmentation with points/boxes (500 lines + 2 refs)
  214. - **[BLIP-2](18-multimodal/blip-2/)** - Vision-language pretraining with Q-Former, image captioning, VQA (500 lines + 2 refs)
  215. - **[AudioCraft](18-multimodal/audiocraft/)** - Meta's MusicGen/AudioGen for text-to-music and text-to-sound (470 lines + 2 refs)
  216. ### 🎯 Prompt Engineering (4 skills)
  217. - **[DSPy](16-prompt-engineering/dspy/)** - Declarative prompt programming with optimizers, Stanford NLP, 22k stars (438 lines + 3 refs)
  218. - **[Instructor](16-prompt-engineering/instructor/)** - Structured LLM outputs with Pydantic validation, 15k stars (726 lines + 3 refs)
  219. - **[Guidance](16-prompt-engineering/guidance/)** - Constrained generation with regex/grammars, Microsoft Research, 18k stars (485 lines + 3 refs)
  220. - **[Outlines](16-prompt-engineering/outlines/)** - Structured text with FSM, zero-overhead, 8k stars (601 lines + 3 refs)
  221. ### 📊 MLOps (3 skills)
  222. - **[Weights & Biases](13-mlops/weights-and-biases/)** - Experiment tracking, sweeps, artifacts, model registry (427 lines + 3 refs)
  223. - **[MLflow](13-mlops/mlflow/)** - Model registry, tracking, deployment, autologging (514 lines + 3 refs)
  224. - **[TensorBoard](13-mlops/tensorboard/)** - Visualization, profiling, embeddings, scalars/images (538 lines + 3 refs)
  225. ### 👁️ Observability (2 skills)
  226. - **[LangSmith](17-observability/langsmith/)** - LLM observability, tracing, evaluation, monitoring for AI apps (422 lines + 2 refs)
  227. - **[Phoenix](17-observability/phoenix/)** - Open-source AI observability with OpenTelemetry tracing and LLM evaluation (380 lines + 2 refs)
  228. ### 🔬 Emerging Techniques (6 skills)
  229. - **[MoE Training](19-emerging-techniques/moe-training/)** - Mixture of Experts training with DeepSpeed, Mixtral 8x7B, 5× cost reduction (515 lines + 3 refs)
  230. - **[Model Merging](19-emerging-techniques/model-merging/)** - Combine models with TIES, DARE, SLERP using mergekit (528 lines + 3 refs)
  231. - **[Long Context](19-emerging-techniques/long-context/)** - Extend context windows with RoPE, YaRN, ALiBi, 32k-128k tokens (624 lines + 3 refs)
  232. - **[Speculative Decoding](19-emerging-techniques/speculative-decoding/)** - 1.5-3.6× faster inference with Medusa, Lookahead (379 lines)
  233. - **[Knowledge Distillation](19-emerging-techniques/knowledge-distillation/)** - Compress models 70B→7B with MiniLLM, temperature scaling (424 lines)
  234. - **[Model Pruning](19-emerging-techniques/model-pruning/)** - 50% sparsity with Wanda, SparseGPT, <1% accuracy loss (417 lines)
  235. ### 📝 ML Paper Writing (2 skills)
  236. - **[ML Paper Writing](20-ml-paper-writing/)** - Write publication-ready papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM with LaTeX templates, citation verification, and writing best practices (532 lines + 5 refs)
  237. - **[Academic Plotting](20-ml-paper-writing/academic-plotting/)** - Generate publication-quality figures for ML papers: architecture diagrams via Gemini AI and data-driven charts via matplotlib/seaborn with venue-specific styling (479 lines + 3 refs)
  238. ### 💡 Ideation (2 skills)
  239. - **[Research Brainstorming](21-research-ideation/brainstorming-research-ideas/)** - Structured ideation frameworks for discovering high-impact research directions with 10 complementary lenses (384 lines)
  240. - **[Creative Thinking](21-research-ideation/creative-thinking-for-research/)** - Cognitive science frameworks (bisociation, structure-mapping, constraint manipulation) for genuinely novel research ideas (366 lines)
  241. ### 🧬 Agent-Native Research Artifact (3 skills)
  242. - **[ARA Compiler](22-agent-native-research-artifact/compiler/)** - Compiles any research input (PDF papers, repos, experiment logs, raw notes) into a complete Agent-Native Research Artifact with claims, exploration graph, evidence, and code stubs (245 lines + 3 refs)
  243. - **[ARA Research Manager](22-agent-native-research-artifact/research-manager/)** - Post-task research recorder that runs at session end to extract decisions, experiments, dead ends, and pivots from conversation history into the `ara/` directory with user-vs-AI provenance tags (324 lines + 3 refs)
  244. - **[ARA Rigor Reviewer](22-agent-native-research-artifact/rigor-reviewer/)** - ARA Seal Level 2 semantic epistemic review scoring six dimensions of research rigor (evidence relevance, falsifiability, scope, coherence, exploration integrity, methodology) with severity-ranked findings (322 lines + 1 ref)
  245. </details>
  246. ## Demos
  247. All 98 skills in this repo are automatically synced to [Orchestra Research](https://www.orchestra-research.com/research-skills), where you can add them to your projects with one click and use them with AI research agents.
  248. **See skills in action → [demos/](demos/README.md)**
  249. We maintain a curated collection of demo repositories showing how to use skills for real AI research tasks:
  250. | Demo | Skills Used | What It Does |
  251. |------|-------------|--------------|
  252. | **[Norm Heterogeneity → LoRA Brittleness](demos/autoresearch-norm-heterogeneity/)** | Autoresearch, ML Paper Writing, Ideation | Agent autonomously discovered norm heterogeneity predicts fine-tuning difficulty (r=-0.99), pivoting from a null result on ETF overlaps |
  253. | **[RL Algorithm Brain Scan](demos/autoresearch-rl-brain-scan/)** | Autoresearch, GRPO, TRL, SAELens, TransformerLens, ML Paper Writing | Agent found DPO is a rank-1 perturbation (95.6% recovery from one SVD direction) while online RL is distributed and structure-preserving |
  254. | **[NeMo Eval: GPQA Benchmark](https://github.com/zechenzhangAGI/Nemo-Eval-Skill-Demo)** | NeMo Evaluator | Compare Llama 8B/70B/405B on graduate-level science questions |
  255. | **[LoRA Without Regret Reproduction](https://www.orchestra-research.com/perspectives/LLM-with-Orchestra)** | GRPO, TRL | Reproduce SFT + GRPO RL experiments via prompting |
  256. | **[Layer-Wise Quantization Experiment](https://github.com/AmberLJC/llama-quantization-experiment)** | llama.cpp, GGUF | Investigate optimal layer precision allocation—early layers at Q8 achieve 1.9× compression with 1.3% perplexity loss |
  257. | **[Cross-Lingual Alignment Analysis](https://github.com/AmberLJC/faiss-demo)** | FAISS | Quantify how well multilingual embeddings align semantic concepts across 8 languages using FAISS similarity search |
  258. | **[Scientific Plotting Demo](demos/scientific-plotting-demo/)** | Academic Plotting | Generate publication-quality figures for the Andes QoE-aware LLM serving paper — Gemini AI architecture diagrams + matplotlib data charts (CDF, multi-panel grids, bar charts) |
  259. **Featured Demos**: Two papers produced entirely by AI agents using the **autoresearch** skill. The [Norm Heterogeneity paper](demos/autoresearch-norm-heterogeneity/) demonstrates autonomous research pivoting — the agent refuted its own hypothesis and discovered a stronger finding. The [RL Brain Scan paper](demos/autoresearch-rl-brain-scan/) demonstrates multi-skill orchestration — the agent trained RL models, analyzed internals with interpretability tools, and synthesized the insight that "DPO is rank-1 alignment." Both papers written end-to-end by the agent.
  260. ## Skill Structure
  261. Each skill follows a battle-tested format for maximum usefulness:
  262. ```
  263. skill-name/
  264. ├── SKILL.md # Quick reference (50-150 lines)
  265. │ ├── Metadata (name, description, version)
  266. │ ├── When to use this skill
  267. │ ├── Quick patterns & examples
  268. │ └── Links to references
  269. │
  270. ├── references/ # Deep documentation (300KB+)
  271. │ ├── README.md # From GitHub/official docs
  272. │ ├── api.md # API reference
  273. │ ├── tutorials.md # Step-by-step guides
  274. │ ├── issues.md # Real GitHub issues & solutions
  275. │ ├── releases.md # Version history & breaking changes
  276. │ └── file_structure.md # Codebase navigation
  277. │
  278. ├── scripts/ # Helper scripts (optional)
  279. └── assets/ # Templates & examples (optional)
  280. ```
  281. <details>
  282. <summary><b>Quality Standards</b></summary>
  283. - 300KB+ documentation from official sources
  284. - Real GitHub issues & solutions (when available)
  285. - Code examples with language detection
  286. - Version history & breaking changes
  287. - Links to official docs
  288. </details>
  289. ## Roadmap
  290. The library spans 98 comprehensive skills across the full AI research lifecycle. See our [detailed roadmap](docs/ROADMAP.md) for the complete development plan.
  291. [View Full Roadmap →](docs/ROADMAP.md)
  292. <details>
  293. <summary><b>View Detailed Statistics</b></summary>
  294. | Metric | Current | Target |
  295. |--------|---------|--------|
  296. | **Skills** | **87** (high-quality, standardized YAML) | 80 ✅ |
  297. | **Avg Lines/Skill** | **420 lines** (focused + progressive disclosure) | 200-600 lines |
  298. | **Documentation** | **~130,000 lines** total (SKILL.md + references) | 100,000+ lines |
  299. | **Gold Standard Skills** | **65** with comprehensive references | 50+ |
  300. | **Contributors** | 1 | 100+ |
  301. | **Coverage** | Architecture, Tokenization, Fine-Tuning, Mechanistic Interpretability, Data Processing, Post-Training, Safety, Distributed, Optimization, Evaluation, Infrastructure, Inference, Agents, RAG, Multimodal, Prompt Engineering, MLOps, Observability, ML Paper Writing, Ideation, Autoresearch | Full Lifecycle ✅ |
  302. **Recent Progress**: npm package `@orchestra-research/ai-research-skills` for one-command installation across all coding agents
  303. **Philosophy**: Quality > Quantity. Following [Anthropic official best practices](anthropic_official_docs/best_practices.md) - each skill provides 200-500 lines of focused, actionable guidance with progressive disclosure.
  304. </details>
  305. ## Repository Structure
  306. ```
  307. claude-ai-research-skills/
  308. ├── README.md ← You are here
  309. ├── CONTRIBUTING.md ← Contribution guide
  310. ├── demos/ ← Curated demo gallery (links to demo repos)
  311. ├── docs/
  312. ├── 0-autoresearch-skill/ (1 skill ✓ - Autonomous research orchestration)
  313. ├── 01-model-architecture/ (5 skills ✓ - LitGPT, Mamba, RWKV, NanoGPT, TorchTitan)
  314. ├── 02-tokenization/ (2 skills ✓ - HuggingFace Tokenizers, SentencePiece)
  315. ├── 03-fine-tuning/ (4 skills ✓ - Axolotl, LLaMA-Factory, Unsloth, PEFT)
  316. ├── 04-mechanistic-interpretability/ (4 skills ✓ - TransformerLens, SAELens, pyvene, nnsight)
  317. ├── 05-data-processing/ (2 skills ✓ - Ray Data, NeMo Curator)
  318. ├── 06-post-training/ (8 skills ✓ - TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge)
  319. ├── 07-safety-alignment/ (4 skills ✓ - Constitutional AI, LlamaGuard, NeMo Guardrails, Prompt Guard)
  320. ├── 08-distributed-training/ (6 skills ✓ - Megatron-Core, DeepSpeed, FSDP, Accelerate, Lightning, Ray Train)
  321. ├── 09-infrastructure/ (3 skills ✓ - Modal, SkyPilot, Lambda Labs)
  322. ├── 10-optimization/ (6 skills ✓ - Flash Attention, bitsandbytes, GPTQ, AWQ, HQQ, GGUF)
  323. ├── 11-evaluation/ (3 skills ✓ - lm-evaluation-harness, BigCode, NeMo Evaluator)
  324. ├── 12-inference-serving/ (4 skills ✓ - vLLM, TensorRT-LLM, llama.cpp, SGLang)
  325. ├── 13-mlops/ (3 skills ✓ - Weights & Biases, MLflow, TensorBoard)
  326. ├── 14-agents/ (4 skills ✓ - LangChain, LlamaIndex, CrewAI, AutoGPT)
  327. ├── 15-rag/ (5 skills ✓ - Chroma, FAISS, Sentence Transformers, Pinecone, Qdrant)
  328. ├── 16-prompt-engineering/ (4 skills ✓ - DSPy, Instructor, Guidance, Outlines)
  329. ├── 17-observability/ (2 skills ✓ - LangSmith, Phoenix)
  330. ├── 18-multimodal/ (7 skills ✓ - CLIP, Whisper, LLaVA, Stable Diffusion, SAM, BLIP-2, AudioCraft)
  331. ├── 19-emerging-techniques/ (6 skills ✓ - MoE, Model Merging, Long Context, Speculative Decoding, Distillation, Pruning)
  332. ├── 20-ml-paper-writing/ (2 skills ✓ - ML Paper Writing with LaTeX templates, Academic Plotting)
  333. ├── 21-research-ideation/ (2 skills ✓ - Research Brainstorming, Creative Thinking)
  334. ├── 22-agent-native-research-artifact/ (3 skills ✓ - ARA Compiler, Research Manager, Rigor Reviewer)
  335. └── packages/ai-research-skills/ (npm package for one-command installation)
  336. ```
  337. ## Use Cases
  338. ### For Researchers
  339. "I need to fine-tune Llama 3 with custom data"
  340. → **03-fine-tuning/axolotl/** - YAML configs, 100+ model support
  341. ### For ML Engineers
  342. "How do I optimize inference latency?"
  343. → **12-inference-serving/vllm/** - PagedAttention, batching
  344. ### For Students
  345. "I want to learn how transformers work"
  346. → **01-model-architecture/litgpt/** - Clean implementations
  347. ### For Teams
  348. "We need to scale training to 100 GPUs"
  349. → **08-distributed-training/deepspeed/** - ZeRO stages, 3D parallelism
  350. ## License
  351. MIT License - See [LICENSE](LICENSE) for details.
  352. **Note**: Individual skills may reference libraries with different licenses. Please check each project's license before use.
  353. ## Citation
  354. If you use AI Research Skills in your work or find it helpful for a publication, we'd appreciate a citation:
  355. **BibTeX**
  356. ```bibtex
  357. @software{ai_research_skills,
  358. title = {AI Research Skills Library},
  359. author = {{Orchestra Research}},
  360. year = {2025},
  361. url = {https://github.com/orchestra-research/AI-research-SKILLs},
  362. note = {Open-source skills library enabling AI agents to autonomously conduct AI research}
  363. }
  364. ```
  365. **APA**
  366. > Orchestra Research. (2025). *AI Research Skills Library* [Computer software]. https://github.com/orchestra-research/AI-research-SKILLs
  367. **Chicago**
  368. > Orchestra Research. "AI Research Skills Library." GitHub, 2025. https://github.com/orchestra-research/AI-research-SKILLs.
  369. **IEEE**
  370. > Orchestra Research, "AI Research Skills Library," 2025. [Online]. Available: https://github.com/orchestra-research/AI-research-SKILLs
  371. > **Tip**: You can also click **"Cite this repository"** in the GitHub sidebar for auto-formatted citations.
  372. ## Acknowledgments
  373. Built with:
  374. - **[Claude Code](https://www.claude.com/product/claude-code)** - AI pair programming
  375. - **[Skill Seeker](https://github.com/yusufkaraaslan/Skill_Seekers)** - Automated doc scraping
  376. - **Open Source AI Community** - For amazing tools and docs
  377. Special thanks to:
  378. - EleutherAI, HuggingFace, NVIDIA, Lightning AI, Meta AI, Anthropic
  379. - All researchers who maintain excellent documentation
  380. ## Contributors
  381. Thanks to all the people who have contributed to the AI Research Skills Library:
  382. <a href="https://github.com/orchestra-research/AI-research-SKILLs/graphs/contributors">
  383. <img src="https://contrib.rocks/image?repo=orchestra-research/AI-research-SKILLs" />
  384. </a>
  385. We welcome contributions from the AI research community! See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines on:
  386. - Adding new skills
  387. - Improving existing skills
  388. - Quality standards and best practices
  389. - Submission process
  390. ## Recent Updates
  391. <details open>
  392. <summary><b>June 2026 - v1.7.1 🧹 Inventory Consistency, Drift Guard & Security Hardening</b></summary>
  393. - 📊 **Repo-wide inventory reconciled to 98 skills / 23 categories** — corrected stale counts that had drifted apart across files: CLAUDE.md (said 90), CONTRIBUTING.md (said 86/22), the README sync line (said 87), WELCOME.md and the npm package README (said 86/22). Also fixed wrong per-category listings (TorchTitan, SwanLab, A-Evolve, ML Training Recipes, Cosmos Policy/OpenPI/OpenVLA-OFT, the paper-writing skills)
  394. - 🛡️ **New CI drift guard** — `scripts/check-inventory.sh` + `check-inventory.yml` fail CI whenever the documented skill/category counts diverge from the actual `SKILL.md` count on disk, so the inventory can't silently drift again
  395. - 📦 **Marketplace sync hardening** — `sync-skills.yml` now prunes build artifacts (`node_modules`/`__pycache__`/`*.pyc`/`.ipynb_checkpoints`) before zipping and fails loudly above 190 files instead of hitting the marketplace's 200-file rejection
  396. - 🔒 **Security** — pinned CLI dependencies (`chalk`/`inquirer`/`ora`) to exact, patched versions with a regenerated lockfile (`inquirer@9.3.8` clears the `tmp` path-traversal advisory; `npm audit` now reports 0 vulnerabilities)
  397. - 🧹 Full open PR/issue triage pass against scope + contribution standards
  398. </details>
  399. <details>
  400. <summary><b>April 2026 - v1.6.0 🧬 Agent-Native Research Artifact (ARA) — 23rd Category, 98 Skills</b></summary>
  401. - 🧬 **NEW CATEGORY**: `22-agent-native-research-artifact/` (the 23rd category) — three skills that turn research outputs into a falsifiable, agent-traversable artifact:
  402. - 🛠️ **[ARA Compiler](22-agent-native-research-artifact/compiler/)** — compiles any input (PDF papers, GitHub repos, experiment logs, raw notes) into a structured ARA with cognitive layer (claims, concepts, heuristics), physical layer (configs, code stubs), exploration graph (research DAG), and grounded evidence
  403. - 📋 **[ARA Research Manager](22-agent-native-research-artifact/research-manager/)** — post-task epilogue that scans conversation history at session end and writes decisions, experiments, dead ends, claims, heuristics, and pivots into the `ara/` directory with `user` / `ai-suggested` / `ai-executed` / `user-revised` provenance tags
  404. - 🔍 **[ARA Rigor Reviewer](22-agent-native-research-artifact/rigor-reviewer/)** — Seal Level 2 semantic epistemic review scoring six dimensions of research rigor (evidence relevance, falsifiability, scope calibration, argument coherence, exploration integrity, methodological rigor) and emitting a severity-ranked report with a Strong Accept-to-Reject recommendation
  405. - 🔗 Sourced from the [Agent-Native-Research-Artifact-Init](https://github.com/Orchestra-Research/Agent-Native-Research-Artifact-Init) reference repo, restructured to AI-research-SKILLs standards (kebab-case names, third-person descriptions, Title-Case tags, one-level-deep references)
  406. - 🧩 Plugin entry `agent-native-research-artifact` added to `.claude-plugin/marketplace.json`; CLI category registered as `22-agent-native-research-artifact` with three individual skill entries in the npm installer
  407. - 🔄 Auto-syncs to Orchestra marketplace via `sync-skills.yml` on push; npm package republished as `@orchestra-research/ai-research-skills@1.6.0` via `publish-npm.yml` on version bump
  408. - 📊 **98 total skills** across **23 categories** — full lifecycle from idea → paper → falsifiable, auditable artifact
  409. </details>
  410. <details>
  411. <summary><b>March 2026 - v1.4.0 🔬 Autoresearch & 86 Skills — Full Research Lifecycle</b></summary>
  412. - 🔬 **NEW SKILL**: **Autoresearch** — autonomous research orchestration using a two-loop architecture (inner optimization loop + outer synthesis loop)
  413. - 🧠 Manages the full research lifecycle: literature survey → ideation → experiments → synthesis → paper writing
  414. - 🔄 Routes to all 86 domain skills automatically — agents don't need to know which skill to use
  415. - ⏰ Mandatory `/loop` (Claude Code) and cron job (OpenClaw) for continuous autonomous operation
  416. - 📊 Generates research presentations (HTML/PDF) with optimization trajectory plots for human review
  417. - 📝 Findings.md as persistent project memory across sessions with "Lessons and Constraints" tracking
  418. - 🗂️ Structured workspace: research-state.yaml, findings.md, research-log.md, literature/, experiments/, src/, data/, to_human/
  419. - 📄 **Two demo papers produced by autoresearch**: [Norm Heterogeneity → LoRA Brittleness](demos/autoresearch-norm-heterogeneity/) and [RL Algorithm Brain Scan](demos/autoresearch-rl-brain-scan/)
  420. - 🚀 WELCOME.md for cold-start agent bootstrap — one URL to go from zero to autonomous research
  421. - 📦 npm v1.4.x with Windows symlink fallback, all 22 categories installable
  422. - 🤖 **Supported agents**: Claude Code, Hermes Agent, OpenCode, OpenClaw, Qoder, Cursor, Codex, Gemini CLI, Qwen Code
  423. - 📊 **87 total skills** across **22 categories** — complete research lifecycle coverage
  424. </details>
  425. <details>
  426. <summary><b>February 2026 - v0.15.0 🛡️ Prompt Guard & 83 Skills</b></summary>
  427. - 🛡️ **NEW SKILL**: Prompt Guard - Meta's 86M prompt injection & jailbreak detector
  428. - ⚡ 99%+ TPR, <1% FPR, <2ms GPU latency, multilingual (8 languages)
  429. - 🔒 3 workflows: user input filtering, third-party data filtering, batch RAG processing
  430. - 📊 **83 total skills** across 20 categories
  431. </details>
  432. <details>
  433. <summary><b>January 2026 - v0.14.0 📦 npm Package & 82 Skills</b></summary>
  434. - 📦 **NEW**: `npx @orchestra-research/ai-research-skills` - One-command installation for all coding agents
  435. - 🤖 **Supported agents**: Claude Code, OpenCode, Cursor, Codex, Gemini CLI, Qwen Code
  436. - ✨ Interactive installer with category/individual skill selection
  437. - 🔄 Update installed skills, selective uninstall
  438. - 📊 **82 total skills** (5 new post-training skills: verl, slime, miles, torchforge + TorchTitan)
  439. - 🏗️ Megatron-Core moved to Distributed Training category
  440. </details>
  441. <details>
  442. <summary><b>January 2026 - v0.13.0 📝 ML Paper Writing & Demos Gallery</b></summary>
  443. - 📝 **NEW CATEGORY**: ML Paper Writing (20th category, 77th skill)
  444. - 🎯 Write publication-ready papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM
  445. - 📚 Writing philosophy from top researchers (Neel Nanda, Farquhar, Gopen & Swan, Lipton, Perez)
  446. - 🔬 Citation verification workflow - never hallucinate references
  447. - 📄 LaTeX templates for 6 major conferences
  448. - 🎪 **NEW**: Curated demos gallery (`demos/`) showcasing skills in action
  449. - 🔗 Demo repos: NeMo Evaluator benchmark, LoRA Without Regret reproduction
  450. - 📖 936-line comprehensive SKILL.md with 4 workflows
  451. </details>
  452. <details>
  453. <summary><b>January 2026 - v0.12.0 📊 NeMo Evaluator SDK</b></summary>
  454. - 📊 **NEW SKILL**: NeMo Evaluator SDK for enterprise LLM benchmarking
  455. - 🔧 NVIDIA's evaluation platform with 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM)
  456. - ⚡ Multi-backend execution: local Docker, Slurm HPC, Lepton cloud
  457. - 📦 Container-first architecture for reproducible evaluation
  458. - 📝 454 lines SKILL.md + 4 comprehensive reference files (~48KB documentation)
  459. </details>
  460. <details>
  461. <summary><b>December 2025 - v0.11.0 🔬 Mechanistic Interpretability</b></summary>
  462. - 🔬 **NEW CATEGORY**: Mechanistic Interpretability (4 skills)
  463. - 🔍 TransformerLens skill: Neel Nanda's library for mech interp with HookPoints, activation caching, circuit analysis
  464. - 🧠 SAELens skill: Sparse Autoencoder training and analysis for feature discovery, monosemanticity research
  465. - ⚡ pyvene skill: Stanford's causal intervention library with declarative configs, DAS, activation patching
  466. - 🌐 nnsight skill: Remote interpretability via NDIF, run experiments on 70B+ models without local GPUs
  467. - 📝 ~6,500 new lines of documentation across 16 files
  468. - **76 total skills** (filling the missing 04 category slot)
  469. </details>
  470. <details>
  471. <summary><b>November 25, 2025 - v0.10.0 🎉 70 Skills Complete!</b></summary>
  472. - 🎉 **ROADMAP COMPLETE**: Reached 70-skill milestone!
  473. - 🚀 Added 4 skills: Lambda Labs, Segment Anything (SAM), BLIP-2, AudioCraft
  474. - ☁️ Lambda Labs skill: Reserved/on-demand GPU cloud with H100/A100, persistent filesystems, 1-Click Clusters
  475. - 🖼️ SAM skill: Meta's Segment Anything for zero-shot image segmentation with points/boxes/masks
  476. - 👁️ BLIP-2 skill: Vision-language pretraining with Q-Former, image captioning, VQA
  477. - 🎵 AudioCraft skill: Meta's MusicGen/AudioGen for text-to-music and text-to-sound generation
  478. - 📝 ~10,000 new lines of documentation across 12 files
  479. - **70 total skills** (100% roadmap complete!)
  480. </details>
  481. <details>
  482. <summary><b>November 25, 2025 - v0.9.0</b></summary>
  483. - 🚀 Added 2 infrastructure skills: Modal, SkyPilot
  484. - ☁️ Modal skill: Serverless GPU cloud with Python-native API, T4-H200 on-demand, auto-scaling
  485. - 🌐 SkyPilot skill: Multi-cloud orchestration across 20+ providers with spot recovery
  486. - ✨ New Infrastructure category (2 skills - serverless GPU and multi-cloud orchestration)
  487. - 📝 ~2,500 new lines of documentation across 6 files
  488. - **66 total skills** (94% towards 70-skill target)
  489. </details>
  490. <details>
  491. <summary><b>November 25, 2025 - v0.8.0</b></summary>
  492. - 🚀 Added 5 high-priority skills: HQQ, GGUF, Phoenix, AutoGPT, Stable Diffusion
  493. - ⚡ HQQ skill: Half-Quadratic Quantization without calibration data, multi-backend support
  494. - 📦 GGUF skill: llama.cpp quantization format, K-quant methods, CPU/Metal inference
  495. - 👁️ Phoenix skill: Open-source AI observability with OpenTelemetry tracing and LLM evaluation
  496. - 🤖 AutoGPT skill: Autonomous AI agent platform with visual workflow builder
  497. - 🎨 Stable Diffusion skill: Text-to-image generation via Diffusers, SDXL, ControlNet, LoRA
  498. - 📝 ~9,000 new lines of documentation across 15 files
  499. - **64 total skills** (91% towards 70-skill target)
  500. </details>
  501. <details>
  502. <summary><b>November 25, 2025 - v0.7.0</b></summary>
  503. - 🚀 Added 5 high-priority skills: PEFT, CrewAI, Qdrant, AWQ, LangSmith
  504. - ✨ New Observability category with LangSmith for LLM tracing and evaluation
  505. - 🎯 PEFT skill: Parameter-efficient fine-tuning with LoRA, QLoRA, DoRA, 25+ methods
  506. - 🤖 CrewAI skill: Multi-agent orchestration with role-based collaboration
  507. - 🔍 Qdrant skill: High-performance Rust vector search with hybrid filtering
  508. - ⚡ AWQ skill: Activation-aware 4-bit quantization with minimal accuracy loss
  509. - 📝 ~8,000 new lines of documentation across 15 files
  510. - **59 total skills** (84% towards 70-skill target)
  511. </details>
  512. <details>
  513. <summary><b>November 15, 2025 - v0.6.0</b></summary>
  514. - 📊 Added 3 comprehensive MLOps skills: Weights & Biases, MLflow, TensorBoard
  515. - ✨ New MLOps category (3 skills - experiment tracking, model registry, visualization)
  516. - 📝 ~10,000 new lines of documentation across 13 files
  517. - 🔧 Comprehensive coverage: experiment tracking, hyperparameter sweeps, model registry, profiling, embeddings visualization
  518. - **54 total skills** (77% towards 70-skill target)
  519. </details>
  520. <details>
  521. <summary><b>November 12, 2025 - v0.5.0</b></summary>
  522. - 🎯 Added 4 comprehensive prompt engineering skills: DSPy, Instructor, Guidance, Outlines
  523. - ✨ New Prompt Engineering category (4 skills - DSPy, Instructor, Guidance, Outlines)
  524. - 📝 ~10,000 new lines of documentation across 16 files
  525. - 🔧 Comprehensive coverage: declarative programming, structured outputs, constrained generation, FSM-based generation
  526. - **47 total skills** (67% towards 70-skill target)
  527. </details>
  528. <details>
  529. <summary><b>November 9, 2025 - v0.4.0</b></summary>
  530. - 🤖 Added 11 comprehensive skills: LangChain, LlamaIndex, Chroma, FAISS, Sentence Transformers, Pinecone, CLIP, Whisper, LLaVA
  531. - ✨ New Agents category (2 skills - LangChain, LlamaIndex)
  532. - 🔍 New RAG category (4 skills - Chroma, FAISS, Sentence Transformers, Pinecone)
  533. - 🎨 New Multimodal category (3 skills - CLIP, Whisper, LLaVA)
  534. - 📝 ~15,000 new lines of documentation
  535. - **43 total skills** (61% towards 70-skill target)
  536. </details>
  537. <details>
  538. <summary><b>November 8, 2025 - v0.3.0</b></summary>
  539. - 🚀 Added 8 comprehensive skills: TensorRT-LLM, llama.cpp, SGLang, GPTQ, HuggingFace Tokenizers, SentencePiece, Ray Data, NeMo Curator
  540. - ⚡ Completed Inference & Serving category (4/4 skills)
  541. - 🔤 New Tokenization category (2 skills)
  542. - 📊 New Data Processing category (2 skills)
  543. - 📝 9,617 new lines of documentation across 30 files
  544. - **32 total skills** (45% towards 70-skill target)
  545. </details>
  546. <details>
  547. <summary><b>November 6, 2025 - v0.2.0</b></summary>
  548. - Added 10 skills from GitHub (Megatron-Core, Lightning, Ray Train, etc.)
  549. - Improved skill structure with comprehensive references
  550. - Created strategic roadmap to 70 skills
  551. - Added contribution guidelines
  552. </details>
  553. <details>
  554. <summary><b>November 3, 2025 - v0.1.0</b></summary>
  555. - 🎉 Initial release with 5 fine-tuning skills
  556. </details>
  557. ## Community
  558. Join our community to stay updated, ask questions, and connect with other AI researchers:
  559. - **[SkillEvolve Meta-Skill](https://github.com/Skill-Evolve/meta-skill)** - Connect your agent to the collective intelligence of the community. Captures techniques discovered during sessions and shares them back as curated skills.
  560. - **[Slack Community](https://join.slack.com/t/orchestrarese-efu1990/shared_invite/zt-3iu6gr8io-zJvpkZTPToEviQ9KFZvNSg)** - Chat with the team and other users
  561. - **[Twitter/X](https://x.com/orch_research)** - Follow for updates and announcements
  562. - **[LinkedIn](https://www.linkedin.com/company/orchestra-research/)** - Connect professionally
  563. ## Star History
  564. <a href="https://star-history.com/#orchestra-research/AI-research-SKILLs&Date">
  565. <picture>
  566. <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=orchestra-research/AI-research-SKILLs&type=Date&theme=dark" />
  567. <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=orchestra-research/AI-research-SKILLs&type=Date" />
  568. <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=orchestra-research/AI-research-SKILLs&type=Date" />
  569. </picture>
  570. </a>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.