Skip to content
Back to search

paper2code

Agent skill to turn any arxiv paper into a working implementation

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/prathamlearnstocode--paper2code/badge.svg)](/listings/prathamlearnstocode--paper2code)

HTML

<a href="/listings/prathamlearnstocode--paper2code"><img src="/api/listings/prathamlearnstocode--paper2code/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

npx skills add PrathamLearnsToCode/paper2code/skills/paper2code

You'll be prompted to: 1. **Select agents** — pick the coding agents you want to use this skill with (e.g., Claude Code) 2. **Choose scope** — Global (recommended) or project-level 3. **Choose method** — Symlink (recommended) or copy

Once installed, open your agent and run the skill:

claude  # or your preferred agent

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 223 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # paper2code
  2. > **arxiv URL in → citation-anchored implementation out**
  3. ```
  4. ┌─────────────────────────────┐ ┌──────────────────────────────────────┐
  5. │ │ │ {paper_slug}/ │
  6. │ /paper2code │ │ ├── README.md │
  7. │ https://arxiv.org/abs/ │ ───▶ │ ├── REPRODUCTION_NOTES.md │
  8. │ 1706.03762 │ │ ├── requirements.txt │
  9. │ │ │ ├── src/ │
  10. │ │ │ │ ├── model.py # §3.2 cited │
  11. │ │ │ │ ├── loss.py # §3.4 cited │
  12. │ │ │ │ ├── train.py # §4.1 cited │
  13. │ │ │ │ ├── data.py │
  14. │ │ │ │ ├── evaluate.py │
  15. │ │ │ │ └── utils.py │
  16. │ │ │ ├── configs/ │
  17. │ │ │ │ └── base.yaml # all params │
  18. │ │ │ └── notebooks/ │
  19. │ │ │ └── walkthrough.ipynb │
  20. └─────────────────────────────┘ └──────────────────────────────────────┘
  21. ```
  22. Built by [Pratham Grover](https://github.com/PrathamLearnsToCode), founder of [TensorTonic](https://www.tensortonic.com/?ref=paper2code), where you implement ML papers yourself and get graded.
  23. ---
  24. ## Why this exists
  25. **The problem:** ML papers are vague. Critical hyperparameters are buried in appendices or omitted entirely. Prose contradicts equations. "Standard settings" refers to nothing specific. When you implement a paper, you spend more time detective-working than coding.
  26. **What LLMs get wrong:** Naive code generation fills in every gap silently and confidently. You get something that runs but doesn't match the paper. Worse, you can't tell which parts are from the paper and which were invented by the model.
  27. **What paper2code does differently:**
  28. 1. **Citation anchoring** — every line of generated code references the exact paper section and equation it implements (`§3.2, Eq. 4`)
  29. 2. **Ambiguity auditing** — before writing a single line of code, every implementation choice is classified as `SPECIFIED`, `PARTIALLY_SPECIFIED`, or `UNSPECIFIED`
  30. 3. **Honest uncertainty** — unspecified choices are flagged with `[UNSPECIFIED]` comments at the exact line where the choice is made, with common alternatives listed
  31. 4. **Appendix mining** — appendices, footnotes, and figure captions are treated as first-class sources, not ignored
  32. The result: code you can trust because you can verify every decision against the paper.
  33. ---
  34. ## Install
  35. ```bash
  36. npx skills add PrathamLearnsToCode/paper2code/skills/paper2code
  37. ```
  38. You'll be prompted to:
  39. 1. **Select agents** — pick the coding agents you want to use this skill with (e.g., Claude Code)
  40. 2. **Choose scope** — Global (recommended) or project-level
  41. 3. **Choose method** — Symlink (recommended) or copy
  42. Once installed, open your agent and run the skill:
  43. ```bash
  44. claude # or your preferred agent
  45. ```
  46. ---
  47. ## Usage
  48. ### Basic — generate a minimal implementation
  49. ```
  50. /paper2code https://arxiv.org/abs/1706.03762
  51. ```
  52. ### Specify framework
  53. ```
  54. /paper2code https://arxiv.org/abs/2006.11239 --framework jax
  55. ```
  56. ### Full mode — includes training loop and data pipeline
  57. ```
  58. /paper2code 2106.09685 --mode full
  59. ```
  60. ### Educational mode — extra comments and pedagogical notebook
  61. ```
  62. /paper2code https://arxiv.org/abs/2010.11929 --mode educational
  63. ```
  64. ### Using bare arxiv ID
  65. ```
  66. /paper2code 1706.03762
  67. ```
  68. ---
  69. ## What you get
  70. ```
  71. attention_is_all_you_need/
  72. ├── README.md # Paper summary, contribution statement, quick-start
  73. ├── REPRODUCTION_NOTES.md # Ambiguity audit, unspecified choices, known deviations
  74. ├── requirements.txt # Pinned dependencies
  75. ├── src/
  76. │ ├── model.py # Architecture — every layer cited to paper section
  77. │ ├── loss.py # Loss functions with equation references
  78. │ ├── data.py # Dataset class skeleton with preprocessing TODOs
  79. │ ├── train.py # Training loop (if in scope)
  80. │ ├── evaluate.py # Metric computation code
  81. │ └── utils.py # Shared utilities (masking, positional encoding, etc.)
  82. ├── configs/
  83. │ └── base.yaml # All hyperparams — each one cited or flagged [UNSPECIFIED]
  84. └── notebooks/
  85. └── walkthrough.ipynb # Pedagogical notebook linking paper sections → code → sanity checks
  86. ```
  87. ### Key files explained
  88. | File | Purpose |
  89. |------|---------|
  90. | `model.py` | Architecture only. Each class maps to a paper section. Variable names match paper notation. |
  91. | `REPRODUCTION_NOTES.md` | The ambiguity audit. Lists every choice, whether the paper specified it, and what alternatives exist. |
  92. | `base.yaml` | Single source of truth for all hyperparameters. |
  93. | `walkthrough.ipynb` | Runnable on CPU with toy dimensions. Quotes paper passages, shows corresponding code, runs shape checks. |
  94. ---
  95. ## What this skill will NOT do
  96. - **Won't guarantee correctness.** The implementation matches what the paper describes. If the paper is wrong, the code is wrong. If the paper is vague, the code flags it.
  97. - **Won't invent details.** If the paper doesn't specify a hyperparameter, the code uses a common default and marks it `[UNSPECIFIED]`. It will never silently fill in gaps.
  98. - **Won't download datasets.** The `data.py` provides a `Dataset` class skeleton with clear instructions on where to get the data and how to preprocess it.
  99. - **Won't set up training infrastructure.** No distributed training, no experiment tracking, no checkpointing beyond what the paper's contribution requires.
  100. - **Won't implement baselines.** Only the core contribution of the paper is implemented.
  101. - **Won't reimplement standard components.** If the paper says "standard transformer encoder," the code imports it or notes the dependency — it doesn't reimplement attention from scratch.
  102. ---
  103. ## Design principles
  104. ### Citation anchoring convention
  105. Every non-trivial code decision is anchored to the paper:
  106. ```python
  107. # §3.2 — "We apply layer normalization before each sub-layer" (Pre-LN variant)
  108. class TransformerBlock(nn.Module):
  109. def forward(self, x):
  110. # §3.2, Eq. 2 — attention_weights = softmax(QK^T / sqrt(d_k))
  111. attn_out = self.attention(self.norm1(x)) # (batch, seq_len, d_model)
  112. x = x + attn_out # §3.2 — residual connection
  113. ```
  114. ### The UNSPECIFIED flag system
  115. ```python
  116. # [UNSPECIFIED] Paper does not state epsilon for LayerNorm — using 1e-6 (common default)
  117. # Alternatives: 1e-5 (PyTorch default), 1e-8 (some implementations)
  118. self.norm = nn.LayerNorm(d_model, eps=1e-6)
  119. ```
  120. ```python
  121. # [ASSUMPTION] Using pre-norm based on "we found pre-norm more stable" in §4.1
  122. # The paper uses post-norm in Figure 1 but pre-norm in experiments — ambiguous
  123. ```
  124. ### Ambiguity classification
  125. | Tag | Meaning |
  126. |-----|---------|
  127. | `§X.Y` | Directly specified in paper section X.Y |
  128. | `§X.Y, Eq. N` | Implements equation N from section X.Y |
  129. | `[UNSPECIFIED]` | Paper does not state this — our choice with alternatives listed |
  130. | `[PARTIALLY_SPECIFIED]` | Paper mentions this but is ambiguous — quote included |
  131. | `[ASSUMPTION]` | Reasonable inference from paper context — reasoning explained |
  132. | `[FROM_OFFICIAL_CODE]` | Taken from the authors' official implementation |
  133. ---
  134. ## Contributing
  135. ### Adding worked examples
  136. Worked examples are the most trust-building part of this project. To add one:
  137. 1. Pick a well-known paper (people should be able to verify the output)
  138. 2. Run the skill: `/paper2code https://arxiv.org/abs/XXXX.XXXXX`
  139. 3. Save the full output to `skills/paper2code/worked/{paper_slug}/`
  140. 4. Write a `review.md` that honestly evaluates:
  141. - What the skill got right
  142. - What it correctly flagged as unspecified
  143. - Any mistakes it made
  144. - Any edge cases it handled well or poorly
  145. 5. Submit a PR with all of the above
  146. ### Improving guardrails
  147. If you find a pattern where the skill hallucinates or makes a silent assumption, add it to the appropriate file in `guardrails/`.
  148. ### Adding domain knowledge
  149. If papers in your subfield consistently reference components that the skill doesn't know about (e.g., graph neural network primitives, RL components), add a knowledge file in `knowledge/`.
  150. ---
  151. ## Worked examples
  152. This repo includes fully worked examples to demonstrate output quality:
  153. | Paper | Type | Command |
  154. |-------|------|---------|
  155. | Attention Is All You Need (1706.03762) | Architecture | `/paper2code https://arxiv.org/abs/1706.03762` |
  156. | DDPM (2006.11239) | Training method | `/paper2code https://arxiv.org/abs/2006.11239` |
  157. Each includes the complete generated output plus an honest `review.md` evaluating what the skill got right and wrong.
  158. ---
  159. ## Implement the paper yourself
  160. paper2code writes the implementation for you. If you would rather write it yourself, [TensorTonic](https://www.tensortonic.com/research?ref=paper2code) breaks famous papers such as the Transformer, ResNet, BERT and diffusion models into small coding problems that are graded in the browser.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.