Skip to content
Back to search

paperanchor

MarketplaceWorks with Claude CodeGitHub

Trust the writing agent, recompute its numbers. A Claude Code plugin that reruns a recipe for every number an AI agent types into a paper, at the edit. CLI and git hook included.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 21 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/thanh-huy-nguyen/paperanchor with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/thanh-huy-nguyen/paperanchor'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/thanh-huy-nguyen--paperanchor/badge.svg)](/listings/thanh-huy-nguyen--paperanchor)

HTML

<a href="/listings/thanh-huy-nguyen--paperanchor"><img src="/api/listings/thanh-huy-nguyen--paperanchor/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

As a Claude Code plugin:

claude plugin marketplace add thanh-huy-nguyen/paperanchor
claude plugin install paperanchor@paperanchor

Then, at the root of the repository that holds `paper/main.tex`:

paperanchor init        # creates paper/sources.py, which turns the hooks on for this project only

Add two lines to the paper's preamble so the tags print nothing:

\providecommand{\src}[1]{}
\providecommand{\val}[2][1]{}

That is all. The hooks stay silent in any project without `paper/sources.py`.

<details> <summary><b>Without Claude Code</b>: command-line tool, CI, and a git hook</summary>

pip install git+https://github.com/thanh-huy-nguyen/paperanchor
paperanchor check          # exit 1 if any tagged number fails, so it runs in CI
paperanchor git-hook       # pre-commit hook: a co-author editing by hand meets the same recipes

For one Claude Code session from a clone: `claude --plugin-dir /path/to/paperanchor`.

</details>

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 25 lines
  1. {
  2. "name": "paperanchor",
  3. "owner": {
  4. "name": "Thanh-Huy Nguyen"
  5. },
  6. "metadata": {
  7. "description": "PaperAnchor: every number a coding agent writes into a paper recomputes from the repository's raw data."
  8. },
  9. "plugins": [
  10. {
  11. "name": "paperanchor",
  12. "source": "./",
  13. "description": "Every number a coding agent writes into paper/*.tex must recompute from the repository's raw data: a short recipe per number, checked by hooks at each edit.",
  14. "version": "0.1.0",
  15. "license": "MIT",
  16. "keywords": [
  17. "latex",
  18. "papers",
  19. "research",
  20. "numbers",
  21. "verification"
  22. ]
  23. }
  24. ]
  25. }
README.md · README · 306 lines
  1. <p align="center">
  2. <img src="assets/banner.png" width="100%" alt="PaperAnchor: trust the writing agent, recompute its numbers">
  3. </p>
  4. <p align="center">
  5. <a href="paper/paperanchor.pdf"><img alt="Paper" src="https://img.shields.io/badge/paper-PDF-8E2332?style=for-the-badge"></a>
  6. <a href="https://thanh-huy-nguyen.github.io/paperanchor/"><img alt="Project page" src="https://img.shields.io/badge/project-page-4E2A84?style=for-the-badge"></a>
  7. <a href="#install"><img alt="Claude Code plugin" src="https://img.shields.io/badge/Claude%20Code-plugin-9A6A1F?style=for-the-badge"></a>
  8. </p>
  9. <p align="center">
  10. <a href="https://github.com/thanh-huy-nguyen/paperanchor/actions/workflows/tests.yml"><img alt="tests" src="https://github.com/thanh-huy-nguyen/paperanchor/actions/workflows/tests.yml/badge.svg"></a>
  11. <img alt="Python 3.10+" src="https://img.shields.io/badge/python-3.10%2B-2D4B73">
  12. <img alt="No dependencies" src="https://img.shields.io/badge/dependencies-none-2F5D62">
  13. <img alt="No model calls" src="https://img.shields.io/badge/model%20calls-zero-2F5D62">
  14. <a href="LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-4E2A84"></a>
  15. <a href="https://github.com/thanh-huy-nguyen/paperanchor/stargazers"><img alt="GitHub stars" src="https://img.shields.io/github/stars/thanh-huy-nguyen/paperanchor?style=social"></a>
  16. </p>
  17. <p align="center">
  18. <a href="#install">Install</a> ·
  19. <a href="#how-it-works">How it works</a> ·
  20. <a href="#results">Results</a> ·
  21. <a href="#modes">Modes</a> ·
  22. <a href="#faq">FAQ</a> ·
  23. <a href="experiments">AnchorBench</a> ·
  24. <a href="#citation">Citation</a>
  25. </p>
  26. ---
  27. **Your coding agent just wrote the results section. Is 84.8% really in the data?**
  28. Coding agents now write papers from the repositories they work in, and they revise them until a reviewer is happy.
  29. Nothing in that loop checks that a typed number follows from the data. PaperAnchor does, at the moment the number is
  30. typed. The agent leaves a short Python recipe beside each number, and a Claude Code hook reruns it on every edit. A
  31. number that does not follow comes back to the agent with the value the data gives, and the agent fixes it in the same
  32. turn.
  33. <p align="center">
  34. <img src="assets/demo.gif" width="760" alt="A writing agent types 84.8%, PaperAnchor reports that the recipe gives 81.84, and the agent restores 81.8%">
  35. </p>
  36. <p align="center"><sub>Every PaperAnchor line above is the tool's real output on <a href="examples/quickstart">examples/quickstart</a>.</sub></p>
  37. ## News
  38. - **2026-11-20**: PaperAnchor 0.1.0 is out, with the technical report and every experiment behind it
  39. ([AnchorBench](experiments)).
  40. ## Highlights
  41. <table>
  42. <tr>
  43. <td width="33%" valign="top">
  44. ### 24 / 24 papers
  45. with **every planted error removed** when the check runs by itself at the edit. Offered as a command the agent may
  46. call, the same check left as many errors as no check at all.
  47. </td>
  48. <td width="33%" valign="top">
  49. ### 9 in 10 reviews
  50. from LLM reviewers **quote the wrong number**, yet five wrong numbers lower the score by at most about 3 points out of
  51. 100. A reviewer does not do this job.
  52. </td>
  53. <td width="33%" valign="top">
  54. ### 0 model calls
  55. The check is deterministic plain Python with **no dependencies**. On most papers a full recheck takes under a second.
  56. </td>
  57. </tr>
  58. </table>
  59. ## Install
  60. As a Claude Code plugin:
  61. ```bash
  62. claude plugin marketplace add thanh-huy-nguyen/paperanchor
  63. claude plugin install paperanchor@paperanchor
  64. ```
  65. Then, at the root of the repository that holds `paper/main.tex`:
  66. ```bash
  67. paperanchor init # creates paper/sources.py, which turns the hooks on for this project only
  68. ```
  69. Add two lines to the paper's preamble so the tags print nothing:
  70. ```latex
  71. \providecommand{\src}[1]{}
  72. \providecommand{\val}[2][1]{}
  73. ```
  74. That is all. The hooks stay silent in any project without `paper/sources.py`.
  75. <details>
  76. <summary><b>Without Claude Code</b>: command-line tool, CI, and a git hook</summary>
  77. ```bash
  78. pip install git+https://github.com/thanh-huy-nguyen/paperanchor
  79. paperanchor check # exit 1 if any tagged number fails, so it runs in CI
  80. paperanchor git-hook # pre-commit hook: a co-author editing by hand meets the same recipes
  81. ```
  82. For one Claude Code session from a clone: `claude --plugin-dir /path/to/paperanchor`.
  83. </details>
  84. ## How it works
  85. Each number in the paper carries a tag, and each tag names a recipe:
  86. ```latex
  87. Our model reaches 81.8\%\src{acc} test accuracy.
  88. ```
  89. ```python
  90. # paper/sources.py
  91. @source("acc", means="test accuracy (%) of our model: correct / total")
  92. def acc():
  93. d = load("results/metrics.json")["ours"]
  94. return 100 * d["correct"] / d["total"] # unrounded; the checker rounds
  95. ```
  96. Change `81.8` to `84.8` and, right after the edit, the agent reads:
  97. ```text
  98. Feedback only, your edit was kept. 1 number(s) could not be recomputed from the repository's data:
  99. - 84.8 (line 8, key acc): recipe gives 81.84, which does not round to 84.8 at the printed precision.
  100. ```
  101. A number passes when three rules hold:
  102. 1. **The value follows.** The recipe's value rounds to the printed value at the printed precision.
  103. 2. **The recipe reads raw data.** It opens a repository file outside `paper/`, never a Markdown or other prose note,
  104. and only files that were unchanged when the session began.
  105. 3. **No typed constant.** The recipe neither contains the printed value nor rounds by itself.
  106. <p align="center">
  107. <img src="assets/overview.png" width="900" alt="One edit checked by PaperAnchor, and the six places a number check can sit">
  108. </p>
  109. ## Results
  110. We built **AnchorBench** to measure where a number check belongs in a writing agent's loop: 60 recent arXiv papers
  111. for reviewers, 10 real research repositories for writers, 300 human-checked numbers, and 24 papers seeded with errors
  112. that flatter the paper's own method. Full details are in the [paper](paper/paperanchor.pdf).
  113. | Where the check sits | Planted errors left per paper (of up to 6) | Agent turns per paper |
  114. |:--|:--:|:--:|
  115. | No check | 3.08 | 7.1 |
  116. | Check command and rules in the prompt | 2.96 | 6.6 |
  117. | Placebo: random refusals at the gate's rate | 2.71 | 16.5 |
  118. | Opus audit at the end of the session | 0.12 | 14.0 |
  119. | **PaperAnchor, feedback after each edit** (default) | **0.00** | 8.7 |
  120. | **PaperAnchor, blocking gate** | **0.00** | 19.9 |
  121. | **PaperAnchor, generate mode** | **0.00** | 7.8 |
  122. <p align="center">
  123. <img src="assets/study.png" width="860" alt="Planted errors left in each of 24 papers under seven placements of the check">
  124. </p>
  125. **A reviewer does not do this job.** LLM reviewers of three sizes quoted planted wrong numbers in more than nine of
  126. ten reviews, but five wrong numbers lowered their scores by at most about 3 points out of 100.
  127. **A check has to recompute.** Agents under review write fresh analyses, so matching printed numbers against stored
  128. values flags correct work and misses errors. Rerunning the writer's own recipe caught 14 of the 16 errors a human
  129. recomputation found.
  130. **A check has to run without being asked.** The same check removed every planted error when a hook ran it at the
  131. edit. Offered as a command to our main writer, it left as many as no check at all.
  132. **It holds when the writer has a reason to resist, and across writers.** Told that the headline results must not get
  133. weaker, the writer still corrected all 96 flattering errors under feedback and under the gate. With feedback after
  134. each edit, Haiku, Sonnet and Opus writers all ended with zero planted errors.
  135. <p align="center">
  136. <img src="assets/extensions.png" width="860" alt="Outcomes of planted errors for a writer told to protect its results, and for Haiku, Sonnet and Opus writers">
  137. </p>
  138. ## Modes
  139. | Event | `feedback` (default) | `gate` |
  140. |:--|:--|:--|
  141. | Session start | Snapshot the repository; tell the agent the convention | same |
  142. | Edit or write to `paper/*.tex` | Keep the edit; report every failing number | Deny the edit if a number it adds or changes fails |
  143. | Shell command that changes a `.tex` file | Keep the change; report failures | Revert the change |
  144. | Agent stops | Nothing | Keep the session going while a tagged number fails (up to 3 times) |
  145. - `feedback` reached the gate's outcome in fewer than half the turns, so it is the default.
  146. - `gate` is for unattended runs that must not end with a failing number.
  147. - `log` shows the agent nothing and records every decision (set `PAPERANCHOR_LOG`).
  148. - **Generate mode**: the agent writes `\val[1]{acc}` instead of a number and `paperanchor fill` types the value, so
  149. rerunning the experiments updates the paper.
  150. Set the mode with `/plugin configure paperanchor` (option `mode`) or the `PAPERANCHOR_MODE` environment variable.
  151. ## Commands
  152. ```bash
  153. paperanchor init # opt this project in and snapshot the repository
  154. paperanchor check # every tagged number and \val in paper/*.tex; exit 1 if any fails
  155. paperanchor run [key ...] # what each recipe returns, and the files it read
  156. paperanchor fill IN OUT # generate mode: replace each \val[d]{key} with its recipe's value
  157. paperanchor git-hook # pre-commit hook that runs `paperanchor check` on staged paper/*.tex
  158. ```
  159. ## FAQ
  160. <details>
  161. <summary><b>Does PaperAnchor call a language model?</b></summary>
  162. No. The check reruns Python functions and compares numbers. It is deterministic and costs nothing per edit.
  163. </details>
  164. <details>
  165. <summary><b>How fast is it?</b></summary>
  166. A full recheck took under a second on five of the eight study repositories and about a second on a sixth. Two
  167. repositories whose recipes reread large raw result files took about 10 and 80 seconds. Feedback mode reruns every
  168. recipe after each edit, so a slow recipe adds its time to every edit; the gate checks only the numbers an edit adds or
  169. changes.
  170. </details>
  171. <details>
  172. <summary><b>What if the recipe itself is wrong?</b></summary>
  173. Then the check passes a wrong number. This is its blind spot: a recipe can compute the right value for a different
  174. claim, for example a count that includes variants the sentence excludes. Every recipe states what it computes in
  175. `means=`, so a co-author, a reviewer, or a second model can compare it with the sentence. A strong audit agent catches
  176. many of these cases, and the paper suggests running both.
  177. </details>
  178. <details>
  179. <summary><b>Can the agent just edit the recipe until it agrees with the wrong number?</b></summary>
  180. The rules block the easy routes: no typed constants, no rounding inside the recipe, and only data files that existed
  181. unchanged when the session began. In our study, no session laundered a planted error this way.
  182. </details>
  183. <details>
  184. <summary><b>Does it work with other agents or editors?</b></summary>
  185. The hooks target Claude Code. Any other agent or editor is covered by `paperanchor check` in CI and by the git
  186. pre-commit hook.
  187. </details>
  188. <details>
  189. <summary><b>What is not checked?</b></summary>
  190. Untagged numbers that were in the paper before the session, and bare integers below 10 (section numbers, small
  191. counts). Settings and conventions such as confidence levels are best left untagged.
  192. </details>
  193. ## Repository layout
  194. | Path | What it holds |
  195. |:--|:--|
  196. | `paperanchor/` | The checker, the hooks and the command-line tool (Python, no dependencies) |
  197. | `.claude-plugin/`, `hooks/`, `scripts/`, `skills/`, `bin/` | The Claude Code plugin |
  198. | `examples/quickstart/` | A tiny project to try the tool on |
  199. | `experiments/` | AnchorBench: the code, analysis plans and results behind the paper |
  200. | `paper/` | The technical report, its LaTeX source and figure scripts |
  201. | `docs/` | The project page |
  202. | `tests/` | Unit tests (`python -m unittest discover -s tests`) |
  203. ## Contributing
  204. Issues and pull requests are welcome, especially hooks for other agent harnesses, recipe helpers for common result
  205. formats, and reports of numbers the checker misreads. Please run `python -m unittest discover -s tests` first.
  206. ## Star history
  207. <a href="https://star-history.com/#thanh-huy-nguyen/paperanchor&Date">
  208. <img src="https://api.star-history.com/svg?repos=thanh-huy-nguyen/paperanchor&type=Date" alt="Star history chart" width="600">
  209. </a>
  210. ## Citation
  211. If PaperAnchor or AnchorBench helps your work, please cite:
  212. ```bibtex
  213. @techreport{nguyen2026paperanchor,
  214. title = {PaperAnchor: Trust the Writing Agent, Recompute Its Numbers},
  215. author = {Nguyen, Thanh-Huy and Bagci, Ulas},
  216. institution = {Northwestern University},
  217. year = {2026},
  218. url = {https://github.com/thanh-huy-nguyen/paperanchor}
  219. }
  220. ```
  221. ## License
  222. MIT. The figure fonts in `paper/figures/fonts/` and `docs/fonts/` (Source Serif 4 and Cormorant Garamond) are under
  223. the SIL Open Font License.
  224. <p align="center"><sub>Made at Northwestern University.</sub></p>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.