Skip to content
Back to search

paperbanana

Open source implementation and extension of Google Research’s PaperBanana for automated academic figures, diagrams, and research visuals, expanded to new domains like slide generation.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/llmsresearch--paperbanana/badge.svg)](/listings/llmsresearch--paperbanana)

HTML

<a href="/listings/llmsresearch--paperbanana"><img src="/api/listings/llmsresearch--paperbanana/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

> **Try it in your browser:** the > [Colab quickstart notebook](https://colab.research.google.com/github/llmsresearch/paperbanana/blob/main/notebooks/PaperBanana_Colab_Quickstart.ipynb) > walks through install → API key → diagram generation end-to-end, no local setup required.

Prerequisites

- Python 3.10+ - An OpenAI API key ([platform.openai.com](https://platform.openai.com/api-keys)) or Azure OpenAI / Foundry endpoint - Or a Google Gemini API key (free, [Google AI Studio](https://makersuite.google.com/app/apikey))

Step 1: Install

pip install paperbanana

Or install from source for development:

git clone https://github.com/llmsresearch/paperbanana.git
cd paperbanana
pip install -e ".[dev,openai,google]"

Docker

Build the image from a clone of the repo and pass your API key at runtime:

docker build -t paperbanana .
docker run --rm -e GOOGLE_API_KEY paperbanana generate --help

To generate a diagram, mount your input and an outputs folder into `/work`:

docker run --rm -e GOOGLE_API_KEY \
  -v "$(pwd)/method.txt:/work/method.txt:ro" \
  -v "$(pwd)/outputs:/work/outputs" \
  paperbanana generate --input method.txt --caption "Overview of our framework"

Step 2: Get Your API Key

cp .env.example .env
# Edit .env and add your API key:
#   OPENAI_API_KEY=your-key-here
#   GOOGLE_API_KEY=your-key-here
#
# For Azure OpenAI / Foundry:
#   OPENAI_BASE_URL=https://<resource>.openai.azure.com/openai/v1
#
# Optional Gemini overrides:
#   GOOGLE_BASE_URL=https://your-gemini-proxy.example.com
#   GOOGLE_VLM_MODEL=gemini-2.5-flash
#   GOOGLE_IMAGE_MODEL=gemini-3-pro-image-preview

Or use the setup wizard for Gemini:

paperbanana setup

Step 3: Generate a Diagram

paperbanana generate \
  --input examples/sample_inputs/transformer_method.txt \
  --caption "Overview of our encoder-decoder architecture with sparse routing"

With input optimization and auto-refine:

paperbanana generate \
  --input my_method.txt \
  --caption "Overview of our encoder-decoder framework" \
  --optimize --auto

Output is saved to `outputs/run_<timestamp>/final_output.png` along with all intermediate iterations and metadata.

PaperBanana Studio (local web UI)

Install the optional Gradio dependency, then start the app:

pip install 'paperbanana[studio]'
paperbanana studio

Open the URL shown in the terminal (default `http://127.0.0.1:7860/`). The Studio exposes the same workflows as the CLI: methodology diagrams, statistical plots, comparative evaluation, continuing a prior run, batch manifests (methodology or **plot** batch via the Batch tab), and a simple browser for `run_*` / `batch_*` output folders. Use `--host`, `--port`, `--config`, and `--output-dir` as needed.

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 856 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <!-- mcp-name: io.github.llmsresearch/paperbanana -->
  2. <table align="center" width="100%" style="border: none; border-collapse: collapse;">
  3. <tr>
  4. <td width="220" align="left" valign="middle" style="border: none;">
  5. <img src="https://dwzhu-pku.github.io/PaperBanana/static/images/logo.jpg" alt="PaperBanana Logo" width="180"/>
  6. </td>
  7. <td align="left" valign="middle" style="border: none;">
  8. <h1>PaperBanana</h1>
  9. <p><strong>Automated Academic Illustration for AI Scientists</strong></p>
  10. <p>
  11. <a href="https://github.com/llmsresearch/paperbanana/actions/workflows/ci.yml"><img src="https://github.com/llmsresearch/paperbanana/actions/workflows/ci.yml/badge.svg" alt="CI"/></a>
  12. <a href="https://pypi.org/project/paperbanana/"><img src="https://img.shields.io/pypi/dm/paperbanana?label=PyPI%20downloads&logo=pypi&logoColor=white" alt="PyPI Downloads"/></a>
  13. <a href="https://huggingface.co/spaces/llmsresearch/paperbanana"><img src="https://img.shields.io/badge/Demo-HuggingFace-yellow?logo=huggingface&logoColor=white" alt="Demo"/></a>
  14. <a href="https://colab.research.google.com/github/llmsresearch/paperbanana/blob/main/notebooks/PaperBanana_Colab_Quickstart.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab"/></a>
  15. <br/>
  16. <a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.10%2B-blue?logo=python&logoColor=white" alt="Python 3.10+"/></a>
  17. <a href="https://arxiv.org/abs/2601.23265"><img src="https://img.shields.io/badge/arXiv-2601.23265-b31b1b?logo=arxiv&logoColor=white" alt="arXiv"/></a>
  18. <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green?logo=opensourceinitiative&logoColor=white" alt="License: MIT"/></a>
  19. <br/>
  20. <a href="https://pydantic.dev"><img src="https://img.shields.io/badge/Pydantic-v2-e92063?logo=pydantic&logoColor=white" alt="Pydantic v2"/></a>
  21. <a href="https://typer.tiangolo.com"><img src="https://img.shields.io/badge/CLI-Typer-009688?logo=gnubash&logoColor=white" alt="Typer"/></a>
  22. <a href="https://ai.google.dev/"><img src="https://img.shields.io/badge/Gemini-Free%20Tier-4285F4?logo=google&logoColor=white" alt="Gemini Free Tier"/></a>
  23. </p>
  24. </td>
  25. </tr>
  26. </table>
  27. <p align="center">
  28. <a href="https://trendshift.io/repositories/20880?utm_source=trendshift-badge&amp;utm_medium=badge&amp;utm_campaign=badge-trendshift-20880" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/20880/daily?language=Python" alt="llmsresearch%2Fpaperbanana | Trendshift" width="250" height="55"/></a>
  29. <a href="https://trendshift.io/repositories/20880?utm_source=trendshift-badge&amp;utm_medium=badge&amp;utm_campaign=badge-trendshift-20880" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/20880/weekly?language=Python" alt="llmsresearch%2Fpaperbanana | Trendshift" width="250" height="55"/></a>
  30. </p>
  31. ---
  32. > **Disclaimer**: This is an **unofficial, community-driven open-source implementation** of the paper
  33. > *"PaperBanana: Automating Academic Illustration for AI Scientists"* by Dawei Zhu, Rui Meng, Yale Song,
  34. > Xiyu Wei, Sujian Li, Tomas Pfister, and Jinsung Yoon ([arXiv:2601.23265](https://arxiv.org/abs/2601.23265)).
  35. > This project is **not affiliated with or endorsed by** the original authors or Google Research.
  36. > The implementation is based on the publicly available paper and may differ from the original system.
  37. An agentic framework for generating publication-quality academic diagrams and statistical plots from text descriptions. Supports OpenAI (GPT-5.2 + GPT-Image-1.5), Azure OpenAI / Foundry, Google Gemini, and Atlas Cloud providers.
  38. - Two-phase multi-agent pipeline with iterative refinement
  39. - Multiple VLM and image generation providers (OpenAI, Azure, Gemini, Atlas Cloud)
  40. - Input optimization layer for better generation quality
  41. - Auto-refine mode and run continuation with user feedback
  42. - CLI, Python API, and MCP server for IDE integration
  43. - **Batch generation** from a manifest file (YAML/JSON) for multiple diagrams in one run
  44. - **Batch plots** — `paperbanana plot-batch` runs many statistical plots from one manifest (CSV/JSON per item)
  45. - **PDF inputs** for methodology context (optional `paperbanana[pdf]` / PyMuPDF), with per-page selection
  46. - **PaperBanana Studio** — local Gradio web UI (`paperbanana studio`) for diagrams, plots, evaluation, batch, and run browser
  47. - Claude Code skills for `/generate-diagram`, `/generate-plot`, and `/evaluate-diagram`
  48. <p align="center">
  49. <img src="assets/img/hero_image.png" alt="PaperBanana takes paper as input and provide diagram as output" style="max-width: 960px; width: 100%; height: auto;"/>
  50. </p>
  51. **Learning LLM concepts?** We also publish [211 free visual cards](https://github.com/llmsresearch/llm-flashcards)
  52. on attention, RAG, agents, and inference.
  53. [Read them online](https://llmsresearch.com/cards?utm_source=github&utm_medium=referral&utm_campaign=visual_library&utm_content=paperbanana-readme).
  54. ## Atlas Cloud
  55. <p align="center">
  56. <img src="assets/sponsors/atlas_cloud_logo.png" alt="Atlas Cloud Logo" width="180"/>
  57. </p>
  58. Atlas Cloud is a full-modal AI inference platform that gives developers a single AI API to access video generation, image generation, and LLM APIs. Instead of managing multiple vendor integrations, you connect once and get unified access to 300+ curated models across all modalities.
  59. Check out Atlas Cloud's new coding plan promotion for more budget-friendly API access:
  60. [https://www.atlascloud.ai/console/coding-plan](https://www.atlascloud.ai/console/coding-plan?utm_source=github&utm_medium=link&utm_campaign=paperbanana)
  61. ---
  62. ## Quick Start
  63. > **Try it in your browser:** the
  64. > [Colab quickstart notebook](https://colab.research.google.com/github/llmsresearch/paperbanana/blob/main/notebooks/PaperBanana_Colab_Quickstart.ipynb)
  65. > walks through install → API key → diagram generation end-to-end, no local setup required.
  66. ### Prerequisites
  67. - Python 3.10+
  68. - An OpenAI API key ([platform.openai.com](https://platform.openai.com/api-keys)) or Azure OpenAI / Foundry endpoint
  69. - Or a Google Gemini API key (free, [Google AI Studio](https://makersuite.google.com/app/apikey))
  70. ### Step 1: Install
  71. ```bash
  72. pip install paperbanana
  73. ```
  74. Or install from source for development:
  75. ```bash
  76. git clone https://github.com/llmsresearch/paperbanana.git
  77. cd paperbanana
  78. pip install -e ".[dev,openai,google]"
  79. ```
  80. #### Docker
  81. Build the image from a clone of the repo and pass your API key at runtime:
  82. ```bash
  83. docker build -t paperbanana .
  84. docker run --rm -e GOOGLE_API_KEY paperbanana generate --help
  85. ```
  86. To generate a diagram, mount your input and an outputs folder into `/work`:
  87. ```bash
  88. docker run --rm -e GOOGLE_API_KEY \
  89. -v "$(pwd)/method.txt:/work/method.txt:ro" \
  90. -v "$(pwd)/outputs:/work/outputs" \
  91. paperbanana generate --input method.txt --caption "Overview of our framework"
  92. ```
  93. ### Step 2: Get Your API Key
  94. ```bash
  95. cp .env.example .env
  96. # Edit .env and add your API key:
  97. # OPENAI_API_KEY=your-key-here
  98. # GOOGLE_API_KEY=your-key-here
  99. #
  100. # For Azure OpenAI / Foundry:
  101. # OPENAI_BASE_URL=https://<resource>.openai.azure.com/openai/v1
  102. #
  103. # Optional Gemini overrides:
  104. # GOOGLE_BASE_URL=https://your-gemini-proxy.example.com
  105. # GOOGLE_VLM_MODEL=gemini-2.5-flash
  106. # GOOGLE_IMAGE_MODEL=gemini-3-pro-image-preview
  107. ```
  108. Or use the setup wizard for Gemini:
  109. ```bash
  110. paperbanana setup
  111. ```
  112. ### Step 3: Generate a Diagram
  113. ```bash
  114. paperbanana generate \
  115. --input examples/sample_inputs/transformer_method.txt \
  116. --caption "Overview of our encoder-decoder architecture with sparse routing"
  117. ```
  118. With input optimization and auto-refine:
  119. ```bash
  120. paperbanana generate \
  121. --input my_method.txt \
  122. --caption "Overview of our encoder-decoder framework" \
  123. --optimize --auto
  124. ```
  125. Output is saved to `outputs/run_<timestamp>/final_output.png` along with all intermediate iterations and metadata.
  126. ### PaperBanana Studio (local web UI)
  127. Install the optional Gradio dependency, then start the app:
  128. ```bash
  129. pip install 'paperbanana[studio]'
  130. paperbanana studio
  131. ```
  132. Open the URL shown in the terminal (default `http://127.0.0.1:7860/`). The Studio exposes the same workflows as the CLI: methodology diagrams, statistical plots, comparative evaluation, continuing a prior run, batch manifests (methodology or **plot** batch via the Batch tab), and a simple browser for `run_*` / `batch_*` output folders. Use `--host`, `--port`, `--config`, and `--output-dir` as needed.
  133. ---
  134. ## How It Works
  135. PaperBanana implements a multi-agent pipeline with up to 7 specialized agents:
  136. **Phase 0 -- Input Optimization (optional, `--optimize`):**
  137. 0. **Input Optimizer** runs two parallel VLM calls:
  138. - **Context Enricher** structures raw methodology text into diagram-ready format (components, flows, groupings, I/O)
  139. - **Caption Sharpener** transforms vague captions into precise visual specifications
  140. **Phase 1 -- Linear Planning:**
  141. 1. **Retriever** selects the most relevant reference examples from a curated set of 13 methodology diagrams spanning agent/reasoning, vision/perception, generative/learning, and science/applications domains
  142. 2. **Planner** generates a detailed textual description of the target diagram via in-context learning from the retrieved examples
  143. 3. **Stylist** refines the description for visual aesthetics using NeurIPS-style guidelines (color palette, layout, typography)
  144. **Phase 2 -- Iterative Refinement:**
  145. 4. **Visualizer** renders the description into an image
  146. 5. **Critic** evaluates the generated image against the source context and provides a revised description addressing any issues
  147. 6. Steps 4-5 repeat for a fixed number of iterations (default 3), or until the critic is satisfied (`--auto`)
  148. ## Providers
  149. PaperBanana supports multiple VLM and image generation providers:
  150. | Component | Provider | Model | Notes |
  151. |-----------|----------|-------|-------|
  152. | VLM (planning, critique) | OpenAI | `gpt-5.2` | Default |
  153. | Image Generation | OpenAI | `gpt-image-1.5` | Default |
  154. | VLM | Atlas Cloud | `deepseek-ai/DeepSeek-V3-0324` | OpenAI-compatible chat endpoint |
  155. | Image Generation | Atlas Cloud | `openai/gpt-image-2/text-to-image` | Async prediction API |
  156. | VLM | Google Gemini | `gemini-2.5-flash` | Low cost |
  157. | Image Generation | Google Gemini | `gemini-3-pro-image-preview` | $0.134/image (1K) |
  158. | VLM / Image | OpenRouter | Any supported model | Flexible routing |
  159. Azure OpenAI / Foundry endpoints are auto-detected — set `OPENAI_BASE_URL` to your endpoint.
  160. Gemini-compatible gateways are also supported — set `GOOGLE_BASE_URL` when needed.
  161. Atlas Cloud uses `ATLASCLOUD_BASE_URL=https://api.atlascloud.ai/v1` for chat and `ATLASCLOUD_IMAGE_BASE_URL=https://api.atlascloud.ai/api/v1` for image generation.
  162. Atlas Cloud official site:
  163. [https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=paperbanana](https://www.atlascloud.ai/?utm_source=github&utm_medium=link&utm_campaign=paperbanana)
  164. Recommended Atlas LLM models for `ATLASCLOUD_VLM_MODEL`:
  165. - `deepseek-ai/DeepSeek-V3-0324` (default)
  166. - `openai/gpt-4o`
  167. - `openai/gpt-4.1`
  168. - `google/gemini-2.5-flash`
  169. - `anthropic/claude-sonnet-4.5-20250929`
  170. These are stable, generally available models verified against the Atlas Cloud API. The full, always-current model pool (300+ models) is documented on Atlas Cloud's own docs — see [https://www.atlascloud.ai/models](https://www.atlascloud.ai/models?utm_source=github&utm_medium=link&utm_campaign=paperbanana) — and any model id listed there can be passed via `ATLASCLOUD_VLM_MODEL`.
  171. Recommended Atlas image models for `ATLASCLOUD_IMAGE_MODEL`:
  172. - `openai/gpt-image-2/text-to-image`
  173. - `openai/gpt-image-2/edit`
  174. - `baidu/ERNIE-Image-Turbo/text-to-image`
  175. - `black-forest-labs/flux-dev`
  176. - `black-forest-labs/flux-schnell`
  177. - `qwen/qwen-image`
  178. ---
  179. ## CLI Reference
  180. ### `paperbanana generate` -- Methodology Diagrams
  181. ```bash
  182. # Basic generation
  183. paperbanana generate \
  184. --input method.txt \
  185. --caption "Overview of our framework"
  186. # With input optimization and auto-refine
  187. paperbanana generate \
  188. --input method.txt \
  189. --caption "Overview of our framework" \
  190. --optimize --auto
  191. # Continue the latest run with user feedback
  192. paperbanana generate --continue \
  193. --feedback "Make arrows thicker and colors more distinct"
  194. # Continue a specific run
  195. paperbanana generate --continue-run run_20260218_125448_e7b876 \
  196. --iterations 3
  197. # PDF as input (install PyMuPDF: pip install 'paperbanana[pdf]')
  198. paperbanana generate \
  199. --input paper.pdf \
  200. --caption "Overview of our method" \
  201. --pdf-pages "3-8"
  202. # Guide generation with a reference/sketch image (repeatable)
  203. paperbanana generate \
  204. --input method.txt \
  205. --caption "Overview of our framework" \
  206. --image sketch.png --image prior_figure.png
  207. ```
  208. | Flag | Short | Description |
  209. |------|-------|-------------|
  210. | `--input` | `-i` | Path to methodology text file or PDF (required for new runs) |
  211. | `--caption` | `-c` | Figure caption / communicative intent (required for new runs) |
  212. | `--image` | | Reference/sketch image (hand-drawn sketch, whiteboard photo, prior figure) that guides the Planner. Repeatable for multiple images |
  213. | `--output` | `-o` | Output image path (default: auto-generated in `outputs/`) |
  214. | `--iterations` | `-n` | Number of Visualizer-Critic refinement rounds (default: 3) |
  215. | `--num-candidates` | `-k` | Generate N candidate images in parallel, 1-8 (default: 1). Planning runs once; refinement fans out per candidate with seed offsets. Outputs land in `candidates/cand_<i>/`; the run-root `final_output` is candidate 1. Cost estimates and `--budget` account for the fan-out |
  216. | `--auto` | | Loop until critic is satisfied (with `--max-iterations` safety cap) |
  217. | `--max-iterations` | | Safety cap for `--auto` mode (default: 30) |
  218. | `--optimize` | | Preprocess inputs with parallel context enrichment and caption sharpening |
  219. | `--continue` | | Continue from the latest run in `outputs/` |
  220. | `--continue-run` | | Continue from a specific run ID |
  221. | `--feedback` | | User feedback for the critic when continuing a run |
  222. | `--pdf-pages` | | PDF input only: 1-based pages (e.g. `1-5`, `2,4,6-8`; default: all) |
  223. | `--vlm-provider` | | VLM provider name (default: `openai`) |
  224. | `--vlm-model` | | VLM model name (default: `gpt-5.2`) |
  225. | `--image-provider` | | Image gen provider (default: `openai_imagen`) |
  226. | `--image-model` | | Image gen model (default: `gpt-image-1.5`) |
  227. | `--format` | `-f` | Output format: `png`, `jpeg`, or `webp` (default: `png`) |
  228. | `--config` | | Path to YAML config file (see `configs/config.yaml`) |
  229. | `--verbose` | `-v` | Show detailed agent progress and timing |
  230. | `--progress-json` | | Emit JSON progress events to stdout during generation |
  231. ### `paperbanana plot` -- Statistical Plots
  232. ```bash
  233. paperbanana plot \
  234. --data results.csv \
  235. --intent "Bar chart comparing model accuracy across benchmarks"
  236. ```
  237. | Flag | Short | Description |
  238. |------|-------|-------------|
  239. | `--data` | `-d` | Path to data file, CSV or JSON (required) |
  240. | `--intent` | | Communicative intent for the plot (required) |
  241. | `--output` | `-o` | Output image path |
  242. | `--iterations` | `-n` | Refinement iterations (default: 3) |
  243. | `--vlm-provider` | | VLM provider name |
  244. | `--vlm-model` | | VLM model name |
  245. Plots are rendered via VLM-generated matplotlib code — no image-generation provider or credentials are required.
  246. ### `paperbanana venues` -- Custom Venue Style Packs
  247. `--venue` selects a *venue style pack*: a directory with `methodology_style_guide.md`, `plot_style_guide.md`, and an optional `venue.yaml`. Built-in packs (`neurips`, `icml`, `acl`, `ieee`) ship with PaperBanana; you can add your own under `~/.config/paperbanana/venues/` (override with `--venue-dir` or `PAPERBANANA_VENUE_DIR`) without touching the repo:
  248. ```bash
  249. # 1. Scaffold a pack (seeds both guides from the NeurIPS templates)
  250. paperbanana venues init mylab
  251. # 2. Edit the style guides — or generate them from a corpus of example figures:
  252. # paperbanana guidelines synthesize --reference-set ./examples \
  253. # --output ~/.config/paperbanana/venues/mylab/methodology_style_guide.md
  254. # 3. Use it anywhere --venue is accepted
  255. paperbanana generate --input method.txt --caption "Overview" --venue mylab
  256. # See everything that's available (built-in + user, with source)
  257. paperbanana venues list
  258. ```
  259. `venue.yaml` (all fields optional):
  260. ```yaml
  261. display_name: "My Lab Style" # shown by `paperbanana venues list`
  262. aspect_ratio: "16:9" # default --aspect-ratio for this venue's runs
  263. fonts: # preferred fonts, appended to the style guides
  264. - "Helvetica"
  265. ```
  266. On a name clash, built-in packs win — user packs cannot shadow built-in venues. Unknown venue names fail fast with the list of available packs from both sources.
  267. ### `paperbanana batch` -- Batch Generation
  268. Generate multiple methodology diagrams from a single manifest file (YAML or JSON). Each item runs the full pipeline; outputs are written under `outputs/batch_<id>/run_<id>/` and a `batch_report.json` summarizes all runs.
  269. ```bash
  270. paperbanana batch --manifest examples/batch_manifest.yaml --optimize
  271. ```
  272. Manifest format (YAML or JSON with an `items` list):
  273. ```yaml
  274. items:
  275. - input: path/to/method1.txt
  276. caption: "Overview of our encoder-decoder"
  277. id: fig1
  278. - input: method2.txt
  279. caption: "Training pipeline"
  280. id: fig2
  281. - input: paper.pdf
  282. caption: "System overview"
  283. id: fig3
  284. pdf_pages: "4-9" # optional; PDF inputs only
  285. ```
  286. Paths in the manifest are resolved relative to the manifest file's directory.
  287. **Composite figures:** Add an optional `composite` section to automatically stitch all generated panels into a single labeled figure after the batch completes:
  288. ```yaml
  289. composite:
  290. layout: "1x3" # rows x cols, or "auto"
  291. labels: auto # (a), (b), (c)... or explicit list, or null
  292. spacing: 20 # pixels between panels
  293. label_position: bottom # top or bottom
  294. output: "composite.png"
  295. items:
  296. - input: method_encoder.txt
  297. caption: "Encoder architecture"
  298. id: panel_a
  299. # ...
  300. ```
  301. The composite image is saved alongside the individual panels in the batch output directory. See `examples/composite_batch_manifest.yaml` for a complete example.
  302. **Generate a human-readable report** from an existing batch run (Markdown or HTML):
  303. ```bash
  304. paperbanana batch-report --batch-dir outputs/batch_20250109_123456_abc --format markdown
  305. # or by batch ID (under default output dir)
  306. paperbanana batch-report --batch-id batch_20250109_123456_abc --format html --output report.html
  307. ```
  308. Diagram batch reports include `batch_kind: methodology`; plot batches use `batch_kind: statistical_plot`. Human-readable reports (`paperbanana batch-report`) show the batch kind when present.
  309. **Sweep manifests** let you store the full sweep plan as YAML/JSON instead of eight comma-separated CLI flags. Mutually exclusive with the axis flags; see `examples/sweep_manifest.yaml`.
  310. ```bash
  311. paperbanana sweep --manifest examples/sweep_manifest.yaml
  312. ```
  313. **Sweep reports** produced by `paperbanana sweep` can be rendered the same way:
  314. ```bash
  315. paperbanana sweep-report --sweep-dir outputs/sweep_20250109_123456_abc --format html
  316. # or by sweep ID
  317. paperbanana sweep-report --sweep-id sweep_20250109_123456_abc --format markdown
  318. ```
  319. Rendered sweep reports include a summary, a top-5 ranked table, the full variants table (with per-variant provider/model, iterations, critic-suggestion count, proxy score, and output path), and the `quality_proxy_score` note. Dry-run reports render a simplified "Planned Variants" section.
  320. | Flag | Short | Description |
  321. |------|-------|-------------|
  322. | `--manifest` | `-m` | Path to manifest file (required) |
  323. | `--output-dir` | `-o` | Parent directory for batch run (default: outputs) |
  324. | `--config` | | Path to config YAML |
  325. | `--iterations` | `-n` | Refinement iterations per item |
  326. | `--optimize` | | Preprocess inputs for each item |
  327. | `--auto` | | Loop until critic satisfied per item |
  328. | `--format` | `-f` | Output image format (png, jpeg, webp) |
  329. | `--auto-download-data` | | Auto-download the PaperBananaBench reference set (~254 MB) if not cached |
  330. ### `paperbanana plot-batch` -- Batch Statistical Plots
  331. Generate multiple plots from a manifest (YAML or JSON). Each item specifies a **data** file (CSV or JSON) and an **intent** string, mirroring `paperbanana plot`. Outputs live under `outputs/batch_<id>/run_<id>/` with the same `batch_report.json` and `paperbanana batch-report` workflow as diagram batches.
  332. ```bash
  333. paperbanana plot-batch --manifest examples/plot_batch_manifest.yaml --optimize
  334. ```
  335. Manifest format (`items` list):
  336. ```yaml
  337. items:
  338. - data: path/to/results.csv
  339. intent: "Bar chart comparing accuracy across models"
  340. id: fig_acc
  341. - data: other.json
  342. intent: "Scatter plot with trend line"
  343. aspect_ratio: "16:9" # optional per item; CLI --aspect-ratio is the default when omitted
  344. ```
  345. Paths are resolved relative to the manifest file’s directory.
  346. | Flag | Short | Description |
  347. |------|-------|-------------|
  348. | `--manifest` | `-m` | Path to manifest (required) |
  349. | `--output-dir` | `-o` | Parent directory for `batch_*` (default: outputs) |
  350. | `--config` | | Path to config YAML |
  351. | `--vlm-provider` | | VLM provider (default: gemini) |
  352. | `--vlm-model` | | VLM model override |
  353. | `--image-provider` | | Image gen provider |
  354. | `--image-model` | | Image gen model |
  355. | `--iterations` | `-n` | Refinement iterations per item |
  356. | `--auto` | | Loop until critic satisfied per item |
  357. | `--max-iterations` | | Safety cap for `--auto` |
  358. | `--optimize` | | Input optimization per item |
  359. | `--format` | `-f` | png, jpeg, or webp |
  360. | `--save-prompts` / `--no-save-prompts` | | Persist prompts (default: on, same as `plot`) |
  361. | `--venue` | | Venue style pack: built-in (neurips, icml, acl, ieee), a user pack, or `custom` |
  362. | `--aspect-ratio` | `-ar` | Default aspect ratio when not set in the manifest |
  363. | `--verbose` | `-v` | Verbose logging |
  364. ### `paperbanana orchestrate` -- Full-Paper Figure Package
  365. Generate a publication-focused figure bundle from a full paper source, with optional data-driven plots. The command:
  366. - parses the paper (`.txt`, `.md`, or `.pdf`)
  367. - plans multiple methodology figures from section structure
  368. - optionally discovers CSV/JSON files to plan statistical plots
  369. - runs generation for all planned items
  370. - writes a package folder containing `figure_package.json`, `figures/`, `figures.tex`, and `captions.md`
  371. ```bash
  372. paperbanana orchestrate \
  373. --paper paper.pdf \
  374. --data-dir ./results \
  375. --max-method-figures 4 \
  376. --max-plot-figures 3 \
  377. --optimize
  378. ```
  379. Use `--dry-run` to only plan and inspect `orchestration_plan.json` without API calls.
  380. Use `--resume-orchestrate <id-or-path>` to continue an interrupted orchestration from checkpoint state.
  381. | Flag | Description |
  382. |------|-------------|
  383. | `--paper` / `-p` | Paper source path (`.txt`, `.md`, or `.pdf`) |
  384. | `--resume-orchestrate` | Resume an existing orchestration by ID or directory |
  385. | `--retry-failed` | When resuming, include previously failed tasks |
  386. | `--max-retries` | Extra retries per task after first failure |
  387. | `--data-dir` | Optional directory containing CSV/JSON files for plot planning |
  388. | `--output-dir` / `-o` | Parent output directory (creates `orchestrate_*`) |
  389. | `--max-method-figures` | Max methodology figures to plan/generate |
  390. | `--max-plot-figures` | Max plot figures to plan/generate |
  391. | `--pdf-pages` | PDF-only page selection (e.g. `1-5`, `2,4,6-8`) |
  392. | `--optimize` | Enable input optimization for generated items |
  393. | `--iterations` / `-n` | Refinement iterations per generated item |
  394. | `--auto` + `--max-iterations` | Critic-driven auto-refine mode with safety cap |
  395. | `--concurrency` | Parallel figure generation workers |
  396. | `--format` / `-f` | Output format (`png`, `jpeg`, `webp`) |
  397. | `--dry-run` | Plan package only; no generation calls |
  398. ### `paperbanana composite` -- Compose Multi-Panel Figures
  399. Stitch multiple images into a single labeled figure with `(a)`, `(b)`, `(c)` sub-panel labels:
  400. ```bash
  401. paperbanana composite \
  402. panel_a.png panel_b.png panel_c.png \
  403. --layout 1x3 \
  404. --output figure2.png
  405. ```
  406. | Flag | Short | Description |
  407. |------|-------|-------------|
  408. | `IMAGES` | | Positional: paths to images to compose |
  409. | `--layout` | `-l` | Grid layout: `RxC` (e.g. `1x3`, `2x2`) or `auto` (default: auto) |
  410. | `--labels` | | Comma-separated labels, or `none` to disable (default: auto `(a),(b),...`) |
  411. | `--spacing` | `-s` | Pixel spacing between panels (default: 20) |
  412. | `--label-position` | | `top` or `bottom` (default: bottom) |
  413. | `--label-font-size` | | Font size for labels (default: 32) |
  414. | `--output` | `-o` | Output path (default: composite_output.png) |
  415. This command works on any existing images — no API calls needed. It is also triggered automatically when a batch manifest includes a `composite` section (see `paperbanana batch` above).
  416. ### `paperbanana evaluate` -- Quality Assessment
  417. Comparative evaluation of a generated diagram against a human reference using VLM-as-a-Judge:
  418. ```bash
  419. paperbanana evaluate \
  420. --generated diagram.png \
  421. --reference human_diagram.png \
  422. --context method.txt \
  423. --caption "Overview of our framework"
  424. ```
  425. | Flag | Short | Description |
  426. |------|-------|-------------|
  427. | `--generated` | `-g` | Path to generated image (required) |
  428. | `--reference` | `-r` | Path to human reference image (required) |
  429. | `--context` | | Path to source context text file or PDF (required) |
  430. | `--caption` | `-c` | Figure caption (required) |
  431. | `--pdf-pages` | | PDF context only: 1-based page selection (default: all) |
  432. Scores on 4 dimensions (hierarchical aggregation per the paper):
  433. - **Primary**: Faithfulness, Readability
  434. - **Secondary**: Conciseness, Aesthetics
  435. ### `paperbanana polish` -- Refine an Existing Figure
  436. Bring your own figure: a VLM audits it against the venue style guide and proposes up to 10 concrete, actionable improvements, then an image-edit capable provider applies them to the original figure (guided edit). Suggestions are printed to the console so you can see exactly what changed.
  437. ```bash
  438. paperbanana polish --input figure.png
  439. paperbanana polish --input figure.png --venue icml --iterations 2 --output polished.png
  440. ```
  441. | Flag | Short | Description |
  442. |------|-------|-------------|
  443. | `--input` | `-i` | Path to the existing figure image (required) |
  444. | `--output` | `-o` | Output path (default: `outputs/polish_<timestamp>/final_output.png`) |
  445. | `--venue` | | Venue style guide: `neurips` (default), `icml`, `acl`, `ieee`, `custom` |
  446. | `--iterations` | `-n` | Polish rounds; each round suggests and applies improvements on the previous result (default: 1) |
  447. | `--aspect-ratio` | `-ar` | Target aspect ratio (default: preserve the input figure's ratio) |
  448. | `--num-candidates` | `-k` | Apply each round's suggestions N times in parallel (1-8) |
  449. | `--budget` | | Budget cap in USD; polishing stops gracefully when exceeded |
  450. | `--seed` | | Random seed for reproducible edits |
  451. Requires an image provider that supports guided image edits (Google Gemini image models). If the figure already conforms to the style guide, polish exits without making changes.
  452. ### `paperbanana studio` -- Local web UI
  453. Requires `pip install 'paperbanana[studio]'` (Gradio).
  454. ```bash
  455. paperbanana studio
  456. paperbanana studio --port 8080 --output-dir ./my_outputs
  457. ```
  458. | Flag | Description |
  459. |------|-------------|
  460. | `--host` | Bind address (default `127.0.0.1`) |
  461. | `--port` | Port (default `7860`) |
  462. | `--share` | Create a temporary public Gradio link (do not use with sensitive data) |
  463. | `--config` | Path to YAML config |
  464. | `--output-dir` / `-o` | Default output directory for runs |
  465. | `--root-path` | URL subpath when behind a reverse proxy |
  466. ### `paperbanana setup` -- First-Time Configuration
  467. ```bash
  468. paperbanana setup
  469. ```
  470. Interactive wizard that first asks whether to use the official Gemini API.
  471. If you choose official API, it follows the default AI Studio key flow; if not, it asks for a custom Gemini-compatible URL and API key.
  472. ### `paperbanana data` -- Reference Dataset
  473. ```bash
  474. # Download the PaperBananaBench reference set (~254 MB, one command)
  475. paperbanana data download
  476. # Import plot references too (or both)
  477. paperbanana data download --task plot
  478. paperbanana data download --task both
  479. # Inspect / clear the cache
  480. paperbanana data info
  481. paperbanana data clear
  482. ```
  483. The dataset is served from a project-hosted GitHub release mirror
  484. ([`bench-data-v1`](https://github.com/llmsresearch/paperbanana/releases/tag/bench-data-v1))
  485. and its SHA256 checksum is verified before extraction. Credit to the
  486. [PaperBananaBench](https://huggingface.co/datasets/dwzhu/PaperBananaBench) authors —
  487. the mirror tracks their 2026-03-22 revision. The set is cached under
  488. `~/.cache/paperbanana/` (override with `PAPERBANANA_CACHE_DIR`); generation
  489. commands can also fetch it on first use via `--auto-download-data`.
  490. ---
  491. ## Python API
  492. ```python
  493. import asyncio
  494. from paperbanana import PaperBananaPipeline, GenerationInput, DiagramType
  495. from paperbanana.core.config import Settings
  496. settings = Settings(
  497. vlm_provider="openai",
  498. vlm_model="gpt-5.2",
  499. image_provider="openai_imagen",
  500. image_model="gpt-image-1.5",
  501. optimize_inputs=True, # Enable input optimization
  502. auto_refine=True, # Loop until critic is satisfied
  503. )
  504. pipeline = PaperBananaPipeline(settings=settings)
  505. result = asyncio.run(pipeline.generate(
  506. GenerationInput(
  507. source_context="Our framework consists of...",
  508. communicative_intent="Overview of the proposed method.",
  509. diagram_type=DiagramType.METHODOLOGY,
  510. )
  511. ))
  512. print(f"Output: {result.image_path}")
  513. ```
  514. **Progress callbacks:** `generate()` and `continue_run()` accept an optional `progress_callback` argument. The pipeline invokes it with `PipelineProgressEvent` objects (stage, message, seconds, iteration, extra) at each step (optimizer, retriever, planner, stylist, visualizer, critic), so you can show progress in UIs or log timing without patching agents.
  515. To continue a previous run:
  516. ```python
  517. from paperbanana.core.resume import load_resume_state
  518. state = load_resume_state("outputs", "run_20260218_125448_e7b876")
  519. result = asyncio.run(pipeline.continue_run(
  520. resume_state=state,
  521. additional_iterations=3,
  522. user_feedback="Make the encoder block more prominent",
  523. ))
  524. ```
  525. See `examples/generate_diagram.py` and `examples/generate_plot.py` for complete working examples.
  526. ---
  527. ## MCP Server
  528. PaperBanana includes an MCP server for use with Claude Code, Cursor, or any MCP-compatible client. Add the following config to use it via `uvx` without a local clone:
  529. ```json
  530. {
  531. "mcpServers": {
  532. "paperbanana": {
  533. "command": "uvx",
  534. "args": ["--from", "paperbanana[mcp]", "paperbanana-mcp"],
  535. "env": { "GOOGLE_API_KEY": "your-google-api-key" }
  536. }
  537. }
  538. }
  539. ```
  540. Eleven MCP tools are exposed: `generate_diagram`, `generate_plot`, `continue_run` (resume a prior `run_*` with optional feedback), `continue_diagram`, `continue_plot`, `evaluate_diagram`, `evaluate_plot`, `orchestrate_figures` (full-paper figure packages), `batch_diagrams`, `batch_plots`, and `download_references`.
  541. The repo also ships with 3 Claude Code skills:
  542. - `/generate-diagram <file> [caption]` - generate a methodology diagram from a text file
  543. - `/generate-plot <data-file> [intent]` - generate a statistical plot from CSV/JSON data
  544. - `/evaluate-diagram <generated> <reference>` - evaluate a diagram against a human reference
  545. See [`mcp_server/README.md`](mcp_server/README.md) for full setup details (Claude Code, Cursor, local development).
  546. ---
  547. ## Overleaf Integration (GitHub Action)
  548. Keep your paper's methodology figure in sync with the text — automatically. PaperBanana ships a GitHub Action that pairs with Overleaf's built-in GitHub sync: push your `.tex` changes, the action extracts the methodology section, generates the figure, and commits back the image plus a ready-to-`\input` LaTeX snippet. Pull in Overleaf and it's in your file tree.
  549. ```yaml
  550. - uses: actions/checkout@v4
  551. - uses: llmsresearch/paperbanana/integrations/github-action@main
  552. with:
  553. tex-file: sections/method.tex
  554. caption: "Overview of our proposed framework"
  555. env:
  556. OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
  557. ```
  558. See [`integrations/github-action/README.md`](integrations/github-action/README.md) for the full workflow, all inputs, and cost-control options.
  559. ---
  560. ## Configuration
  561. Default settings are in `configs/config.yaml`. Override via CLI flags or a custom YAML:
  562. ```bash
  563. paperbanana generate \
  564. --input method.txt \
  565. --caption "Overview" \
  566. --config my_config.yaml
  567. ```
  568. Key settings:
  569. ```yaml
  570. vlm:
  571. provider: openai # openai, atlas, gemini, or openrouter
  572. model: gpt-5.2
  573. image:
  574. provider: openai_imagen # openai_imagen, atlas_imagen, google_imagen, or openrouter_imagen
  575. model: gpt-image-1.5
  576. pipeline:
  577. num_retrieval_examples: 10
  578. refinement_iterations: 3
  579. # auto_refine: true # Loop until critic is satisfied
  580. # max_iterations: 30 # Safety cap for auto_refine mode
  581. # optimize_inputs: true # Preprocess inputs for better generation
  582. output_resolution: "2k"
  583. reference:
  584. path: data/reference_sets
  585. output:
  586. dir: outputs
  587. save_iterations: true
  588. save_metadata: true
  589. ```
  590. Environment variables (`.env`):
  591. ```bash
  592. # OpenAI (default)
  593. OPENAI_API_KEY=your-key
  594. OPENAI_BASE_URL=https://api.openai.com/v1 # or Azure endpoint
  595. OPENAI_VLM_MODEL=gpt-5.2 # override model
  596. OPENAI_IMAGE_MODEL=gpt-image-1.5 # override model
  597. # Atlas Cloud
  598. ATLASCLOUD_API_KEY=your-key
  599. ATLASCLOUD_BASE_URL=https://api.atlascloud.ai/v1
  600. ATLASCLOUD_VLM_MODEL=deepseek-ai/DeepSeek-V3-0324
  601. ATLASCLOUD_IMAGE_BASE_URL=https://api.atlascloud.ai/api/v1
  602. ATLASCLOUD_IMAGE_MODEL=openai/gpt-image-2/text-to-image
  603. # Google Gemini (alternative, free)
  604. GOOGLE_API_KEY=your-key
  605. GOOGLE_BASE_URL= # optional custom Gemini-compatible endpoint
  606. GOOGLE_VLM_MODEL=gemini-2.5-flash # override Gemini VLM model
  607. GOOGLE_IMAGE_MODEL=gemini-3-pro-image-preview # override Gemini image model
  608. ```
  609. ---
  610. ## Project Structure
  611. ```
  612. paperbanana/
  613. ├── paperbanana/
  614. │ ├── core/ # Pipeline orchestration, types, config, resume, utilities
  615. │ ├── agents/ # Optimizer, Retriever, Planner, Stylist, Visualizer, Critic
  616. │ ├── providers/ # VLM and image gen provider implementations
  617. │ │ ├── vlm/ # OpenAI, Atlas Cloud, Gemini, OpenRouter VLM providers
  618. │ │ └── image_gen/ # OpenAI, Atlas Cloud, Gemini, OpenRouter image gen providers
  619. │ ├── reference/ # Reference set management (13 curated examples)
  620. │ ├── guidelines/ # Style guidelines loader
  621. │ └── evaluation/ # VLM-as-Judge evaluation system
  622. ├── configs/ # YAML configuration files
  623. ├── prompts/ # Prompt templates for all agents + evaluation
  624. │ ├── diagram/ # context_enricher, caption_sharpener, retriever, planner, stylist, visualizer, critic
  625. │ ├── plot/ # plot-specific prompt variants
  626. │ └── evaluation/ # faithfulness, conciseness, readability, aesthetics
  627. ├── data/
  628. │ ├── reference_sets/ # 13 verified methodology diagrams
  629. │ └── guidelines/ # NeurIPS-style aesthetic guidelines
  630. ├── examples/ # Working example scripts + sample inputs
  631. ├── scripts/ # Data curation and build scripts
  632. ├── tests/ # Test suite
  633. ├── mcp_server/ # MCP server for IDE integration
  634. └── .claude/skills/ # Claude Code skills (generate-diagram, generate-plot, evaluate-diagram)
  635. ```
  636. ## Development
  637. ```bash
  638. # Install with dev dependencies
  639. pip install -e ".[dev,openai,google]"
  640. # Run tests
  641. pytest tests/ -v
  642. # Lint
  643. ruff check paperbanana/ mcp_server/ tests/ scripts/
  644. # Format
  645. ruff format paperbanana/ mcp_server/ tests/ scripts/
  646. ```
  647. ## Citation
  648. This is an **unofficial** implementation. If you use this work, please cite the **original paper**:
  649. ```bibtex
  650. @article{zhu2026paperbanana,
  651. title={PaperBanana: Automating Academic Illustration for AI Scientists},
  652. author={Zhu, Dawei and Meng, Rui and Song, Yale and Wei, Xiyu
  653. and Li, Sujian and Pfister, Tomas and Yoon, Jinsung},
  654. journal={arXiv preprint arXiv:2601.23265},
  655. year={2026}
  656. }
  657. ```
  658. **Original paper**: [https://arxiv.org/abs/2601.23265](https://arxiv.org/abs/2601.23265)
  659. ## Disclaimer
  660. This project is an independent open-source reimplementation based on the publicly available paper.
  661. It is not affiliated with, endorsed by, or connected to the original authors, Google Research, or
  662. Peking University in any way. The implementation may differ from the original system described in the paper.
  663. Use at your own discretion.
  664. ## License
  665. MIT

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.