Skip to content
Back to search

Camyla

Scaling Autonomous Research in Medical Image Segmentation

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/yifangao112--camyla/badge.svg)](/listings/yifangao112--camyla)

HTML

<a href="/listings/yifangao112--camyla"><img src="/api/listings/yifangao112--camyla/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

1. Install

git clone https://github.com/yifangao112/Camyla.git camyla
cd camyla
python -m venv .venv && source .venv/bin/activate   # or: conda create -n camyla python=3.10
pip install -r requirements.txt

# Sister packages (segmentation framework + raw-data conversion agent)
pip install git+https://github.com/yifangao112/CamylaNet.git
pip install git+https://github.com/yifangao112/nnPrep.git

> A Python 3.10+ environment is expected. GPU is required for actually running > the segmentation experiments (baseline uses CamylaNet / nnU-Net v2).

2. Configure

Set CamylaNet's data-path environment variables (same convention as CamylaNet's own README — the baseline pipeline reads and writes under these paths):

export camylanet_raw="/path/to/camylanet_raw"
export camylanet_preprocessed="/path/to/camylanet_preprocessed"
export camylanet_results="/path/to/camylanet_results"   # baseline artifacts land here

Then copy the example Camyla config and edit it:

cp config_example.yaml config.yaml

At minimum, set:

- At least one entry under `llm_endpoints` with a valid `api_key` (or export the matching `api_key_env`, e.g. `OPENROUTER_API_KEY`) - `default_endpoint` — the endpoint name most roles will use

See [Configuration](#configuration) below for the full layout, role-routing semantics, and N-way model competition. The shipped [config_example.yaml](config_example.yaml) already has a complete multi-provider setup you can copy from.

3. Prepare the dataset

Camyla operates on a dataset that already lives in nnU-Net v2 layout under `$camylanet_raw/Dataset{ID}_{Abbr}/`. You have three options:

- **Use CamylaBench (recommended).** The 31 datasets used in the paper are pre-formatted and ready to drop in — download from [📦 Google Drive](https://drive.google.com/drive/folders/11BrGWWZw8yr2o2AkVV9o5e7de-T9nngz?usp=sharing) and extract each `Dataset{ID}_{Abbr}/` folder under `$camylanet_raw/`. The ready-made idea descriptions in [`ideas/`](ideas/) match these IDs. - **Bring your own, already in nnU-Net v2 format.** Just point `camylanet_raw` at it. - **Bring your own raw data in some arbitrary layout.** Use [nnPrep](https://github.com/yifangao112/nnPrep) (an LLM agent that converts arbitrary medical segmentation datasets into the nnU-Net v2 format) — or write your own conversion script.

4. Run

Pick an idea from [`ideas/`](ideas/) (31 ready-to-use dataset descriptions) and launch:

python launch_camyla.py \
    --config config.yaml \
    --load_ideas ideas/900.json \
    --idea_idx 0

On first run for a given dataset, Camyla automatically runs the baseline pipeline (trainer screening + full training for everything that passes) under `$camylanet_results`. Subsequent runs reuse the artifacts and skip this step.

The pipeline writes everything to `experiments/<date>_<idea>_attempt_<id>/`:

experiments/2026-04-12_liver_segmentation_attempt_0/
├── idea.json / idea.md           # task spec
├── config.yaml                   # resolved config used for this run
├── logs/0-run/
│   ├── experiment_report.md      # human-readable summary
│   └── experiment_results/       # metrics, checkpoints, plots
├── research_proposals/           # auto-generated proposals
└── paper/                        # LaTeX + compiled PDF (if writeup enabled)

Common flags:

| Flag | Purpose | |------|---------| | `--resume_from_checkpoint PATH` | Resume from a previous `checkpoint.pkl` | | `--skip_writeup` / `--skip_review` | Run experiments only, skip paper generation | | `--debug-baseline` | Fake the baseline metrics so Stage 2 runs immediately (dev only) | | `--verbose` | DEBUG-level logging |

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 403 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <p align="center">
  2. <img src="assets/logo.png" width="420" alt="Camyla">
  3. </p>
  4. <p align="center">
  5. [ English | <a href="README_zh.md">中文</a> ]
  6. </p>
  7. ## Large-scale, fully autonomous medical image segmentation research — dataset in, paper out.
  8. ---
  9. ## 📄 Paper
  10. ### Camyla: Scaling Autonomous Research in Medical Image Segmentation
  11. Yifan Gao¹², Haoyue Li¹, Feng Yuan¹, Xin Gao¹\*, Weiran Huang²³\*, Xiaosong Wang²⁴\*
  12. <sup>¹ USTC · ² Shanghai Innovation Institute · ³ SJTU · ⁴ Shanghai AI Lab · \* Corresponding authors</sup>
  13. 📑 **[Read the paper (PDF)](assets/camyla.pdf)** · **[alphaXiv](https://www.alphaxiv.org/abs/2604.10696)** · **[arXiv](https://arxiv.org/abs/2604.10696)**
  14. <p align="center">
  15. <img src="assets/hero.png" width="760" alt="Camyla hero">
  16. </p>
  17. **Headline results** (28 days, **zero human intervention**):
  18. - **40** complete manuscripts written end-to-end
  19. - **Cost-efficient**: only **$20–30 per paper** in LLM API spend
  20. - **Beats the strongest per-dataset baseline** (chosen from 14 established architectures including nnU-Net) on **24 of 31** datasets under identical training budgets
  21. - **CamylaBench**: contamination-free benchmark of 31 datasets, built exclusively from 2025 publications
  22. - **Stronger long-horizon orchestration on the experiment stage.** During experiment execution, driven by cost-efficient backends (**GLM-4.7** + **MiniMax-M2.5**), Camyla outperforms **AI Scientist**, **autoresearch Claude Code (Opus 4.6)**, and **autoresearch Codex (GPT-5.4-xhigh)** on execution success, completion rate, and fidelity to the original proposal
  23. **Manuscript quality — double-blind evaluation.** Camyla-generated manuscripts were mixed with real 2025 publications and judged without reviewers knowing which was AI-written. Four independent panels — **5 senior reviewers**, **10 junior reviewers**, **5 different AI models**, and the **Stanford Agentic Reviewer** — all place Camyla's output between the T1 and T2 tier of contemporary medical-imaging journals. The T1 anchor is IEEE TMI / Medical Image Analysis; the T2 band is JCR Q1 medical-imaging journals:
  24. | Tier | Journals | Papers | Representative venues |
  25. |------|:-------:|:------:|-----------------------|
  26. | **T1** (top-tier) | 2 | 10 | IEEE Transactions on Medical Imaging; Medical Image Analysis |
  27. | **T2** (JCR Q1) | 7 | 35 | IEEE Journal of Biomedical and Health Informatics; Artificial Intelligence in Medicine; et al. |
  28. | **T3** | 9 | 45 | International Journal of Computer Assisted Radiology and Surgery; Biomedical Physics & Engineering Express; et al. |
  29. | **Total** | 18 | 90 | |
  30. ---
  31. ## 📰 News
  32. - **2026-04-13** — Initial public release of the code, the paper, and **CamylaBench** (31 pre-formatted datasets, [📦 Google Drive](https://drive.google.com/drive/folders/11BrGWWZw8yr2o2AkVV9o5e7de-T9nngz?usp=sharing)). We are still cleaning the full **CamylaTrace-232K** trajectory dataset (per-run logs + intermediate artifacts); release will follow shortly.
  33. ---
  34. Camyla is an automated research pipeline that takes a medical image segmentation task,
  35. searches the literature for relevant ideas, proposes research hypotheses, runs
  36. end-to-end deep-learning experiments (Baseline → Creative Research → Ablation), and
  37. writes up the results as a publication-ready paper.
  38. It combines:
  39. - **Quality-Weighted Branch Exploration (QWBE)** over experiment configurations
  40. - **OpenHands**-driven code generation and iterative debugging
  41. - Multi-source literature search (ArXiv, OpenAlex, PubMed, Semantic Scholar)
  42. - A paper agent that drafts, compiles, and cites LaTeX papers (Elsevier format supported)
  43. - A flexible LLM routing layer (`llm_endpoints` + `llm_roles`) so you can mix providers
  44. (OpenRouter, GLM, MiniMax, …) without touching code
  45. > **Status.** Research prototype, preparing for open-source release. APIs may change.
  46. ---
  47. ## 📚 Sample generated papers
  48. A 10-paper subset of the manuscripts Camyla produced end-to-end — no hand-editing,
  49. LaTeX compiled as-is by the pipeline. Each PDF lives under
  50. [`assets/paper_pdf/`](assets/paper_pdf/).
  51. | # | Title | Modality / task |
  52. |:-:|-------|-----------------|
  53. | 1 | [Cross-Directional Feature Lattice for Brain Tumor Segmentation](assets/paper_pdf/01.pdf) | MRI · brain tumor |
  54. | 2 | [Scale-Frequency Adaptive Fusion for Multiple Sclerosis Lesion Segmentation](assets/paper_pdf/02.pdf) | MRI · MS lesions |
  55. | 3 | [Hierarchical Context Gating for Neonatal Brain Lesion Segmentation](assets/paper_pdf/03.pdf) | MRI · neonatal HIE |
  56. | 4 | [Cross-Scale Mutual Refinement for Bronchoalveolar Lavage Fluid Cell Segmentation](assets/paper_pdf/04.pdf) | Microscopy · BALF cells |
  57. | 5 | [Symmetry-Aware Cascaded Attention for Panoramic Tooth Segmentation](assets/paper_pdf/05.pdf) | Dental X-ray · tooth |
  58. | 6 | [Specular-Residual Decoupled Encoding for Surgical Scene Segmentation](assets/paper_pdf/06.pdf) | Laparoscopy · surgical scene |
  59. | 7 | [Adaptive Scale-Aware Feature Integration for Liver Lesion Segmentation](assets/paper_pdf/07.pdf) | CT · liver lesion |
  60. | 8 | [Boundary-Hierarchical Decomposition for Fetal Brain Tissue Segmentation](assets/paper_pdf/08.pdf) | MRI · fetal brain |
  61. | 9 | [Vessel-Guided Boundary Residual Networks for Dermatological Vessel Segmentation](assets/paper_pdf/09.pdf) | OCTA · dermatological vessel |
  62. | 10 | [Hierarchical Resolution-Retentive Feature Encoding for Brain Metastasis Segmentation](assets/paper_pdf/10.pdf) | MRI · brain metastasis |
  63. ---
  64. ## Quick start
  65. ### 1. Install
  66. ```bash
  67. git clone https://github.com/yifangao112/Camyla.git camyla
  68. cd camyla
  69. python -m venv .venv && source .venv/bin/activate # or: conda create -n camyla python=3.10
  70. pip install -r requirements.txt
  71. # Sister packages (segmentation framework + raw-data conversion agent)
  72. pip install git+https://github.com/yifangao112/CamylaNet.git
  73. pip install git+https://github.com/yifangao112/nnPrep.git
  74. ```
  75. > A Python 3.10+ environment is expected. GPU is required for actually running
  76. > the segmentation experiments (baseline uses CamylaNet / nnU-Net v2).
  77. ### 2. Configure
  78. Set CamylaNet's data-path environment variables (same convention as CamylaNet's
  79. own README — the baseline pipeline reads and writes under these paths):
  80. ```bash
  81. export camylanet_raw="/path/to/camylanet_raw"
  82. export camylanet_preprocessed="/path/to/camylanet_preprocessed"
  83. export camylanet_results="/path/to/camylanet_results" # baseline artifacts land here
  84. ```
  85. Then copy the example Camyla config and edit it:
  86. ```bash
  87. cp config_example.yaml config.yaml
  88. ```
  89. At minimum, set:
  90. - At least one entry under `llm_endpoints` with a valid `api_key` (or export the
  91. matching `api_key_env`, e.g. `OPENROUTER_API_KEY`)
  92. - `default_endpoint` — the endpoint name most roles will use
  93. See [Configuration](#configuration) below for the full layout, role-routing
  94. semantics, and N-way model competition. The shipped
  95. [config_example.yaml](config_example.yaml) already has a complete multi-provider
  96. setup you can copy from.
  97. ### 3. Prepare the dataset
  98. Camyla operates on a dataset that already lives in nnU-Net v2 layout under
  99. `$camylanet_raw/Dataset{ID}_{Abbr}/`. You have three options:
  100. - **Use CamylaBench (recommended).** The 31 datasets used in the paper are
  101. pre-formatted and ready to drop in — download from
  102. [📦 Google Drive](https://drive.google.com/drive/folders/11BrGWWZw8yr2o2AkVV9o5e7de-T9nngz?usp=sharing)
  103. and extract each `Dataset{ID}_{Abbr}/` folder under `$camylanet_raw/`.
  104. The ready-made idea descriptions in [`ideas/`](ideas/) match these IDs.
  105. - **Bring your own, already in nnU-Net v2 format.** Just point `camylanet_raw` at it.
  106. - **Bring your own raw data in some arbitrary layout.** Use
  107. [nnPrep](https://github.com/yifangao112/nnPrep) (an LLM agent that converts
  108. arbitrary medical segmentation datasets into the nnU-Net v2 format) — or
  109. write your own conversion script.
  110. ### 4. Run
  111. Pick an idea from [`ideas/`](ideas/) (31 ready-to-use dataset descriptions) and launch:
  112. ```bash
  113. python launch_camyla.py \
  114. --config config.yaml \
  115. --load_ideas ideas/900.json \
  116. --idea_idx 0
  117. ```
  118. On first run for a given dataset, Camyla automatically runs the baseline pipeline
  119. (trainer screening + full training for everything that passes) under
  120. `$camylanet_results`. Subsequent runs reuse the artifacts and skip this step.
  121. The pipeline writes everything to `experiments/<date>_<idea>_attempt_<id>/`:
  122. ```
  123. experiments/2026-04-12_liver_segmentation_attempt_0/
  124. ├── idea.json / idea.md # task spec
  125. ├── config.yaml # resolved config used for this run
  126. ├── logs/0-run/
  127. │ ├── experiment_report.md # human-readable summary
  128. │ └── experiment_results/ # metrics, checkpoints, plots
  129. ├── research_proposals/ # auto-generated proposals
  130. └── paper/ # LaTeX + compiled PDF (if writeup enabled)
  131. ```
  132. Common flags:
  133. | Flag | Purpose |
  134. |------|---------|
  135. | `--resume_from_checkpoint PATH` | Resume from a previous `checkpoint.pkl` |
  136. | `--skip_writeup` / `--skip_review` | Run experiments only, skip paper generation |
  137. | `--debug-baseline` | Fake the baseline metrics so Stage 2 runs immediately (dev only) |
  138. | `--verbose` | DEBUG-level logging |
  139. ---
  140. ## How it works
  141. <p align="center">
  142. <img src="assets/camyla-overview.png" width="820" alt="Camyla system overview">
  143. </p>
  144. - **Phase 1-3 (idea generation).** Searches 1-4 literature sources, extracts open
  145. research challenges, and generates multiple proposals scored by an assessment LLM.
  146. - **Stage 1-3 (experiment).** QWBE expands a tree of code variants. Stage 2 can run
  147. **N-way model competition** by listing multiple `experiment.code.candidates`.
  148. - **Paper Agent.** Takes the final experiment results and produces a cited paper.
  149. ---
  150. ## Configuration
  151. Every LLM call Camyla makes resolves through a two-layer system:
  152. 1. **`llm_endpoints`** — your named LLM connections (one per provider/backend).
  153. 2. **`llm_roles`** — every internal component ("role") picks an endpoint and
  154. optionally overrides its model, temperature, or `max_tokens`.
  155. Anything you don't configure at the role level falls back to `default_endpoint`.
  156. ### 1. `llm_endpoints` — connections
  157. Each entry is an OpenAI-compatible endpoint. The shape is fixed:
  158. ```yaml
  159. llm_endpoints:
  160. my_openrouter:
  161. api_key: "" # inline key (leave empty → use env)
  162. api_key_env: OPENROUTER_API_KEY # env var name to read when api_key is empty
  163. base_url: "https://openrouter.ai/api/v1"
  164. model: "deepseek/deepseek-v3.2" # default model for this endpoint
  165. temperature: 0.5 # default temperature
  166. ```
  167. **Key resolution order:** non-empty `api_key` > environment variable named by
  168. `api_key_env` > empty (error at first call).
  169. **Naming is free-form.** `my_openrouter`, `cheap_backend`, `gpt4` — all valid.
  170. Roles reference endpoints by name, so changing models across the whole pipeline
  171. is just swapping one string. Common backends we've tested:
  172. | Backend | `base_url` | Notes |
  173. |------------------|-----------------------------------------------|-------|
  174. | OpenRouter | `https://openrouter.ai/api/v1` | one key, 300+ models |
  175. | DashScope (Qwen) | `https://coding.dashscope.aliyuncs.com/v1` | cheap Qwen/GLM routing |
  176. | MiniMax | `https://api.minimaxi.com/v1` | M-series models |
  177. | OpenAI | `https://api.openai.com/v1` | native |
  178. | Local vLLM / Ollama | `http://localhost:8000/v1` | any OpenAI-compatible server |
  179. ### 2. `default_endpoint`
  180. The global fallback. Every role without an explicit `endpoint` uses this one.
  181. ```yaml
  182. default_endpoint: my_openrouter
  183. ```
  184. ### 3. `llm_roles` — per-role overrides
  185. A role is one logical LLM use inside the pipeline (feedback, paper writer, idea
  186. generator, etc.). You only specify the fields you want to override:
  187. ```yaml
  188. llm_roles:
  189. # Tree-search roles
  190. feedback: { temperature: 0.9, max_tokens: 8192 }
  191. log_summary: { temperature: 1.0 }
  192. # Idea-generation roles — swap to a cheaper model for these
  193. literature_backbone: { model: google/gemini-3-flash-preview }
  194. challenge_extraction: { temperature: 0.3 }
  195. # Paper agent sub-agents: `_default` applies to the whole group unless
  196. # an individual sub-agent overrides it.
  197. paper_agent:
  198. _default: { temperature: 0.6 }
  199. BibtexAgent: { model: z-ai/glm-4.7, temperature: 0.3 }
  200. IdeaGenerationAgent: { model: google/gemini-3-flash-preview, temperature: 0.8 }
  201. # Paper writing roles can be routed to a different endpoint entirely.
  202. paper_writing:
  203. latex_editor: { endpoint: my_dashscope, temperature: 0.7 }
  204. image_generator: { endpoint: my_openrouter,
  205. model: google/gemini-3.1-flash-image-preview,
  206. aspect_ratio: "16:9", image_size: "2K" }
  207. ```
  208. Override precedence for any given role: role fields > its endpoint's defaults.
  209. You can point a role at a completely different endpoint with `endpoint: <name>`.
  210. ### 4. N-way model competition (Stage 2)
  211. Under `experiment.code.candidates`, list the **endpoint names** that should compete
  212. as code authors for Stage 2. Camyla will run one branch per candidate and keep
  213. the strongest.
  214. ```yaml
  215. experiment:
  216. code:
  217. candidates: [my_dashscope, my_minimax]
  218. max_tokens: 16384
  219. ```
  220. Set it to a single-element list to disable competition.
  221. ### 5. Non-LLM API keys
  222. Literature search keys live under `api_keys` (same inline/env fallback pattern):
  223. ```yaml
  224. api_keys:
  225. s2: { value: "", env: S2_API_KEY } # Semantic Scholar
  226. ncbi: { value: "", env: NCBI_API_KEY } # PubMed
  227. ```
  228. Camyla works without these, but rate limits will be tighter.
  229. ### 6. Tuning the run
  230. The rest of the config controls **what** Camyla does, not **which LLMs** it uses:
  231. - `idea_generation.*` — how many papers to search, how to score proposals,
  232. how many generator personalities to ensemble
  233. - `experiment.stages.*` — per-stage iteration budgets (Stage 1 = baseline
  234. replication, Stage 2 = creative research, Stage 3 = ablation)
  235. - `experiment.openhands.*` — OpenHands coder settings (python path, iteration
  236. cap, condenser)
  237. - `experiment.search.*` — QWBE hyperparameters (UCB constant, debug probability,
  238. draft count)
  239. All of these have sensible defaults in [config_example.yaml](config_example.yaml);
  240. you usually only touch `stages.*_max_iters` when you want a faster / cheaper run.
  241. ---
  242. ## Repository layout
  243. ```
  244. camyla/
  245. ├── LICENSE
  246. ├── README.md # this file
  247. ├── config_example.yaml # documented config template
  248. ├── requirements.txt
  249. ├── launch_camyla.py # main entry point
  250. ├── ideas/ # 31 ready-made idea JSONs
  251. └── camyla/ # core package
  252. ├── model_config.py # LLM config loader (get_endpoint/get_role)
  253. ├── baseline/ # screening + full training before QWBE
  254. ├── infrastructure/literature/ # arxiv / openalex / pubmed / multi-source
  255. ├── paper_agent/ # LaTeX writer + plotters + bibtex agent
  256. ├── tools/ # OpenAlex / Semantic Scholar tools
  257. ├── treesearch/ # QWBE core, parallel agents, OpenHands coder
  258. └── utils/
  259. ```
  260. ---
  261. ## Related repositories
  262. - [CamylaNet](https://github.com/yifangao112/CamylaNet) — segmentation framework
  263. built on nnU-Net v2, shipping a curated set of CNN / Transformer / state-space
  264. backbones. Camyla's baseline stage runs CamylaNet trainers.
  265. - [nnPrep](https://github.com/yifangao112/nnPrep) — LLM agent that converts
  266. arbitrary medical segmentation datasets into the nnU-Net v2 format consumed
  267. by CamylaNet (and therefore by Camyla).
  268. ---
  269. ## Limitations & notes
  270. - The baseline stage currently uses CamylaNet trainers. Swapping in another
  271. baseline framework requires a corresponding skill under `skills/frameworks/`.
  272. - OpenHands runs code in your local Python env — make sure you point
  273. `experiment.openhands.python_path` at an env with the needed packages.
  274. - Long runs: a full idea (proposals → 3 stages → paper) typically takes
  275. several hours to a day on a single A100-class GPU.
  276. ---
  277. ## Citation
  278. If you find this project useful in academic work, please cite:
  279. ```bibtex
  280. @misc{gao2026camyla,
  281. title = {Camyla: Scaling Autonomous Research in Medical Image Segmentation},
  282. author = {Gao, Yifan and Li, Haoyue and Yuan, Feng and Gao, Xin and Huang, Weiran and Wang, Xiaosong},
  283. year = {2026},
  284. eprint = {2604.10696},
  285. archivePrefix = {arXiv},
  286. primaryClass = {cs.AI}
  287. }
  288. ```
  289. ---
  290. ## Acknowledgements
  291. Camyla builds on ideas and code from two upstream projects:
  292. - [**AI Scientist** (Sakana AI)](https://github.com/SakanaAI/AI-Scientist) — pioneered the autonomous research-agent paradigm that inspired Camyla's overall pipeline design.
  293. - [**nnU-Net**](https://github.com/MIC-DKFZ/nnUNet) — the self-configuring segmentation framework that Camyla's baseline stage (via [CamylaNet](https://github.com/yifangao112/CamylaNet)) is built on.
  294. ---
  295. ## License
  296. Released under the **Apache License, Version 2.0** — see [LICENSE](LICENSE).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.