Skip to content
Back to search

arbor

MarketplaceWorks with Claude CodeGitHub

A generalist autonomous research agent — runs experiments, researches, and iteratively optimizes, autonomously.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/ruc-nlpir--arbor/badge.svg)](/listings/ruc-nlpir--arbor)

HTML

<a href="/listings/ruc-nlpir--arbor"><img src="/api/listings/ruc-nlpir--arbor/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

**Requirements:** Python ≥ 3.10 and Git. A virtual environment is recommended.

pip install arbor-agent   # or: uv pip install arbor-agent
arbor doctor              # verify PATH, git, API keys

> Prefer a global command? `pipx install arbor-agent` makes `arbor` available everywhere.

<details> <summary>Install from source (for development)</summary>

git clone https://github.com/RUC-NLPIR/Arbor.git
cd Arbor
python -m venv .venv && source .venv/bin/activate   # recommended
pip install -e .                                    # or: uv pip install -e .
arbor doctor

For the docs site, `pip install -e ".[docs]" && mkdocs serve`, or read them online via the **Docs** badge above.

</details>

---

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 14 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "arbor",
  3. "owner": {
  4. "name": "RUC-NLPIR",
  5. "url": "https://github.com/RUC-NLPIR"
  6. },
  7. "plugins": [
  8. {
  9. "name": "arbor",
  10. "source": "./",
  11. "description": "Run Arbor's autonomous research workflow inside Claude Code with your own model (keyless). Installs the Arbor skill suite and the `arbor mcp` tools in one step."
  12. }
  13. ]
  14. }
README.md · README · 634 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <p align="center">
  2. <img src="assets/hero.svg" alt="Arbor — Optimize anything" width="100%">
  3. </p>
  4. <h1 align="center">🌳 Arbor</h1>
  5. <h3 align="center">The autonomous research agent that beats Claude Code and Codex by <b>2.5×</b> on the same compute budget</h3>
  6. <p align="center">
  7. <a href="https://arxiv.org/pdf/2606.11926"><img src="https://img.shields.io/badge/Paper-arXiv-B31B1B?style=for-the-badge&logo=arxiv&logoColor=white" alt="Paper"></a>
  8. <a href="https://github.com/RUC-NLPIR/Arbor"><img src="https://img.shields.io/badge/Code-GitHub-181717?style=for-the-badge&logo=github&logoColor=white" alt="GitHub"></a>
  9. <a href="https://RUC-NLPIR.github.io/Arbor/"><img src="https://img.shields.io/badge/Project_Page-Live-0E9B9B?style=for-the-badge&logo=githubpages&logoColor=white" alt="Project Page"></a>
  10. <a href="https://RUC-NLPIR.github.io/Arbor/demo.html"><img src="https://img.shields.io/badge/Live_Demo-Watch-FF5C8A?style=for-the-badge&logo=githubpages&logoColor=white" alt="Live Demo"></a>
  11. <a href="https://RUC-NLPIR.github.io/Arbor/docs/"><img src="https://img.shields.io/badge/Docs-Material-526CFE?style=for-the-badge&logo=materialformkdocs&logoColor=white" alt="Docs"></a>
  12. <a href="https://github.com/RUC-NLPIR/Arbor/discussions"><img src="https://img.shields.io/badge/Discussions-Join-5865F2?style=for-the-badge&logo=github&logoColor=white" alt="Discussions"></a>
  13. <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-D22128?style=for-the-badge&logo=apache&logoColor=white" alt="License: Apache 2.0"></a>
  14. </p>
  15. <p align="center">
  16. <b>English</b> | <a href="README.zh-CN.md">简体中文</a>
  17. </p>
  18. <p align="center">
  19. <i>Give Arbor a benchmark and a goal. It proposes hypotheses, edits code, runs real
  20. experiments, and keeps only the gains that survive held-out data — growing a
  21. <b>hypothesis tree</b> instead of forgetting what failed.</i>
  22. </p>
  23. > **▶️ Try it in 30 seconds — no API key, no config:**
  24. >
  25. > ```bash
  26. > pip install arbor-agent && arbor replay --demo # watch the hypothesis tree grow live
  27. > ```
  28. >
  29. > Or **watch it right now in your browser** — nothing to install: **[▶️ Live Demo](https://RUC-NLPIR.github.io/Arbor/demo.html)**.
  30. ### 🏆 One controller, six tasks — wins the held-out test on all of them
  31. | Task | Metric | Claude Code | Codex | **Arbor** |
  32. | --- | --- | :---: | :---: | :---: |
  33. | BrowseComp | acc ↑ | 53.33 | 50.00 | **67.67** |
  34. | Terminal-Bench 2.0 | pass ↑ | 71.70 | 73.59 | **77.36** |
  35. | Math-Reasoning Data | gap ↑ | 8.33 | 6.25 | **20.83** |
  36. Plus **86.36% Any-Medal on MLE-Bench Lite** (GPT-5.5). → [See all six tasks](#-results) · [project page](https://RUC-NLPIR.github.io/Arbor/) · [paper](https://arxiv.org/pdf/2606.11926) · [docs](https://RUC-NLPIR.github.io/Arbor/docs/)
  37. ## 🎬 Demo
  38. https://github.com/user-attachments/assets/49c1a306-d2e9-49d6-9c83-65e38a62df30
  39. ## 📣 News
  40. - **2026-06-30** — **Arbor learns from its own runs.** Each run leaves concrete, reusable findings — a dataset quirk that helped, a trap to avoid; the next similar task recalls them at intake, so the agent starts from experience instead of scratch. 🧠
  41. - **2026-06-22** — **Built-in literature search & idea novelty checks.** Arbor can now ground its research in prior work via the public [alphaXiv](https://www.alphaxiv.org) API — zero config, no search endpoint or key. Novelty-check any idea before you build it with `arbor idea-check "<your idea>"`, or let the Coordinator vet every new branch automatically. See [Literature Search & Novelty Checks](#-literature-search--novelty-checks). 🔎
  42. - **2026-06-18** — Arbor was featured by [VentureBeat](https://venturebeat.com/), one of the leading tech media outlets in the US: ["New AI optimization framework beats Claude Code and Codex by 2.5x on the same compute budget"](https://venturebeat.com/orchestration/new-ai-optimization-framework-beats-claude-code-and-codex-by-2-5x-on-the-same-compute-budget). 📰
  43. - **2026-06-12** — Arbor's native CLI runtime and Agent Skill Suite (Codex / Claude Code) are released. 🚀
  44. - **2026-06-11** — The Arbor paper is released on [arXiv](https://arxiv.org/abs/2606.11926). 🎉
  45. ## 💡 Why Arbor
  46. * **General-purpose optimization** — optimizes any task with a target to improve
  47. and a metric to measure, from model training to harness engineering to data synthesis.
  48. * **Long-horizon structured exploration** — the hypothesis-tree framework keeps
  49. results, failure modes, and distilled insights in the Idea Tree and propagates them
  50. upward, so later ideas start smarter instead of scrolling off.
  51. * **Real experiment discipline** — Executors iterate on a dev split, validate on a
  52. held-out test split, and only merge gains that clear a configurable margin — each in
  53. its own git worktree, so `main` is never touched until you merge.
  54. * **Literature-grounded ideas** — keyless search backends (alphaXiv + web) check an
  55. idea's novelty and prior art *before* spending compute, via node verdicts or
  56. `arbor idea-check`.
  57. * **Model and workflow flexibility** — Anthropic, OpenAI / Responses API, and
  58. OpenAI-compatible backends via LiteLLM (DeepSeek, Gemini, Qwen, vLLM, Ollama, …),
  59. usable as a native CLI or an Agent Skill Suite inside Codex / Claude Code.
  60. * **Steerable** — a live dashboard, read-only WebUI, optional human-in-the-loop
  61. review, and one-line domain plugins let you steer runs without touching the core.
  62. ## 🧩 Framework
  63. <p align="center">
  64. <img src="assets/framework.png" alt="Arbor framework" width="100%">
  65. </p>
  66. Arbor runs **two cooperating agents**:
  67. - **Coordinator** — the research director. It maintains the Idea Tree, drives the
  68. search via the *arbor cycle*, and dispatches experiments.
  69. - **Executor** — the research engineer. Given one idea, it faithfully implements the
  70. code changes, runs the experiment in an isolated git worktree, and reports evidence.
  71. Together they repeat a six-step **arbor cycle**:
  72. 1. **Observe** — the Coordinator re-grounds itself in the Idea Tree, reading the
  73. active frontier, constraints, ancestor insights, recent evidence, and current
  74. best artifact.
  75. 2. **Ideate** — it chooses a parent node and proposes child hypotheses that refine,
  76. correct, or extend what the tree has already learned.
  77. 3. **Select** — it chooses the most promising pending leaves to test, balancing
  78. the current best direction with unresolved alternatives.
  79. 4. **Dispatch** — selected hypotheses are sent to independent Executors, which
  80. implement them in fresh worktrees and evaluate them on the dev signal.
  81. 5. **Backpropagate** — Arbor records each result, score, insight, and branch, then
  82. abstracts the lesson upward so ancestor nodes and future ideas inherit it.
  83. 6. **Decide** — the Coordinator chooses whether to merge, prune, continue, leave a
  84. node pending, or stop, using held-out validation for merge decisions.
  85. ## 🚀 CLI And Skill Versions
  86. This repository includes three ways to use Arbor:
  87. | Version | Location | Best for | Needs an API key? |
  88. | --- | --- | --- | --- |
  89. | Native CLI runtime | Python package and `arbor` command | Real Arbor research runs, long experiments, dashboard, checkpoints, executor tools, merge/test discipline, plugins, reports | **Yes** — configure a provider/model in `arbor setup`. |
  90. | Keyless harness integration | `arbor install` + `arbor mcp` (or the Claude Code plugin) | Running Arbor **inside Claude Code / Codex using that harness's own model** — e.g. a Claude subscription plan, where there is no API key to give Arbor | **No** — the host agent's model does the reasoning; Arbor only supplies deterministic tools. |
  91. | Agent Skill Suite (standalone) | [`skills/`](skills/README.md) | The same harness flow without even installing the package — pure instructions + a stdlib fallback helper | **No** |
  92. If you can run the CLI and have an API key, the native runtime gives the most
  93. complete Arbor behavior: intake, Research Contract, live dashboard, EventBus,
  94. checkpoint/resume, executor dispatch, protected dev/test evaluation discipline,
  95. SearchAgent, plugins, and final report generation. If you only have a coding
  96. agent with a subscription model (no raw API key), use the **keyless harness
  97. integration** below — see [Use inside Claude Code or any harness](#-use-inside-claude-code-or-any-harness-no-api-key).
  98. The repo-root [`skills/`](skills/README.md) directory is a Codex/Claude Code
  99. skill suite. After installation, invoke `$arbor-research-agent` in Codex or
  100. `/arbor-research-agent` in Claude Code and describe your research objective as
  101. you would in Arbor. The skill suite performs Arbor-style clarification first
  102. when target, metric, data, permissions, budget, or run mode are unclear, then
  103. loads the orchestrator and phase skills. This is separate from the internal
  104. runtime skills stored under `src/skills/`.
  105. ---
  106. ## 📦 Install
  107. **Requirements:** Python ≥ 3.10 and Git. A virtual environment is recommended.
  108. ```bash
  109. pip install arbor-agent # or: uv pip install arbor-agent
  110. arbor doctor # verify PATH, git, API keys
  111. ```
  112. > Prefer a global command? `pipx install arbor-agent` makes `arbor` available everywhere.
  113. <details>
  114. <summary>Install from source (for development)</summary>
  115. ```bash
  116. git clone https://github.com/RUC-NLPIR/Arbor.git
  117. cd Arbor
  118. python -m venv .venv && source .venv/bin/activate # recommended
  119. pip install -e . # or: uv pip install -e .
  120. arbor doctor
  121. ```
  122. For the docs site, `pip install -e ".[docs]" && mkdocs serve`, or read them online
  123. via the **Docs** badge above.
  124. </details>
  125. ---
  126. ## 🔑 Use Inside Claude Code or Any Harness (No API Key)
  127. On a Claude **subscription plan** there is no API key to hand to a separate tool.
  128. Arbor's keyless integration solves this: **Arbor never calls an LLM** — your
  129. coding agent's own model drives the research loop, while Arbor contributes its
  130. durable Idea Tree, evaluation, git-worktree isolation, guarded merges, and
  131. reports as deterministic tools.
  132. **1. Install the skill suite** (no more manual directory copying):
  133. ```bash
  134. pip install arbor-agent
  135. arbor install # auto-detects the harness; also --claude / --codex / --project / --target <dir>
  136. ```
  137. **2. Register the keyless tool server** (optional but recommended — backs the
  138. skills with Arbor's *real* tree/eval/merge/report implementations):
  139. ```bash
  140. pip install "arbor-agent[mcp]" # the MCP server is an optional extra
  141. claude mcp add arbor -- arbor mcp # Claude Code; any MCP-capable harness works
  142. ```
  143. **Or do both in one step with the Claude Code plugin:**
  144. ```bash
  145. claude plugin marketplace add RUC-NLPIR/Arbor
  146. claude plugin install arbor # installs the skills + registers `arbor mcp`
  147. ```
  148. **3. Run it** from inside your project, in the coding agent:
  149. ```text
  150. /arbor-research-agent optimize this repo for <metric>. Ask before training, package installs, or B_test.
  151. ```
  152. **4. Watch progress in the browser** (read-only, also keyless). Either ask the
  153. agent to call the `open_dashboard` tool, or run it yourself:
  154. ```bash
  155. arbor web <run-name> # serves http://127.0.0.1:8765 for the session
  156. ```
  157. To remove the skills later: `arbor uninstall` (it only touches Arbor's own
  158. `arbor-*` directories). See [`skills/README.md`](skills/README.md) for the full
  159. skill-suite reference and the manual install steps.
  160. ---
  161. ## ⚡ Getting Started
  162. **See it work first — no API key, no config:**
  163. ```bash
  164. arbor replay --demo # watch a recorded run: the hypothesis tree grows live
  165. ```
  166. > Want a shareable version? `arbor replay --demo --html` writes a self-contained
  167. > interactive page (no server, no deps) you can open in a browser or attach anywhere.
  168. **Then run it on your own task:**
  169. ```bash
  170. arbor quickstart # ~2 min: a free key (Gemini/Groq) or a local model (Ollama)
  171. arbor # start an interactive session in the current directory
  172. ```
  173. Already have a provider? `arbor setup` runs the full provider / model / base_url / API key
  174. flow; `arbor doctor` diagnoses the install. Both `quickstart` and `setup` write
  175. `~/.arbor/config.yaml`, so day-to-day you can just run `arbor`
  176. with no flags. The first thing Arbor does is an **intake conversation** that turns your
  177. goal, target directory, metric, baseline, budget, dev/test discipline, and artifact
  178. paths into a one-screen **Arbor Research Contract**. Read/discuss requests stay in a
  179. scoped, read-only mode; experiment plans are staged and launch only after a later explicit
  180. confirmation. Once you confirm the resolved contract, the live dashboard takes over.
  181. ```bash
  182. # Point at a benchmark directory and a config
  183. arbor --cwd ./benchmark --config research_config.yaml
  184. # Give an initial goal up front; intake refines the rest
  185. arbor "improve validation score without touching the test split" --cwd ./benchmark
  186. # Small dry run
  187. arbor --cwd ./benchmark --config research_config.yaml --max-cycles 3
  188. ```
  189. During a run you can type `/status`, `/tree`, `/evidence`, `/branches`, `/cost`,
  190. `/pause`, `/resume`, `/report`, or `/abort`.
  191. ### Prepare a benchmark
  192. Your target directory should have:
  193. - a runnable evaluation script (e.g. `run_eval.py`),
  194. - evaluation data (ideally a **dev** split and a held-out **test** split), and
  195. - a clean git repository (no uncommitted changes).
  196. A minimal `research_config.yaml`:
  197. ```yaml
  198. # LLM/API live in `arbor setup`; project config is usually just the task and budget.
  199. task: >
  200. Optimize the agent's accuracy on the benchmark.
  201. Do NOT modify the evaluation harness or data files.
  202. coordinator:
  203. max_cycles: 10 # arbor cycles to explore
  204. max_depth: 2 # Idea Tree depth
  205. merge_threshold: 5.0 # min held-out % gain to merge into trunk
  206. ui:
  207. interaction_mode: review # auto | direction | review | collaborative
  208. executor:
  209. max_turns: 100
  210. ```
  211. A copy-pasteable example with every option lives in
  212. [`examples/research_config.example.yaml`](examples/research_config.example.yaml).
  213. ### Try the runnable example task
  214. If you just want to watch Arbor work end-to-end — **no API budget, no GPU** —
  215. [`examples/algotune_knn/`](examples/algotune_knn) is a tiny, self-contained
  216. benchmark modeled on [AlgoTune](https://algotune.io/). The task is to make a
  217. brute-force k-nearest-neighbours solver **faster** while producing the **same**
  218. output; the metric is the speedup over a reference implementation. It is pure
  219. NumPy, CPU-only, sub-second, and deterministic, with several genuine
  220. optimizations for the Idea Tree to discover.
  221. ```bash
  222. cp -r examples/algotune_knn /tmp/algotune_knn # run outside the Arbor checkout
  223. cd /tmp/algotune_knn
  224. git init -q && git add -A && git commit -qm baseline
  225. arbor
  226. ```
  227. In one 6-cycle run this drove the dev speedup from **1.01x → 7.77x** (held-out
  228. test **1.00x → 7.22x**). See [`examples/algotune_knn/README.md`](examples/algotune_knn/README.md)
  229. for the research contract and tuning knobs.
  230. ### Collect a benchmark from a request (experimental)
  231. You don't have to assemble a benchmark by hand. `arbor benchmark add` turns a
  232. one-line request into a runnable **draft task**: it finds the dataset/benchmark and,
  233. on an interactive terminal, asks you **which dataset** to use and **where the
  234. baseline comes from** — harvest an existing implementation, implement the method you
  235. described, or find one online — then acquires the data and brings up a draft
  236. (baseline + eval + `README` + provenance). It does *not* force-run the eval; a real
  237. run may need your served model / API key.
  238. ```bash
  239. # name a work, or give a goal + a method
  240. arbor benchmark add "get me the datasets WebThinker uses"
  241. arbor benchmark add "I want to climb GPQA with a self-consistency baseline"
  242. # or point straight at a repo / HF dataset (add --bringup to also build the baseline)
  243. arbor benchmark add https://github.com/owner/repo --name my-bench --bringup
  244. ```
  245. Drafting is automated; acceptance stays human. `arbor benchmark verify <dir>`
  246. structurally checks a pack, and `arbor benchmark list` indexes the zoo. See the
  247. [benchmark zoo overview](docs/zoo-overview.md) for the format and the full flow.
  248. ---
  249. ## 🧠 How It Works
  250. ### The arbor cycle
  251. Each cycle runs six steps:
  252. ```
  253. ① OBSERVE analyze current results and failure modes
  254. ② IDEATE propose 1–3 new ideas from the analysis and tree insights
  255. ③ SELECT pick the highest-priority idea to test
  256. ④ DISPATCH run an Executor on it in an isolated git worktree
  257. ⑤ BACKPROP record the result; abstract the insight up to ancestor nodes
  258. ⑥ DECIDE continue / merge into trunk / prune / stop
  259. ```
  260. ### The Idea Tree
  261. ```
  262. ROOT (baseline: 20%)
  263. ├── 1: Retrieval optimization [insight: "retrieval quality is the bottleneck"]
  264. │ ├── 1.1: Constraint decomposition + verification [40%, merged]
  265. │ ├── 1.2: Periodic re-read injection [40%, pruned — no net gain]
  266. │ └── 1.3: Answer-extraction tuning [35%, pruned]
  267. ├── 2: Multi-perspective search [insight: "search scaffolding hurts here"]
  268. │ └── 2.1: Breadth-first search [25%, pruned]
  269. └── 3: Code-level intervention [insight: "code-level > prompt-level"]
  270. ├── 3.1: Continuation injection [70%, merged]
  271. └── 3.2: ANSWER-tag extraction [45%, done]
  272. ```
  273. - **Depth 0 (Root):** the research objective and global insights.
  274. - **Depth 1:** research directions (paper-title-level ideas).
  275. - **Depth 2+:** concrete methods, implemented and tested by Executors.
  276. ### Git strategy & evaluation
  277. Each Executor works in its own worktree on a dedicated branch. Verified improvements merge
  278. into a per-run `trunk`; you promote `trunk` into `main` only when satisfied
  279. (`git merge research/run_xxx/trunk`). Executors iterate on a **dev** split, but a change is
  280. kept only if it clears a margin on the **held-out test** split — guarding against
  281. overfitting.
  282. ### Human-in-the-loop
  283. Set `ui.interaction_mode` (or `--interaction-mode`) to choose how much you steer:
  284. | Mode | Behavior |
  285. | --- | --- |
  286. | `auto` | Fully autonomous. |
  287. | `direction` | Asks you where to go next at ideation. |
  288. | `review` | Pauses before each node and Executor. |
  289. | `collaborative` | `direction` + `review`. |
  290. In autonomous mode, a text-only response that announces an immediate next action is
  291. prompted to execute it before the run is considered finished.
  292. When paused, your input opens an isolated discussion with a read-only companion — it never
  293. pollutes the Coordinator's context. See [`docs/`](docs/index.md) for the full method.
  294. ---
  295. ## ⚙️ Configuration
  296. LLM access is configured once with `arbor setup` (stored in `~/.arbor/config.yaml`) via a
  297. single `provider` field:
  298. | `provider` | Use it for |
  299. | --- | --- |
  300. | `auto` *(default)* | Let Arbor pick. It probes your endpoint's OpenAI **Responses** API and uses it when available (reasoning chain preserved), otherwise falls back to chat completions; Claude models use the native Anthropic API. The detected backend is frozen into the config. |
  301. | `openai-responses` | OpenAI / o-series models via the Responses API (encrypted reasoning chain preserved across turns). |
  302. | `openai-chat` | Any OpenAI-compatible chat-completions endpoint — DeepSeek / Qwen / GLM / vLLM / Ollama / local gateways. |
  303. | `anthropic` | Claude via the native Anthropic Messages API (signed thinking + prompt caching). |
  304. Most users just run `arbor setup`, keep `auto`, and fill in `model` + `base_url`. Keys come
  305. from the environment or the config; per-project task and budget settings live in
  306. `research_config.yaml`. See the
  307. [configuration guide](https://RUC-NLPIR.github.io/Arbor/docs/configuration/) and
  308. [`examples/research_config.example.yaml`](examples/research_config.example.yaml) for every
  309. option.
  310. ---
  311. ## 🧰 CLI Reference
  312. Day to day you only need `arbor`:
  313. | Command | What it does |
  314. | --- | --- |
  315. | `arbor` | Start an interactive research session. |
  316. | `arbor --continue` (`-C`) | Resume the most recent unfinished planning conversation in this directory (like `claude -c`). |
  317. | `arbor replay --demo` | Replay a bundled sample run in the live dashboard — no API key needed. Add `--html` for a shareable browser page. |
  318. | `arbor replay <session>` | Replay any past run's `events.jsonl` from its timeline (`--html` to export an interactive page). |
  319. | `arbor quickstart` | Get running fast with a free key (Gemini/Groq) or a local model (Ollama). |
  320. | `arbor setup` | Configure provider / model / keys → `~/.arbor/config.yaml`. |
  321. | `arbor report <session>` | Re-render `REPORT.md` for a past session. |
  322. | `arbor idea-check "<idea>"` | Novelty / prior-art check for one idea against alphaXiv (built in on Python ≥ 3.12). |
  323. | `arbor export <session> [output]` | Export a past session to self-contained HTML, or JSONL when `output` ends in `.jsonl`. |
  324. | `arbor doctor` | Diagnose install, PATH, git, and API keys. |
  325. | `arbor version` | Print the installed version. |
  326. | `arbor install` / `arbor uninstall` | Install/remove the Agent Skill suite into a coding agent (`--claude` / `--codex` / `--project` / `--target`). |
  327. | `arbor mcp` | Run Arbor's keyless deterministic tools as an MCP server (needs the `[mcp]` extra). |
  328. | `arbor web <session>` | Open a read-only browser monitor for a session (works without a live run). |
  329. Lower-level entry points (`run-research`, `coordinator`, `executor`, `review-research`)
  330. remain for debugging — see the [CLI reference](https://RUC-NLPIR.github.io/Arbor/docs/cli/).
  331. ---
  332. ## 🔎 Literature Search & Novelty Checks
  333. Good research starts by knowing what already exists. Arbor ships a **zero-config
  334. search backend** over the public [alphaXiv](https://www.alphaxiv.org) API — no
  335. self-hosted search service, no alphaXiv key — so it can survey related work and
  336. judge an idea's novelty. The verdict is lightweight and decision-oriented: a short
  337. summary of the space, the closest related papers, a `novel` / `partial-overlap` /
  338. `prior-art-exists` assessment, and the concrete overlap risks.
  339. ### Enable it
  340. Nothing to install — the backend ships with Arbor by default on **Python ≥ 3.12**
  341. (it bundles `alphaxiv-py`). It reuses your existing Arbor LLM credentials, so
  342. `arbor idea-check` works out of the box. (On Python 3.10/3.11 `alphaxiv-py` is
  343. unavailable and the backend degrades with a clear message.)
  344. ### Use it three ways
  345. **1. Standalone — check one idea, no run required.** The fastest way to sanity-check
  346. a direction before you invest in it:
  347. ```bash
  348. arbor idea-check "Use entity-relation scratchpads to improve multi-hop QA"
  349. arbor idea-check "tree search over plans for code generation" --json # raw JSON
  350. ```
  351. **2. Pre-experiment — vet every idea automatically.** Turn on `auto_search_on_add`
  352. and the Coordinator dispatches a background novelty check the moment a new idea is
  353. added to the tree; the verdict lands in that node's `related_work` field *before* an
  354. Executor ever runs, so Arbor can revise or prune non-novel ideas instead of spending
  355. compute on them:
  356. ```yaml title="research_config.yaml"
  357. search:
  358. enabled: true
  359. builtin_backend: alphaxiv # none | alphaxiv
  360. auto_search_on_add: true # novelty-check each new idea before running it
  361. ```
  362. **3. Post-experiment — annotate what worked.** By default (`auto_search_on_add:
  363. false`) Arbor still surveys related work for ideas that *beat the trunk*, attaching
  364. prior-art context right before a merge decision — so wins arrive with citations.
  365. ### Turn it off
  366. In a run it is **off by default**: `builtin_backend` is `none`, so unless you set
  367. `search.builtin_backend: alphaxiv` (and, for pre-experiment checks,
  368. `auto_search_on_add: true`), Arbor never touches the network for literature.
  369. `arbor idea-check` is opt-in by nature — you only pay when you call it. Prefer your
  370. own search service? The bring-your-own-endpoint path still works via
  371. `search.web_search_endpoint` for a self-hosted BrowseComp-style backend.
  372. See the [configuration guide](https://RUC-NLPIR.github.io/Arbor/docs/configuration/)
  373. and [`arbor idea-check`](https://RUC-NLPIR.github.io/Arbor/docs/cli/#arbor-idea-check)
  374. for every option.
  375. ---
  376. ## 🔌 Plugins & Skills
  377. A single line retargets the agent to a new domain — evaluation protocol, protected
  378. data directories, required outputs, and timeout presets all come from the plugin:
  379. ```yaml
  380. plugin: mle_kaggle # switches to Kaggle/MLE mode
  381. ```
  382. A plugin is one YAML file (prompt-injection points + config overrides + profiles +
  383. lifecycle hooks + an eval contract); a Skill is a markdown playbook the agent loads on
  384. demand at runtime. A copy-pasteable Kaggle config lives in
  385. [`examples/kaggle_config.example.yaml`](examples/kaggle_config.example.yaml).
  386. ---
  387. ## 💾 Output & Resume
  388. Each run writes a session directory with `REPORT.md`, `events.jsonl`, `run_stats.json`, the
  389. Idea Tree, and per-experiment artifacts under `.arbor/sessions/`. Runs are resumable —
  390. interrupt with `Ctrl+C` and continue later with `--resume`; Arbor reloads the Idea Tree and
  391. picks up where it left off.
  392. ```bash
  393. arbor report .arbor/sessions/<run_name> # re-render a past report
  394. arbor export <run_name> # write .arbor/sessions/<run_name>/arbor-session-<run_name>.html
  395. arbor export <run_name> session.jsonl # export a JSONL artifact bundle
  396. arbor --resume --run-name <run_name> # continue an interrupted run
  397. ```
  398. ---
  399. ## 📊 Results
  400. Arbor was evaluated as a single controller across model training, harness engineering,
  401. and data synthesis — only the material, objective, evaluator, and budget change. It
  402. wins the held-out test on all six tasks against strong single-agent baselines.
  403. | Task | Direction | Initial | Codex | Claude Code | **Arbor** | Gain |
  404. | --- | --- | --- | --- | --- | --- | --- |
  405. | Optimizer Design | steps ↓ | 3325 | 3325 | 3287.5 | **3237.5** | +2.63% |
  406. | Architecture Design | loss ↓ | 1.098 | 1.083 | 1.033 | **1.028** | +6.38% |
  407. | Terminal-Bench 2.0 | pass ↑ | 69.81 | 73.59 | 71.70 | **77.36** | +7.55 |
  408. | BrowseComp | acc ↑ | 45.33 | 50.00 | 53.33 | **67.67** | +22.34 |
  409. | Search-Agent Data | gap ↑ | 5.00 | 9.00 | 12.00 | **18.00** | +13.0 |
  410. | Math-Reasoning Data | gap ↑ | 1.04 | 6.25 | 8.33 | **20.83** | +19.79 |
  411. On **MLE-Bench Lite** with GPT-5.5, Arbor reaches **86.36% Any-Medal** (100% valid
  412. submissions, 95.45% above median, 77.27% gold). See the [paper](https://arxiv.org/pdf/2606.11926)
  413. for full protocols and ablations.
  414. ---
  415. ## 🗂️ Project Structure
  416. The code lives in `src/` and is imported as the `arbor` package.
  417. ```
  418. src/ # the `arbor` package
  419. ├── core/ Shared infrastructure: ReAct loop, tools, LLM providers, context mgmt
  420. ├── executor/ Executor agent + `executor` CLI
  421. ├── coordinator/ Coordinator agent, Idea Tree, orchestrator, coordinator tools
  422. ├── cli/ `arbor` CLI: intake, live dashboard, setup, doctor, config
  423. ├── events/ Typed event bus and payloads
  424. ├── report/ Report generation
  425. ├── webui/ Read-only run-monitoring web server
  426. ├── plugins/ Domain plugins (e.g. mle_kaggle.yaml)
  427. ├── skills/ On-demand markdown playbooks
  428. ├── dashboard.py HTML dashboard generator
  429. ├── run.py `run-research` CLI
  430. └── review.py `review-research` CLI
  431. ```
  432. ---
  433. ## 🙏 Acknowledgements
  434. Arbor is built on the excellent foundation of
  435. [claw-code](https://github.com/ultraworkers/claw-code).
  436. claw-code is an open-source Rust reimplementation of Claude Code. It provided
  437. the REPL framework, tool-calling infrastructure, and cross-platform compilation
  438. that made Arbor's CLI possible. Huge thanks to the ultraworkers team for their
  439. outstanding work.
  440. 🔗 claw-code: https://github.com/ultraworkers/claw-code
  441. ---
  442. ## 📚 Citation
  443. ```bibtex
  444. @misc{jin2026arbor,
  445. title = {Toward Generalist Autonomous Research via Hypothesis-Tree Refinement},
  446. author = {Jiajie Jin and Yuyang Hu and Kai Qiu and Qi Dai and Chong Luo and
  447. Guanting Dong and Xiaoxi Li and Tong Zhao and Xiaolong Ma and
  448. Gongrui Zhang and Zhirong Wu and Bei Liu and Zhengyuan Yang and
  449. Linjie Li and Lijuan Wang and Hongjin Qian and Yutao Zhu and Zhicheng Dou},
  450. year = {2026},
  451. eprint = {2606.11926},
  452. archivePrefix = {arXiv},
  453. url = {https://arxiv.org/abs/2606.11926}
  454. }
  455. ```
  456. ---
  457. ## Star History
  458. <picture>
  459. <source
  460. media="(prefers-color-scheme: dark)"
  461. srcset="https://star-history.dera.page/svg?repos=RUC-NLPIR/Arbor&type=Date&theme=dark"
  462. />
  463. <source
  464. media="(prefers-color-scheme: light)"
  465. srcset="https://star-history.dera.page/svg?repos=RUC-NLPIR/Arbor&type=Date"
  466. />
  467. <img
  468. alt="Star History Chart"
  469. src="https://star-history.dera.page/svg?repos=RUC-NLPIR/Arbor&type=Date"
  470. />
  471. </picture>
  472. ---
  473. ## 📄 License
  474. Released under the [Apache License 2.0](LICENSE).
  475. ---
  476. Built at the Gaoling School of Artificial Intelligence, Renmin University of China, and
  477. Microsoft Research.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.