Skip to content
Back to search

multi-agents

MarketplaceWorks with Claude CodeGitHub

Multi-agent teams for real work: hand a task to a team of coding agents and review the patch.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 20 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/BichengWang/multi-agents with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/BichengWang/multi-agents'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/bichengwang--multi-agents/badge.svg)](/listings/bichengwang--multi-agents)

HTML

<a href="/listings/bichengwang--multi-agents"><img src="/api/listings/bichengwang--multi-agents/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Requirements: Python 3.10+, [uv](https://docs.astral.sh/uv/), git, and at least one logged-in coding CLI (`claude` or `codex`). CLI harnesses use your existing login; no API keys needed. The package is `multi-agent` on PyPI and the command is `multi-agents`: run it with `uvx` as below, or keep it installed with `uv tool install multi-agent` (or `pipx install multi-agent`).

uvx --from multi-agent multi-agents run \
  --repo ~/code/myproject \
  --team 'best:workers=claude@haiku+codex@gpt-6-luna' \
  "Make the tests in test_slug.py pass without editing the tests."

Real output from that command on a small repository:

Team: best:workers=claude@haiku+codex@gpt-6-luna,judge=claude@haiku
role   ┃ harness@model           ┃ tokens in/cached/out ┃    cost ┃ secs
━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━
worker │ claude@claude-haiku-5-5 │       236k/201k/1.5k │ $0.0096 │ 34.8
judge  │ claude@claude-haiku-5-5 │       194k/157k/2.6k │ $0.0102 │ 42.8
worker │ codex@gpt-6-luna        │         107k/94k/667 │ $0.0025 │ 31.6
judge  │ claude@claude-haiku-5-5 │       192k/149k/3.8k │ $0.0119 │ 52.3
Total: 729k/604k/8.5k tokens, cost $0.0342, 85.0s
scores: [8.0, 8.0]
chosen: 1
Files changed: 1
  slug.py  +6 -3

The run prints the exact `git apply` command for the final patch and saves a JSON record outside your repository (`~/.cache/multi-agents/runs/`).

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 18 lines
  1. {
  2. "name": "multi-agents",
  3. "description": "Multi-agent teams for real work: hand a task to a team of coding agents and review the patch.",
  4. "owner": {
  5. "name": "BichengWang",
  6. "url": "https://github.com/BichengWang"
  7. },
  8. "plugins": [
  9. {
  10. "name": "multi-agents",
  11. "source": "./plugins/multi-agents",
  12. "description": "Hand a coding task to a team of coding agents (Claude Code, Codex) in isolated git worktrees and review the patch they return.",
  13. "version": "0.1.1",
  14. "category": "development",
  15. "tags": ["multi-agent", "teams", "code-review"]
  16. }
  17. ]
  18. }
README.md · README · 227 lines
  1. <div align="center" markdown="1">
  2. # multi-agents
  3. **Multi-agent teams for real work: measured, not assumed.**
  4. [![PyPI](https://img.shields.io/pypi/v/multi-agent?color=2dd4bf&label=PyPI)](https://pypi.org/project/multi-agent/)
  5. [![Python](https://img.shields.io/pypi/pyversions/multi-agent?color=60a5fa)](https://pypi.org/project/multi-agent/)
  6. [![Docs](https://img.shields.io/badge/docs-site-a78bfa)](https://bichengwang.github.io/multi-agents/)
  7. [![License: MIT](https://img.shields.io/badge/license-MIT-4ade80)](https://github.com/BichengWang/multi-agents/blob/main/LICENSE)
  8. [Docs](https://bichengwang.github.io/multi-agents/) ·
  9. [Quickstart](#quickstart) ·
  10. [Teams](#teams) ·
  11. [Benchmark](#benchmark) ·
  12. [Roadmap](https://github.com/BichengWang/multi-agents/blob/main/docs/ROADMAP.md)
  13. Featured in Tech Forum: [Does a Team of AI Coding Agents Beat One?](https://techforum.ca/articles/bicheng-kenneth-wang-multi-agents-coding-agent-teams)
  14. <img src="https://raw.githubusercontent.com/BichengWang/multi-agents/main/docs/assets/architecture.svg" alt="multi-agents architecture: entry points feed a team runner (solo, review, best, plan topologies; Claude Code, Codex, SDK and check harnesses; isolated workspaces; cost budget) that returns a patch and a run record, and a benchmark that runs teams against solo agents to answer the north star" width="100%">
  15. </div>
  16. > **North star:** most agent orchestrators skip one question, and the benchmark here answers it:
  17. > **did the team beat a single agent on the same task, and what did the extra agents cost?**
  18. Run a team of coding agents on a task in your repository: a single agent, a coder ⇄ reviewer
  19. loop, or several agents with a judge. Workers are the coding CLIs you already use (Claude Code,
  20. Codex) or a built-in agent that can use any model through the OpenAI Agents SDK. Each worker
  21. gets its own git worktree. You get one patch back, plus an itemized record of every agent call:
  22. tokens, cost and seconds. Your working tree is untouched unless you pass `--apply`.
  23. ## Why measure
  24. The evidence on teams of coding agents points both ways:
  25. - A 260-configuration study found multi-agent setups 2–15% *worse* than single agents on
  26. SWE-bench Verified ([Towards a Science of Scaling Agent Systems](https://arxiv.org/abs/2512.08296)).
  27. - A team with a coordinator, a researcher, an engineer and a reviewer reached 72.2% on SWE-bench
  28. Verified, 7.2 points *better* than mini-SWE-agent but only 0.4 above the strongest single-agent
  29. baseline, with different models per role ([Agyn](https://arxiv.org/abs/2602.01465)). A
  30. decentralized pair, a patch editor and a validator with isolated context, gained 3.6 and 8.4
  31. points over mini-SWE-agent ([icat-agent](https://arxiv.org/abs/2606.25514)).
  32. - The best-known coding benchmark is contaminated: OpenAI stopped reporting SWE-bench Verified
  33. in February 2026.
  34. So the answer depends on the topology, the models and the task, and published numbers may be
  35. contaminated. This project measures it on tasks created after the models' training cutoffs, with
  36. paired comparisons and cost.
  37. ## Quickstart
  38. Requirements: Python 3.10+, [uv](https://docs.astral.sh/uv/), git, and at least one logged-in
  39. coding CLI (`claude` or `codex`). CLI harnesses use your existing login; no API keys needed.
  40. The package is `multi-agent` on PyPI and the command is `multi-agents`: run it with `uvx` as below,
  41. or keep it installed with `uv tool install multi-agent` (or `pipx install multi-agent`).
  42. ```bash
  43. uvx --from multi-agent multi-agents run \
  44. --repo ~/code/myproject \
  45. --team 'best:workers=claude@haiku+codex@gpt-6-luna' \
  46. "Make the tests in test_slug.py pass without editing the tests."
  47. ```
  48. Real output from that command on a small repository:
  49. ```text
  50. Team: best:workers=claude@haiku+codex@gpt-6-luna,judge=claude@haiku
  51. role ┃ harness@model ┃ tokens in/cached/out ┃ cost ┃ secs
  52. ━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━
  53. worker │ claude@claude-haiku-5-5 │ 236k/201k/1.5k │ $0.0096 │ 34.8
  54. judge │ claude@claude-haiku-5-5 │ 194k/157k/2.6k │ $0.0102 │ 42.8
  55. worker │ codex@gpt-6-luna │ 107k/94k/667 │ $0.0025 │ 31.6
  56. judge │ claude@claude-haiku-5-5 │ 192k/149k/3.8k │ $0.0119 │ 52.3
  57. Total: 729k/604k/8.5k tokens, cost $0.0342, 85.0s
  58. scores: [8.0, 8.0]
  59. chosen: 1
  60. Files changed: 1
  61. slug.py +6 -3
  62. ```
  63. The run prints the exact `git apply` command for the final patch and saves a JSON record
  64. outside your repository (`~/.cache/multi-agents/runs/`).
  65. ## Teams
  66. | Spec | What happens |
  67. |---|---|
  68. | `solo:worker=H` | One agent works on the task. This is the baseline. |
  69. | `review:coder=H,reviewer=H[,rounds=N]` | The coder works; the reviewer scores the patch in a separate copy and sends feedback; the coder resumes its session and revises. Returns the best-scoring patch, not the last one. |
  70. | `best:workers=H+H[+H...][,judge=H]` | Workers run in parallel, each in its own worktree, possibly different CLIs and models. The judge scores each patch and the best one wins. |
  71. | `plan:planner=H,workers=H[+H...][,integrator=H][,max_tasks=N][,replans=N][,reviewer=H][,rounds=N]` | A planner splits the task into independent subtasks, workers solve them in parallel worktrees, and the patches are merged; an integrator resolves conflicts and gets one fix pass if the check fails. Subtasks whose worker failed or changed nothing go back to the planner (`replans`, default 1, 0 = never) for new subtasks that rotated workers solve. With `reviewer` (and `rounds`, default 3) every subtask is a coder ⇄ reviewer loop in its own worktree and its best patch is merged. |
  72. `H` is `claude`, `codex` or `sdk`, optionally with `@model`, for example `claude@haiku`,
  73. `codex@gpt-6-luna` or `sdk@litellm/openrouter/<model>`. `check` (the `--check` command) can be the
  74. reviewer or the judge: a test-driven loop with no LLM reviewer. Details:
  75. [team runner guide](https://github.com/BichengWang/multi-agents/blob/main/agent/teams/README.md).
  76. ## From issue to pull request
  77. `--check` runs your tests (or build, or lint) against the final patch in a separate worktree and
  78. records pass/fail; with an LLM judge, candidates that fail it rank below those that pass. `--issue`
  79. takes the task from a GitHub issue, and `--pr` commits the result to a new branch and opens a draft
  80. pull request with the run summary, without touching your checkout.
  81. ```bash
  82. multi-agents run --repo . --issue 42 --pr \
  83. --team 'plan:planner=claude@haiku,workers=claude@haiku+codex@gpt-6-luna' \
  84. --check 'python3 -m unittest -q'
  85. ```
  86. A real `plan` run on a repository with two unrelated bugs (without `--issue`/`--pr`):
  87. ```text
  88. role ┃ harness@model ┃ tokens in/cached/out ┃ cost ┃ secs
  89. planner │ claude@claude-haiku-5-5 │ 187k/150k/2.0k │ $0.0101 │ 16.8
  90. worker │ claude@claude-haiku-5-5 │ 141k/104k/510 │ $0.0088 │ 8.7
  91. worker │ codex@gpt-6-luna │ 55k/37k/349 │ $0.0024 │ 9.7
  92. Total: 385k/291k/2.9k tokens, cost $0.0212, 26.8s
  93. subtask 1 [applied]: Fix word_count in textstats.py (claude@haiku)
  94. subtask 2 [applied]: Fix format_cents in money.py (codex@gpt-6-luna)
  95. check: passed (exit 0, 0.0s) python3 -m unittest -q
  96. ```
  97. Issue text is untrusted input to the agents: don't combine `--issue` on public repositories with
  98. `--permissions bypass` or the `sdk` harness.
  99. ## Use it from your coding agent
  100. This repository is also a plugin marketplace for both coding CLIs, so a running agent can hand a
  101. task to a team and then show you the patch. Install it once:
  102. ```bash
  103. # Claude Code
  104. claude plugin marketplace add BichengWang/multi-agents
  105. claude plugin install multi-agents@multi-agents
  106. # Codex
  107. codex plugin marketplace add BichengWang/multi-agents
  108. codex plugin add multi-agents@multi-agents
  109. ```
  110. Then, inside Claude Code (the optional spec comes first; the default is a cheap coder ⇄ reviewer
  111. pair that pairs the other CLI with the one you are in):
  112. ```text
  113. /multi-agents:team make the tests pass
  114. /multi-agents:team best:workers=claude@haiku+codex@gpt-6-luna fix the flaky retry in client.py
  115. ```
  116. In Codex, ask for the skill by name, for example `Use $multi-agents:multi-agents to make the tests
  117. pass`. Either way the agent runs the published CLI, shows you the summary and the patch, and applies
  118. it only after you say so. The skill is in
  119. [`plugins/multi-agents`](https://github.com/BichengWang/multi-agents/tree/main/plugins/multi-agents).
  120. ## Safety and cost
  121. - **Permissions.** By default (`--permissions safe`) each CLI runs inside its own sandbox: edits
  122. and test commands are confined to the worktree. `--permissions bypass` turns those checks off and
  123. is meant for disposable machines and containers only. The `sdk` harness has no permission system.
  124. - **Cost.** Costs come from the CLI's own report or from list prices with a cited source. A run
  125. that cannot be priced shows as unpriced and makes the total unpriced; nothing is guessed.
  126. - **Budget.** `--max-cost USD` stops new agent runs once that much has been spent; the team then
  127. returns the patch it has so far. It is a stop on new work, not a ceiling: runs already in flight
  128. finish (parallel workers run side by side), so the total can overshoot by their cost. An unpriced
  129. run cannot be counted, so after one the budget refuses further runs. The CLI therefore rejects
  130. a team with an unpriced harness before it starts, unless you pass `--allow-unpriced`, which counts
  131. those runs as $0 and leaves their spend outside the limit. The record and summary show the
  132. spend and any refused runs.
  133. ## Benchmark
  134. The benchmark runs team specs against solo agents on GitHub issues opened after the models'
  135. training cutoffs ([SWE-bench-Live MultiLang](https://huggingface.co/datasets/SWE-bench-Live/MultiLang):
  136. Rust, Java, Go, C/C++, C#, JS/TS), each in a fresh sandbox (Modal, or `--backend docker` on your
  137. own Docker daemon) with git history stripped. Patches are graded with the dataset's official
  138. hidden tests. The report gives resolve rate with 95%
  139. confidence intervals, paired wins and losses against the solo baseline on the same tasks (exact
  140. McNemar test), cost per task and per resolved task, and wall time. Timeouts are graded rather than
  141. dropped, so slow teams cannot shrink the comparison to the easy tasks.
  142. ```bash
  143. uv sync --locked --extra bench
  144. python -m agent.bench tasks --since 2026-07-01 --limit 5 --seed 7 # no credentials needed
  145. ```
  146. Setup, methodology, cost estimates and limitations:
  147. [benchmark guide](https://github.com/BichengWang/multi-agents/blob/main/agent/bench/README.md).
  148. **First results (October 2026):** on 48 fresh tasks, Claude Haiku 5.5 resolved 27 solo, 29 with a
  149. reviewer and 28 as best of 3, at $0.061, $0.098 and $0.17 per task. The teams' one or two extra
  150. tasks are not significant (paired p = 0.5 and 1.0): the reviewer approved 47 of 48 final patches,
  151. and the judge's top scores were tied on 32 of 48 tasks. Table, analysis and data:
  152. [pilot results](https://github.com/BichengWang/multi-agents/blob/main/docs/pilot-2026-10.md).
  153. ## Status
  154. | Area | State |
  155. |---|---|
  156. | Team runner (`solo`, `review`, `best`, `plan`), Claude Code and Codex harnesses | ✅ Working; smoke-tested with real CLIs |
  157. | `--check`, `--issue`, `--pr`, plugins for both CLIs | ✅ Working; smoke-tested end to end |
  158. | `--max-cost` run budget; `plan` re-planning (`replans`) and per-subtask review (`reviewer`) | ✅ Working; smoke-tested with real CLIs |
  159. | SDK harness (any model) | 🧪 Implemented and tested offline; not yet run against a live model |
  160. | Benchmark | ✅ Runs live on Modal (also detached, `remote`) and on local Docker |
  161. | Results | ✅ [Pilot](https://github.com/BichengWang/multi-agents/blob/main/docs/pilot-2026-10.md): Haiku 5.5, 48 tasks × solo, review, best of 3 |
  162. Plans are in [docs/ROADMAP.md](https://github.com/BichengWang/multi-agents/blob/main/docs/ROADMAP.md).
  163. ## Development
  164. ```bash
  165. uv sync --locked --extra dev
  166. make test # ruff + offline tests, no API keys needed
  167. make check # also the GitHub workflow and attribution tests (Node.js 22+)
  168. make docs # build the docs site with --strict (DOCS=serve to preview it locally)
  169. ```
  170. The Python tests ignore your global and system git configuration (hooks, signing, aliases).
  171. Reusable orchestration patterns (`RefineLoop`, `BestOfN`, `SequentialWorkflow`, `Router`) are in
  172. [`agent/workflows`](https://github.com/BichengWang/multi-agents/blob/main/agent/workflows/README.md).
  173. Pre-pivot experiments are in this repository's history at the
  174. [`pre-pivot`](https://github.com/BichengWang/multi-agents/tree/pre-pivot) tag.
  175. ## License
  176. [MIT](https://github.com/BichengWang/multi-agents/blob/main/LICENSE)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.