agent-orchestra
Use when three or more AI agents are available and a task needs coordinated execution, or when work must be handed to another agent, session or tool. It ships the coordination rules as ENFORCED invariants rather than advice: one writer per resource (a conflicting claim is refused and names the holder), a verifier that is never the implementer and whose PASS must carry criterion-linked evidence, a failed verification that blocks both done and dispatch to evidence, three failed rounds that block the task outright, an external action that cannot be dispatched until an approval covering it is recorded, a domain review by a third party for consequential work, rigorous E1-E4 evidence grading with enforced floors, capability-scored role assignment from a roster of any size (any agent names), a shared handoff record with a checker, a dependency-free task-queue CLI, an important-task intake gate, and a coordination anti-pattern guide. One command - scripts/gate.mjs - runs sixteen checks and injects twelve known faults it must reject before reporting PASS. Activation is configurable: global, keyword-triggered (three-valued: engage / surface / silent), or manual. Do not use for: container or infrastructure orchestration (k8s, docker-compose, service meshes), ordinary job queues and schedulers, comparing or evaluating AI models against each other, or splitting work between people. Those share vocabulary with this skill and none of them is multi-agent coordination.
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/3338902669-ops--agent-orchestra)HTML
<a href="/listings/3338902669-ops--agent-orchestra"><img src="/api/listings/3338902669-ops--agent-orchestra/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
**One repository, one skill.** Copy the contents of this repository into the skill directory your host supports - `SKILL.md`, `references/`, `config/`, and `scripts/` if you want the queue. There is no companion package and no second skill to install.
Optionally, the one-click installers perform the same single-skill copy into common skill directories.
Windows PowerShell:
powershell -ExecutionPolicy Bypass -File scripts/install.ps1
Linux/macOS shell:
./scripts/install.sh
Validate your config locally:
node scripts/validate-config.mjs
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
SKILL.md · Manifest · 319 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- ---
- name: agent-orchestra
- description: >-
- Use when three or more AI agents are available and a task needs coordinated execution, or when work
- must be handed to another agent, session or tool. It ships the coordination rules as ENFORCED
- invariants rather than advice: one writer per resource (a conflicting claim is refused and names
- the holder), a verifier that is never the implementer and whose PASS must carry criterion-linked
- evidence, a failed verification that blocks both done and dispatch to evidence, three failed rounds
- that block the task outright, an external action that cannot be dispatched until an approval
- covering it is recorded, a domain review by a third party for consequential work, rigorous E1-E4
- evidence grading with enforced floors, capability-scored role assignment from a roster of any size
- (any agent names), a shared handoff record with a checker, a dependency-free task-queue CLI, an
- important-task intake gate, and a coordination anti-pattern guide. One command - scripts/gate.mjs -
- runs sixteen checks and injects twelve known faults it must reject before reporting PASS. Activation is
- configurable: global, keyword-triggered (three-valued: engage / surface / silent), or manual.
- Do not use for: container or infrastructure orchestration (k8s, docker-compose, service meshes),
- ordinary job queues and schedulers, comparing or evaluating AI models against each other, or
- splitting work between people. Those share vocabulary with this skill and none of them is
- multi-agent coordination.
- ---
- # Agent Orchestra
- > 中文对照(不随 skill 常驻加载):`references/zh-contrast.md`
- One protocol for two axes of multi-agent work:
- - **In-task coordination** - who does what right now, who may write, who verifies, and what counts as done.
- - **Cross-session handoff** - how the state survives when the work moves to another agent, another session or another tool.
- Both axes share the same hard rules: one writer per resource, a verifier that is never the implementer, graded evidence, and a shared record that outranks any scheduler. It makes coordination failures visible and recoverable. It cannot promise infallibility, and it never replaces user authorization.
- ## Activation (when to use this skill)
- Configure in `config/agents.example.yaml` under `activation:`.
- | mode | behavior | token cost |
- |---|---|---|
- | global | engages for every task | highest (always loaded) |
- | keyword | engages only when task text matches keywords | default; low |
- | manual | engages only when the user explicitly invokes it | lowest |
- 1. Read the activation block from `config/agents.example.yaml`.
- 2. global -> engage. manual -> engage only when the user asked for multi-agent work or a handoff.
- 3. keyword -> run `node scripts/detect-trigger.mjs --text "<task text>"` and read the exit code:
- | exit | label | meaning | what to do |
- |---|---|---|---|
- | **0** | ENGAGED | a curated, self-anchored keyword phrase matched **and** no comparison marker is present | use the skill |
- | **3** | POSSIBLE | coordination phrasing without such a keyword - including every comparison request that also describes coordinating work | decide from the sentence yourself, or ask - **do not treat it as a hit or a miss** |
- | **1** | NOT_ENGAGED | no signal, an exclude keyword, or a plain comparison ("评估这两个模型的准确率") | stay silent |
- Exit 3 exists because seven adversarial verification rounds proved that no word list or character
- window can separate those two readings; each patch simply moved the hole. So the filter does not
- guess. It decides the unambiguous cases and **surfaces** the rest.
- Exclude keywords win over any hit.
- What the keyword layer is, honestly: a **conservative deterministic filter**, not a full
- understanding of intent. Unambiguous phrases engage on their own. Anything a pattern catches must
- also satisfy a **conjunction**: the sentence has to name an agent-ish subject (an agent word, a
- model name) **and** describe an act of coordinating. That conjunction is what keeps
- "两个同事同时改同一个文件" and "两个实习生互相覆盖对方的代码" silent - they share the phrasing and
- the vocabulary, and they are people, not agents. A sentence-level veto then removes requests that
- are really about comparing or choosing between models.
- Adversarial verification rounds kept breaking earlier versions of this table; every case they
- found is now a permanent entry in `scripts/activation-cases.json`, which the acceptance test and
- any verification script both read. What the
- layer does NOT do is understand arbitrary phrasing: an unusual way of asking may stay silent, and
- that is the intended failure direction - one missed phrase costs a nudge, a false engage
- reorganises work that has nothing to do with agents. The description above is what the host model
- routes on, and an explicit request always wins: if a user says to use this skill, no keyword check
- applies.
- ## Seven non-negotiables
- 1. **One primary writer per file or resource.** Everyone else is read-only or writes isolated artifacts until a recorded handoff promotes them.
- 2. **Verifier != implementer.** The verifier is the best-suited eligible agent by capability score, never a fixed name, and never the agent who wrote the code. A self-check is E3, not a verification result.
- 3. **The verification gate cannot be walked past.** A failed verification returns the task to `implement`; the task can then neither reach `done` nor be dispatched to `evidence` until a verify passes or a human records an override with an approver, a scope and a reason. The failed result stays on the record.
- 4. **Read before acting; resume instead of redoing; write progress down before switching.** Any agent picking up a task reads the shared record first and continues from the recorded step.
- 5. **No completion claim without criterion-linked evidence.** "I checked" is not evidence. See `references/evidence-grading.md`.
- 6. **Irreversible actions need separate, explicit user approval.** Deploy, publish, send, upload, delete, account changes: the approval is recorded (`orchestrator.mjs approve`). *A passed verification is not a shipping authorization.*
- 7. **The human-readable shared record outranks the queue.** If the record and the queue disagree, stop and follow the record; the queue is a scheduler, not the authority on intent.
- > 中文对照:① 一个文件同一时刻只有一个主写入者;② 核验者由能力评分选出、绝不是实现者;③ 核验门禁不可绕过(失败即退回,复验通过或带署名覆盖才放行);④ 先读再动、接续不重做、切换前留痕;⑤ 完成必须附可复现证据;⑥ 不可逆动作需单独授权;⑦ 共享记录优先于队列。
- ## Important-task intake gate
- Applies to any task marked important - and to every **L3** task, because consequential work is
- important by definition. An external action is always L3, so it is covered twice over.
- Before the **first write, first dispatch or first verification** of an important task, ask the user and record three choices - never default them:
- 1. Execute solo, or start collaborative mode?
- 2. Run the security scan, or skip it?
- 3. Run an independent verification pass, or skip it?
- "Important" includes at least: multi-file or architectural change, user-facing/product delivery, production/outreach/publish/deploy, data migration or deletion, permission/credential/security change, costly scanning, cross-agent collaboration, or any task the user calls high-quality/fully verified. When the boundary is unclear, treat it as important.
- The gate decides *how* work is coordinated. It does **not** authorize irreversible actions. See `references/important-task-intake.md`.
- ## Fast path
- 1. Inventory each agent's tools, model, workspace, permissions, specialties, cost tier and availability.
- 2. Score capabilities against roles; never infer a role from an agent's name.
- 3. Classify the task: routine / important / critical.
- 4. Write a compact task packet: objective, non-goals, ownership, risks, acceptance tests, owner, verifier.
- 5. Take the ownership lock before editing.
- 6. Run the pipeline `specify -> implement -> verify -> evidence -> done`; important work requires an independent verifier, critical work adds domain review and approval.
- 7. Stop on failed gates, ownership conflicts, missing evidence or unapproved external actions.
- ## Roster: three or more agents, any names
- Roles are assigned from a **roster**, which is data, not code. Three agents work; so do five, eight or twelve, and the agents can be called anything at all - nothing depends on an agent being named `coordinator` or `verifier`:
- ```yaml
- agents:
- planner: { cost: 2, specialties: [], scores: { specify: 3, implement: 1, verify: 1, evidence: 3 } }
- builder: { cost: 4, specialties: [full], scores: { specify: 1, implement: 3, verify: 1, evidence: 1 } }
- webhand: { cost: 5, specialties: [web], scores: { specify: 1, implement: 2, verify: 1, evidence: 1 } }
- auditor: { cost: 6, specialties: [verify], scores: { specify: 1, implement: 1, verify: 3, evidence: 2 } }
- scribe: { cost: 1, specialties: [], scores: { specify: 1, implement: 1, verify: 0, evidence: 3 } }
- routes: # optional: anything omitted is resolved by score
- specify: planner
- evidence: scribe
- implement: { build: builder, web: webhand }
- ```
- - Scores are 0-3 per role. Selection is deterministic and reproducible: **highest score -> capability match in `specialties` -> lower `cost` -> agent id order**. `available: false` parks an agent without deleting it.
- - Stage owners come from `routes` when the roster names one, otherwise from the score. A roster that can fill no agent for a stage is rejected loudly, never routed by accident.
- - Install a roster of your own with `orchestrator.mjs init --roster <file.json>`. Adding an agent is a data edit: declare it, score it, it joins selection.
- ## Roles and ownership
- | Role | Owns |
- |---|---|
- | Coordinator | decomposition, state, conflict resolution, accepting evidence |
- | Implementation worker | edits assigned resources only |
- | Verification worker | independent reproduction, boundary and regression checks (never the implementer) |
- | Environment specialist | runtime, MCP, build, toolchain diagnosis |
- | Domain reviewer | UX, content, security, legal or other domain criteria |
- One agent may hold several roles on **routine** work only. Important work requires an independent verifier; critical work adds a domain reviewer and human approval. Ownership conflicts are resolved by stopping, not by racing: the last timestamped entry in the shared record wins.
- See `references/roles.md` and `references/routing-and-roles.md`.
- ## Rigor: how much verification this change has to carry
- Rigor is chosen from the **blast radius** of the change, not from how interesting it is, and it
- decides which activities are mandatory - the way an integrity level selects V&V tasks in
- IEEE 1012. State it; do not infer it. The engine stores it per task (`--rigor`) and enforces it.
- | Level | Applies when | Mandatory |
- |---|---|---|
- | **L1 local** | one file, no shared interface, reversible | tests for the change; self-produced evidence allowed (never E1) |
- | **L2 shared** (default) | multiple files, shared config, or user-visible behaviour | verifier must differ from the implementer; positive **and** negative cases; evidence graded E1 or E2 |
- | **L3 consequential** | irreversible, published, deployed, or touching credentials/permissions | independent verification, a machine-produced artifact, an approval record (approver + scope + reason), **E1 evidence**, and a kept failure record |
- Two rules the engine enforces rather than trusting: an **external action can never be filed as
- L1** (it leaves the machine), and an **L3 task cannot reach `done` without E1 evidence** - a
- command, its exit code and the revision it applies to.
- See `references/verification-standard.md` for the ten requirements this is derived from.
- ## Verification gate
- Verification is a gate in the same sense as a security scan, except it is run by **an agent from your own team, chosen for the job** - no external tool and no particular vendor is required:
- - the verifier is the highest-scoring eligible agent for `verify`, and the implementer is **never** eligible (if nobody is scored above 0, the run fails closed instead of pretending);
- - completing the `verify` stage is a PASS; `fail` records the failing criteria and sends the task back to `implement` - the verifier never fixes the work itself;
- - the gate blocks two things: reaching `done` from `evidence`, and being dispatched to `evidence` at all;
- - a human can override deliberately with `override --task <id> --by <who> --scope <what> --reason <why>` - all three are required, and the failed result is kept next to the override rather than erased.
- ## Handoff: the shared record
- When the work moves - another session, another tool, another day - the state moves through a **shared record**, not through chat history:
- | File | Holds | Updated |
- |---|---|---|
- | `CURRENT-TASK.md` | the live handoff entry: status, owner, progress, next step, evidence | on every substantial step |
- | `HANDOFF-RULES.md` | the team's rules of engagement | when rules change |
- | `MEMORY-SNAPSHOT.md` | shared decisions, preferences, lessons | periodically / on big decisions |
- | `AGENT-ROLES.md` | who can do what | when the roster changes |
- The five iron rules, in order: **read before acting**, **resume instead of redoing**, **write progress down**, **hand off before switching**, **completion carries graded evidence**. A task waiting to be picked up is `handoff-wait`; a task stuck on a condition names that condition and is only marked `blocked` after three rounds.
- The record is the authority on intent; the queue is the scheduler. When they disagree, the record wins and the queue is reconciled to it.
- Full detail, including the cross-tool checklist and the memory sync routine: `references/handoff.md`.
- ## Deployment profile: hub-coordinated mesh (live instantiation, 2026-10-21)
- The protocol above is running, invariants unmodified, as a concrete team of eight seats across a cloud VM and one local machine. This profile records where the live deployment deliberately goes beyond the generic text; the concrete record (seats, channels, paths, probe results) is `references/deployment-profile.md`.
- **Topology: hub for authority, mesh for traffic.** Anti-pattern 5's hub-and-spoke still owns dispatch, approvals and state — nothing is assigned, approved or marked done except through the coordinator. Execution traffic, however, no longer relays through the hub: every ordered pair of agents has a **tested real-time link**, probe-verified in both directions and re-probed after any outage, so information syncs in seconds instead of at polling intervals. A direct link carries work products and evidence; it never carries a new assignment. The red line against private task hand-offs is unchanged.
- **One human, one contact point.** The user talks only to the coordinator. Every gated decision is batched into a single list for the user; approval is per item, and a blanket authorization does not cover gated actions (recorded precedent).
- **A link is a mechanism, not an intention.** Each seat is reached through the strongest channel it actually has: a headless CLI over SSH where one exists; a scheduled-task bridge where a sandboxed desktop agent's restricted token requires an interactive session; a native headless invocation for the verifier; a GUI wake channel for session-bound bots with no port and no CLI; a file inbox/outbox as the fallback that always works. A channel counts as up only when a probe round-trips — configuration, enrollment status and console presence are not evidence of reachability.
- **Three seats are pinned by the user.** Routine work is still score-routed, but the coordinator seat, the verifier seat (a dedicated agent that never implements) and the single writer of the shared record are pinned in the roster. Pinning is a recorded deployment decision, not a change to the routing rules.
- **The shared memory is a temporal graph.** The memory snapshot is implemented as a knowledge graph (Graphiti on Neo4j): facts carry validity windows, a superseded fact keeps the timestamp it stopped being true, and "what was true when" is a query rather than an excavation. Only coordinator-ingested, source-anchored content enters the graph — anti-pattern 2's verified partition, enforced by ingest discipline. Private per-agent memory is never bulk-copied into the shared graph.
- **Rules ship as versioned fragments.** The rules of engagement live as fragments in a shared rules directory (cloud-commons, v1→v3) and are synced into each member's own rules file by script, then verified per member after every sync. A fragment that does not land everywhere is a failed distribution, not a partial success.
- **Internal traffic is machine-native.** Between agents: structured payloads and task cards carrying evidence anchors (source path, sha16, raw command output). Human language is produced only at the user boundary. A cross-agent claim without an anchor is treated as no claim, and the coordinator stays in the loop on in-flight cross-agent work rather than polling on a timer.
- **The eight seats** (ids in the roster): `zaodianshui` coordinator (cloud) · `codex` implementation — web, code, deploys · `dsh` local lead — watchdogs, scheduled guardianship, single writer of the shared record · `tianshu` verification only · `xiaoyu` coordinator of the Grok Bot line · `yunying` content operations · `jiaoxue` teaching line, speaks when addressed · `openclaw` retired platform (workspace only). Three further personas in the team group chat are mirrors of codex, dsh and tianshu inside Grok Bot, not separate agents.
- ## Cold-start bootstrap: from zero communication to this mesh
- The profile above was not installed; it was bootstrapped from a state in which no agent could reach any other and no shared infrastructure existed. The reproducible path is `references/bootstrap-from-zero.md`; the ladder, in brief:
- - **Stage 0 — Inventory.** For every agent: invocation surface (CLI, API, GUI-only, none), writable ground, session model, and who can write where. Personas inside a host application are surfaces of one seat, not separate agents. Output: a draft channel matrix; empty cells are expected.
- - **Stage 1 — The mailbox.** Stand up a file inbox/outbox on any ground two agents can both see — a shared directory, a synced folder, a repo. One file per message, task-card format, evidence anchors on every claim, receipts for every delivery. This rung alone carries dispatch, handoff and verification; if no ground is shared at all, the user relays the files by hand and the protocol is unchanged.
- - **Stage 2 — Hub, roster, commons.** Designate one hub as the user's only contact point; write the roster; pin the coordinator, the verifier and the single writer of the shared record; publish the rules of engagement as a versioned commons file the hub alone edits.
- - **Stage 3 — Distribution.** Merge the commons into every member's own instructions file and verify per member that it landed; a fragment missing anywhere is a failed distribution.
- - **Stage 4 — The pairwise ladder.** Upgrade pair by pair — mailbox, hub-relayed, direct invocation (headless CLI, remote shell, bridges), session-bound bridges for sandboxed desktop agents — proving each ordered pair with a bidirectional PONG and recording per-channel quirks in the matrix. On outage, fall back one rung and keep working.
- - **Stage 5 — Memory.** Share digests and a hub index, never bulk copies of raw memory. A temporal knowledge graph behind the hub is an optional accelerator; the mesh must work without it.
- - **Stage 6 — Operating rhythm.** The mesh now runs as the deployment profile describes: assignments only through the hub, verification by a non-implementer, anchors at every hand-off, human language only at the user boundary.
- Bootstrap is complete only when the acceptance checklist in the reference passes — including one real task run end-to-end through the mesh and one outage ridden through on a lower rung.
- ## Pipeline and queue
- The specify stage cannot end without its **work products**: `--spec` (what is being built) and
- `--acceptance` (how it will be judged), optionally `--non-goals`. A task that cannot state both is
- not ready to implement, and the engine refuses the transition instead of letting work drift into
- something nobody can falsify later.
- Stage machine: `specify -> implement -> verify -> evidence -> done`, plus `blocked` and `recovery`.
- ```bash
- node scripts/orchestrator/orchestrator.mjs init --roster my-team.json # 3, 5 or 12 agents
- node scripts/orchestrator/orchestrator.mjs create --title "Fix checkout" --type build --workspace "<workspace>" \
- --resources "src/checkout.ts,src/cart.ts" --rigor L2 # one writer per RESOURCE
- node scripts/orchestrator/orchestrator.mjs claim --task task-0001 --agent implementer-a
- node scripts/orchestrator/orchestrator.mjs dispatch --task task-0001 # dry-run command only
- node scripts/orchestrator/orchestrator.mjs fail --task task-0001 --agent verifier-b --criteria "test X fails"
- node scripts/orchestrator/orchestrator.mjs override --task task-0001 --by user --scope "release 2.1" --reason "pre-existing flake"
- node scripts/orchestrator/orchestrator.mjs approve --task task-0002 --by user --scope "deploy to production"
- ```
- - `dispatch` prints the command to run and an `argv` array; it never launches an agent. Prefer `argv` over pasting the command line into a shell.
- - A non-dispatchable task (blocked, gated, no assignee, no command) fails with a non-zero exit code. Exit 0 must never disguise an undispatchable task.
- - **An external action cannot be dispatched until it is approved**, and the approval names an approver and a scope.
- - **A resource is held by one live task at a time.** Declare `--resources`; a claim that overlaps a held
- resource is refused and names the holder. The lock is per stage, so completing a stage frees it.
- - **A verification PASS must carry its witness.** The record is stored on the verification itself, not as
- evidence authored by the verifier, so L3's "the verifier produced none of the evidence" still holds.
- - **Three failed verifications block the task** for a coordinator to recover, instead of retrying forever.
- - `approve` is the only path for external actions, per task and per scope; `fail` closes the verification gate; `override` is the only way past it.
- See `references/task-queue.md` and `scripts/orchestrator/README.md`.
- ## Evidence grading
- | Grade | Meaning | Allowed wording |
- |---|---|---|
- | E1 reproducible | command output, exit code, hash, screenshot, URL response | "verified: `<command>` exited 0, output ..." |
- | E2 same-system peer check | another agent re-ran it and gave reproduction steps | "peer-checked by <agent> (steps attached): ..." |
- | E3 self-assertion | the author says it looks fine | "**self-checked, not independently verified**: ..." |
- | E4 planned | not executed yet | "planned verification: ..." |
- Never let E3 speak in E1's voice. Same-system peer checks are not independent verification. Grades do not survive a revision change: after an edit, a resumed session or context compaction, re-verify before restating an old "verified".
- The engine enforces this rather than trusting the wording: a bare string handed to
- `complete --evidence "..."` is stored as **E3 with a note saying it was downgraded**, and
- `--evidence-grade E1` is refused unless it also carries `--evidence-command`,
- `--evidence-exit-code` and `--evidence-revision`. Grade labels are not decoration - an
- ungraded sentence cannot borrow E1's voice because it is not allowed to carry the label.
- See `references/evidence-grading.md` and `references/verification-standard.md`.
- ## Token and cost policy
- Load only the reference section you need. One coordinator summary instead of full transcripts. Bounded structured packets. Parallelize read-only checks; serialize dependent work and writes. Cheap models for discovery and mechanical checks; strong models for ambiguity, architecture, adversarial review and acceptance. Never compress requirements, code, paths, URLs, hashes, errors, permissions or evidence.
- See `references/token-efficiency.md`.
- ## Completion gate
- Do not claim done, verified or deployed without criterion-linked evidence. User-facing work should test the actual runtime, target environments, core interactions, errors, keyboard/accessibility where relevant, resource loading, and reduced-motion/fallback behaviour. A failed gate returns the task to its responsible stage. A passed gate is not a shipping authorization.
- ## Stop conditions
- Ownership conflict; unrecoverable baseline; material ambiguity; release-blocking finding; missing evidence; a request for secrets; an external action without recorded approval; a verifier that was also the implementer; a shared record and queue that disagree.
- ## Anti-patterns (read before forming a team)
- | # | Pitfall | Symptom | Engineering fix |
- |---|---|---|---|
- | 1 | Context explosion | the shared transcript grows until early constraints and acceptance criteria fall out of the window | the shared record stores conclusions and state only (<=200 lines); process chat stays local; the coordinator re-sends the minimal necessary context each batch |
- | 2 | Hallucination contagion | one member's wrong output spreads as fact through shared memory | partition shared memory: entries become visible only after the coordinator marks them verified; unverified content stays quarantined with source and confidence |
- | 3 | Coordination deadlock | A waits for B, B for C, C for A; the system hangs forever | queue resource requests in a globally fixed order; every wait carries a timeout (default 15 min) that escalates to the coordinator |
- | 4 | State divergence | two members disagree about task state; work is repeated or dropped | single source of truth = the shared record's state machine; state changes are written to disk; verbal/session state does not count |
- | 5 | Communication storm | members message each other; links grow quadratically with headcount | hub-and-spoke: members talk to the coordinator only; coordination is relayed |
- | 6 | Agent sprawl | roles are split until boundaries overlap and nobody owns the outcome | subtraction first: a new role must justify why it cannot fold into an existing loop; every role owns an end-to-end deliverable |
- | 7 | Dispatch left to LLM improvisation | the same input routes differently twice; work is slow or skipped | make dispatch deterministic: board scan + fixed rules; the LLM plans, it does not mechanically distribute |
- | 8 | No acceptance aggregation | each member delivers, nobody assembles and checks; defects surface at integration | the coordinator runs an independent acceptance gate against every criterion with runtime evidence; failing batches are returned whole |
- ### Hard red lines
- - Never let two members write the same file at the same time.
- - Never start work or deliver outside the shared record and the queue.
- - Never treat "I think it is done" as done - runtime evidence is required.
- - Never let members hand tasks to each other privately; all dispatch flows through the coordinator.
- - Never let an implementer certify its own work.
- See `references/anti-patterns.md`, `references/protocol.md` and `config/agents.example.yaml`.
README.md · README · 308 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- # Agent Orchestra Protocol
- **A governed coordination protocol for multi-agent AI work.**
- > Agents can work. **They cannot declare success.**
- Every multi-agent project eventually meets the same failure: an agent says it is done, and nothing in
- the system is able to disagree. This protocol removes that possibility by making the interesting rules
- checks in code rather than requests in a prompt.
- ```
- AGENT MAY AGENT MAY NOT
- claim work silently overwrite another agent's resource
- own resources verify its own L2/L3 work
- implement pass verification without evidence
- hand off retry forever after failing
- verify others' work perform an external action without approval
- produce graded evidence declare success without satisfying the gate
- ```
- ### Measured, not asserted
- `node bench/protocol-benchmark.mjs` - six failure modes, three coordination models, scripted actors,
- no model calls. Re-run it and you get this table:
- | Failure mode | What is counted | Single agent | Naive multi-agent | Agent Orchestra |
- |---|---|---|---|---|
- | Two agents write the same resource | conflicting writes that landed | 1 | 1 | **0** |
- | Author declares done with no evidence | self-declared successes accepted | 1 | 1 | **0** |
- | Implementer verifies its own work | self-verifications accepted | 1 | 1 | **0** |
- | Verification keeps failing | attempts before the loop stops | 25 | 25 | **3** |
- | Deploy attempted without approval | unapproved actions dispatched | 1 | 1 | **0** |
- | Failure followed by a success claim | failures that reached done | 1 | 1 | **0** |
- Scope, stated plainly: this measures the **coordination layer** - which failure modes get through - not
- model quality, and it uses deterministic actors rather than LLMs so that it is reproducible. The
- methodology and its limits are in [`bench/README.md`](bench/README.md).
- ### Conformance, audited
- [**CONFORMANCE.md**](CONFORMANCE.md) maps every MUST in the protocol to the code that enforces it and
- the test that pins it: **41 requirements, 38 ENFORCED, 1 DOCUMENTED ONLY, 2 OUT OF SCOPE**. The table is
- machine-checked - a reference that does not resolve, or an ENFORCED row without a test, fails the gate.
- ### The normative part
- [**AGENT-ORCHESTRA-PROTOCOL.md**](AGENT-ORCHESTRA-PROTOCOL.md) defines eight contracts - Task,
- Resource, Ownership, Handoff, Verification, Evidence, Approval, Recovery - each with the code that
- enforces it, the behaviour on violation, and its honest limit. The contracts are stated in terms of
- operations and observed behaviour, not of this CLI, so another host can implement them over its own
- storage and prove conformance with the same benchmark.
- ### What this costs you
- Zero dependencies, Node 18+.** 106 engine tests, 8 acceptance tests, 8 handoff tests, 83 activation cases (a corpus driven by 8 acceptance checks), and one gate command (`node scripts/gate.mjs`) that CI and your machine both run - it injects
- twelve known faults and must reject every one of them before it reports PASS.
- 中文对照:`references/zh-contrast.md`
- ---
- One portable, capability-first protocol for multi-agent work along **two axes**: coordinating a team inside one batch of tasks, and handing that work across sessions, tools and days without losing state. It ships explicit ownership, independent verification, graded evidence, deterministic dispatch, a shared handoff record and token-aware routing.
- <p align="center"><img src="og-image.png" alt="Agent Orchestra" width="600"></p>
- ## What's in this repository
- | Path | What it is | Read it when |
- |---|---|---|
- | `SKILL.md` (root) | The protocol: roles, ownership, verification gate, handoff record, queue, evidence, cost | you want the short form |
- | `references/handoff.md` | Handoff discipline: the four shared records, the five iron rules, the task state machine, single-writer conflicts, memory sync, the cross-tool checklist | work is relayed across sessions or tools |
- | `references/` | Detail: routing-and-roles, evidence-grading, task-queue, important-task-intake, protocol, anti-patterns, token-efficiency | you need the full rule, not the summary |
- | `config/agents.example.yaml` | Activation, risk, roles, ownership, intake gate, roster, queue, verification gate, approval, evidence, cost | you are wiring the skill to your agents |
- | `scripts/detect-trigger.mjs` | Keyword activation detector (exit 0 engaged / 1 not engaged) | keyword activation mode |
- | `scripts/orchestrator/` | Task queue CLI + library + tests (zero dependencies, Node 18+) | you want deterministic dispatch and claim locks |
- | `scripts/install.ps1`, `scripts/install.sh` | One-click installers that copy this single skill into a detected skills directory | first install |
- | `scripts/skill-catalog/` | Optional, standalone: route a task description to the right skill using the skill's own `description`, with no per-skill keyword list to maintain | you have a skill directory and want it findable |
- | `docs/ARCHITECTURE.md` | The production deployment that runs this protocol daily: seats, communication layer, governance, memory, operations, and its honest limits | you want the live system this ships from, not just the rules |
- ## Features
- - covers both axes in one skill: in-task coordination, and handoff across sessions and tools
- - keeps a human-readable shared record - handoff entry, rules, memory snapshot, role table - as the authority when the queue and intent disagree
- - five iron rules with executable checks: read before acting, resume instead of redoing, write progress down, hand off before switching, completion carries graded evidence
- - a task state machine - `queued` / `in-progress` / `handoff-wait` / `done`, plus `blocked` only after three unresolved rounds - mapped to the queue's stage machine
- - discovers capabilities instead of assuming an agent's name implies a role
- - assigns coordinator, implementer, verifier, environment and domain roles by capability score
- - enforces one primary writer per file or resource, with reviewers limited to isolated artifacts
- - requires a verifier that is not the implementer: an author's self-check is never a verification result
- - picks that verifier by **capability score from your roster**, excluding the implementer, and fails closed when nobody is scored for verification
- - works with **any roster size and any agent names**: 3 agents, 8 or 12; routing comes from roster data, so adding an agent is a data edit, not a code change
- - enforces a **verification gate**: a failed verify sends the task back to implement and it can neither reach done nor be dispatched to evidence until a verify passes or a human records an override with approver, scope and reason
- - grades every claim E1 (reproducible) through E4 (planned), so self-assertion cannot pose as evidence
- - asks the user three questions before an important task starts (solo vs collaborative, security scan, independent verification) and never defaults them
- - **refuses a conflicting claim on the same resource**, naming the task and agent that hold it
- - **requires a witness for every verification PASS** and stores it on the verification record, which is what keeps L3's
- "the verifier produced none of the evidence" from contradicting "verification needs evidence"
- - **blocks a task after three failed verifications** until a coordinator recovers it, reading the ceiling from policy
- rather than hard-coding it
- - **gates dispatch on approval, scope included**: an approval for staging does not unlock a production deploy
- - **runs a domain review for consequential work** - a third agent who neither implemented nor verified it, with its own record
- - adds a stronger pipeline for important and critical work: independent verification, domain review, and explicit approval for external actions
- - runs a deterministic queue: `specify -> implement -> verify -> evidence -> done`, claim locks, dry-run dispatch, per-scope `approve` for deploy/publish/send/upload/delete
- - refuses to disguise an undispatchable task as success (non-zero exit code)
- - reduces token and API cost with compact packets, bounded outputs, model tiering, parallel read-only checks, and escalation only where ambiguity lives
- It does not provide an absolute guarantee; it makes failures visible, recoverable, and less likely.
- ## Activation (when to use)
- The skill engages in one of three modes, configured under `activation:` in `config/agents.example.yaml`:
- - **global** - engages for every task
- - **keyword** - engages only when the task text matches configured keywords (default; token-saving)
- - **manual** - engages only when the user explicitly invokes it
- For keyword mode, run the trigger detector before starting:
- node scripts/detect-trigger.mjs --text "<task text>" --config config/agents.example.yaml
- Three answers, not two: **0** ENGAGED (a curated keyword plus an act of coordinating), **3** POSSIBLE (ambiguous - the
- caller decides), **1** NOT_ENGAGED (silent). Comparison and evaluation requests never auto-engage, and the veto markers are
- phrases rather than single characters, because a false veto silently drops a real request. `exclude_keywords` vetoes
- engagement even when a trigger matched.
- ## The ten invariants
- Each one is a check in the engine with a test behind it, not a sentence an agent is asked to respect.
- | # | Invariant | Where it is enforced |
- |---|---|---|
- | I1 | **One owner per task** - a stage is held by exactly one agent at a time | `claimTask`, `requireOwner` |
- | I2 | **One writer per RESOURCE** - a conflicting claim is refused and names the holder | `claimTask`, resource overlap check |
- | I3 | **No implementer verifies its own L2/L3 work** - and an important task may not self-verify even at L1 | `completeStage` |
- | I4 | **A verification PASS must carry its witness** | `completeStage` (verify) |
- | I5 | **A failed verification blocks downstream** - no `done`, no dispatch to evidence | `failVerification`, `nextDispatch` |
- | I6 | **Three failed rounds block the task** for a coordinator to recover | `failVerification` + `policy.maxVerificationAttempts` |
- | I7 | **L2 and L3 need graded evidence** before `done` | `completeStage` (evidence) |
- | I8 | **L3 needs E1 evidence and a verifier that produced none of it**, plus a domain review by a third party | `completeStage` (evidence, domain_review) |
- | I9 | **An external action needs a matching approval** before dispatch | `nextDispatch` |
- | I10 | **`done` requires every applicable gate to be satisfied** | the stage machine |
- Two honest boundary markers, both written down as known findings rather than hidden: the queue is a plain JSON
- ledger (a discipline, not a security boundary - F-011), and E1 checks that a command, its exit code and a revision are
- recorded, not that the command ran (F-012).
- 1. One primary writer per file or resource at a time.
- 2. Verifier != implementer - and the verifier is the best-suited eligible agent by capability score, never a fixed name.
- 3. The verification gate cannot be walked past: no `done` and no `evidence` dispatch after a failure until a verify passes or an override with approver, scope and reason is recorded.
- 4. Read before acting, resume instead of redoing, and write progress down before switching.
- 5. No completion claim without criterion-linked evidence.
- 6. Irreversible and external actions need separate, explicit user approval (`approve`), which passing verification does not grant.
- 7. The human-readable shared record outranks the queue.
- 8. Consequential work is reviewed by someone in the domain who did none of the work.
- 9. Rigor is chosen from the blast radius, not from how interesting the task is.
- 10. Every rule here is either enforced or listed as an accepted limitation - nothing in between.
- ## Roster (any size)
- Roles are assigned from a **roster**, which is data, not code. Three agents work; so do five, eight or twelve, and the agents can be called anything at all - nothing depends on an agent being named `coordinator` or `verifier`:
- ```yaml
- agents:
- planner: { cost: 2, specialties: [], scores: { specify: 3, implement: 1, verify: 1, evidence: 3 } }
- builder: { cost: 4, specialties: [full], scores: { specify: 1, implement: 3, verify: 1, evidence: 1 } }
- webhand: { cost: 5, specialties: [web], scores: { specify: 1, implement: 2, verify: 1, evidence: 1 } }
- auditor: { cost: 6, specialties: [verify], scores: { specify: 1, implement: 1, verify: 3, evidence: 2 } }
- scribe: { cost: 1, specialties: [], scores: { specify: 1, implement: 1, verify: 0, evidence: 3 } }
- routes: # optional: anything omitted is resolved by score
- specify: planner
- evidence: scribe
- implement: { build: builder, web: webhand }
- ```
- - Scores are 0-3 per role. Selection is deterministic and reproducible: **highest score -> capability match in `specialties` -> lower `cost` -> agent id order**. Setting `available: false` parks an agent without deleting it.
- - Stage owners come from `routes` when the roster names one, otherwise from the score. A roster that can fill no agent for a stage is rejected loudly, never routed by accident.
- - Install a roster of your own with `orchestrator.mjs init --roster <file.json>`. Adding an agent is a data edit: declare it, score it, it joins selection.
- Full rules: `references/routing-and-roles.md`.
- ## Verification gate
- Verification is a gate run by **an agent from your own team, chosen for the job** - no external tool and no particular vendor is required:
- - the verifier is the highest-scoring eligible agent for `verify`, and the implementer is never eligible; if nobody is scored above 0, the run fails closed instead of pretending;
- - completing the `verify` stage is a PASS; `fail` records the failing criteria and sends the task back to `implement`, and the verifier never fixes the work itself;
- - the gate blocks two things: reaching `done` from `evidence`, and being dispatched to `evidence` at all;
- - a human may override deliberately with `override --by <who> --scope <what> --reason <why>` - all three are required, and the failed result is kept beside the override rather than erased;
- - a passed verification is not a shipping authorization.
- Full rules: `references/task-queue.md`.
- ## Handoff record
- When the work moves - another session, another tool, another day - the state moves through a shared record, not through chat history:
- | File | Holds | Updated |
- |---|---|---|
- | `CURRENT-TASK.md` | the live handoff entry: status, owner, progress, next step, evidence | on every substantial step |
- | `HANDOFF-RULES.md` | the team's rules of engagement | when rules change |
- | `MEMORY-SNAPSHOT.md` | shared decisions, preferences, lessons | periodically / on big decisions |
- | `AGENT-ROLES.md` | who can do what | when the roster changes |
- The five iron rules, in order: **read before acting**, **resume instead of redoing**, **write progress down**, **hand off before switching**, **completion carries graded evidence**. A task waiting to be picked up is `handoff-wait`; a task stuck on a condition names that condition and is marked `blocked` only after three rounds.
- Detail, including the cross-tool checklist and the memory sync routine: `references/handoff.md`.
- ## Task queue
- node scripts/orchestrator/orchestrator.mjs init --roster my-team.json # any roster size, any names
- node scripts/orchestrator/orchestrator.mjs create --title "Fix checkout" --type build --workspace "<workspace>"
- node scripts/orchestrator/orchestrator.mjs claim --task task-0001 --agent implementer-a
- node scripts/orchestrator/orchestrator.mjs complete --task task-0001 --agent implementer-a --evidence "tests pass, exit 0"
- node scripts/orchestrator/orchestrator.mjs dispatch --task task-0001
- node scripts/orchestrator/orchestrator.mjs fail --task task-0001 --agent verifier-b --criteria "test X fails"
- node scripts/orchestrator/orchestrator.mjs override --task task-0001 --by user --scope "release 2.1" --reason "pre-existing flake"
- node scripts/orchestrator/orchestrator.mjs approve --task task-0002 --by user --scope "deploy to production"
- node scripts/orchestrator/orchestrator.mjs status
- `dispatch` prints a dry-run command; it never starts an agent. `fail` closes the verification gate; `override` is the only way past it and needs approver, scope and reason. Run the tests with:
- node --test scripts/orchestrator/lib.test.mjs
- ## Install
- **One repository, one skill.** Copy the contents of this repository into the skill directory your host supports - `SKILL.md`, `references/`, `config/`, and `scripts/` if you want the queue. There is no companion package and no second skill to install.
- Optionally, the one-click installers perform the same single-skill copy into common skill directories.
- Windows PowerShell:
- powershell -ExecutionPolicy Bypass -File scripts/install.ps1
- Linux/macOS shell:
- ./scripts/install.sh
- Validate your config locally:
- node scripts/validate-config.mjs
- ## Host adapter
- Implement these five primitives: `discover_agents`, `dispatch`, `lock`, `record`, and `approve`. If a primitive is unavailable, use a documented manual fallback and record the limitation.
- ## Usage
- This is important. Three agents are available. Assign the best implementation agent, have another independently test it, and do not deploy.
- Expected: capability inventory, role assignment, task packet, intake-gate answers, ownership lock, implementation evidence, independent verification with graded evidence, acceptance report, and `awaiting_user_approval` for deployment.
- ## Checking this repository
- One command, and it is the same command CI runs:
- ```bash
- node scripts/gate.mjs # evidence lands in evidence/latest.json
- node scripts/gate.mjs --no-self-test # skip the gate's own failure test (not recommended)
- ```
- The runner checks that the shipped config is usable by the engine, that activation engages on
- intent and stays silent on lookalikes, that the queue engine and the CLI I/O behave, that the
- handoff record discipline holds, that the numbers stated in these docs match the suites, and that
- no private path, secret or host-specific name ships.
- A shared handoff record can also be checked on its own, which is what the skill asks a successor to
- trust before resuming:
- ```bash
- node scripts/check-handoff.mjs --dir <shared-record-dir>
- ```
- It requires the four record files, a live entry carrying status, owner, current step, next step and
- evidence, a recognised status value, and the statement that the record outranks the queue. It writes a machine-readable record
- bound to the git revision and to the SHA-256 of the key artifacts.
- It also **proves it can fail**: before reporting PASS it injects known-bad configurations and
- requires every one to be rejected. A gate that has quietly stopped checking anything therefore
- fails instead of reporting green - see [`references/verification-standard.md`](references/verification-standard.md)
- for the requirements it is held to, and [`KNOWN-FINDINGS.md`](KNOWN-FINDINGS.md) for accepted
- defects and their review dates.
- ## Running in production
- This is not a paper protocol. The same-source deployment (**Agent Team OS**, the companion
- package below) runs under it every day, since 2026-10-20: a cloud coordinator dispatching to
- local execution seats, with a dedicated verification seat that never implements what it
- verifies. As of 2026-10-23 the live deployment has carried **220+ mailbox messages** with
- delivery receipts and acks between seats, run its independent verification gates on every
- consequential change, sent **465 business letters** cumulatively through its outreach radar
- (every send behind a human approval gate), and grown a temporal knowledge graph to
- **2,646 facts** with daily ingest. Its next-generation communication layer — NATS JetStream
- transport, a Postgres task ledger and A2A task contracts — is running in shadow alongside the
- mailbox, mirroring real dispatches before any cutover decision.
- The full current-state record is [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md); the
- channel-level detail is in
- [`references/deployment-profile.md`](references/deployment-profile.md).
- ## License
- Apache-2.0 for the code (with the patent grant that a protocol implementer needs), CC BY 4.0 for
- the protocol specification. See LICENSE, NOTICE and LICENSE-SPEC.
- ## Companion package: Agent Team OS
- This repository also ships **Agent Team OS**, a companion skill package for running a team of AI agents as one governed unit: layered long-term memory, an agent mailbox with delivery receipts, an event-driven doorbell, and human approval gates. The full package lives in [agent-team-os/README.md](agent-team-os/README.md). For the architecture of the production deployment that runs both packages, see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
scripts/orchestrator/lib.test.mjs · Bundled script · 1,426 lines · 1 finding marked
- // lib.test.mjs — node:test suite for the portable orchestrator runtime.
- //
- // Run from the repository root:
- // node --test scripts/orchestrator/lib.test.mjs
- //
- // Covers: the stage machine for every task type, the verifier-independence invariant,
- // single-writer claim locks, capability derivation/validation, important-task start
- // choices, the external-action approval gate, recover semantics, dry-run dispatch,
- // and the sanitisation/dry-run guards on the runtime sources.
- import test from 'node:test';
- import assert from 'node:assert/strict';
- import { mkdtempSync, readFileSync, readdirSync, writeFileSync, utimesSync } from 'node:fs';
- import { tmpdir } from 'node:os';
- import { join } from 'node:path';
- import { fileURLToPath } from 'node:url';
- import { spawn } from 'node:child_process';
- import {
- AGENTS,
- CAPABILITIES,
- DEFAULT_START_CHOICES,
- EVIDENCE_GRADES,
- RIGOR_LEVELS,
- STAGES,
- STATE_VERSION,
- TYPES,
- approveExternalAction,
- canRunExternalAction,
- capabilityForType,
- claimTask,
- completeStage,
- createState,
- createTask,
- failVerification,
- implementerForType,
- migrateState,
- nextDispatch,
- ownerFor,
- overrideVerificationGate,
- recoverTask,
- selectAgent,
- normalizeEvidence,
- normalizeResource,
- resourcesConflict,
- selectVerifier,
- shellQuote,
- validateRoster,
- verifierFor,
- } from './lib.mjs';
- function createOne(input) {
- // An external action must state its start choices (the intake gate no longer answers for the
- // user), so these approval tests supply them and then test what they are actually about.
- // An external action must state its choices AND be L3 (irreversible), so these approval tests
- // supply both and then test what they are actually about.
- const withChoices = input?.externalAction
- ? { important: true, rigor: 'L3', executionMode: 'collaborative', security: 'planned', independentVerify: 'planned', ...input }
- : input;
- return createTask(createState(), withChoices);
- }
- // The default is E2, not a bare sentence: a string degrades to E3, and L2 requires E1 or E2.
- const DEFAULT_EVIDENCE = { grade: 'E2', text: 'a peer re-ran the stage', checkedBy: 'peer-agent' };
- // A bare sentence normalises to E3, and L2 now requires E1 or E2. The walk helpers turn a string
- // into a graded E2 record so a flow test exercises the stage machine; the floor itself is tested
- // explicitly further down.
- const asEvidence = (evidence) => (typeof evidence === 'string'
- ? { grade: 'E2', text: evidence, checkedBy: 'peer-agent' }
- : (evidence ?? DEFAULT_EVIDENCE));
- function step(state, id, agent, evidence = DEFAULT_EVIDENCE) {
- // The default carries a graded record because rigor L2 (the default) now requires one to reach
- // done, and the packet because the specify stage cannot end without its work products.
- const claimed = claimTask(state, id, agent).state;
- const task = claimed.tasks[id];
- const packet = task.phase === 'specify'
- ? { spec: 'the change under test', acceptance: 'the suite exits 0' }
- : {};
- return completeStage(claimed, id, agent, { evidence: asEvidence(evidence), ...packet }).state;
- }
- // ── state shape ──────────────────────────────────────────────────────────────
- test('createState returns an empty queue carrying a validated roster', () => {
- const state = createState();
- assert.equal(state.version, STATE_VERSION);
- assert.equal(state.version, 4);
- assert.equal(state.nextTaskNumber, 1);
- assert.deepEqual(state.tasks, {});
- assert.deepEqual(state.events, []);
- // The roster is data, so the same mechanism serves 3 agents or 12.
- assert.deepEqual(Object.keys(state.roster.agents).sort(), AGENTS.slice().sort());
- });
- test('createTask is pure: the input state is not mutated', () => {
- const before = createState();
- const result = createTask(before, { title: 'pure' });
- assert.deepEqual(before.tasks, {});
- assert.equal(before.nextTaskNumber, 1);
- assert.equal(before.events.length, 0);
- assert.equal(Object.keys(result.state.tasks).length, 1);
- });
- test('createTask assigns zero-padded sequential ids', () => {
- let state = createState();
- const first = createTask(state, { title: 'a' });
- const second = createTask(first.state, { title: 'b' });
- assert.equal(first.task.id, 'task-0001');
- assert.equal(second.task.id, 'task-0002');
- });
- test('createTask requires a title', () => {
- assert.throws(() => createTask(createState(), { title: ' ' }), /Task title is required/);
- assert.throws(() => createTask(createState(), {}), /Task title is required/);
- });
- test('createTask rejects an unknown task id on later operations', () => {
- assert.throws(() => claimTask(createState(), 'task-9999', 'generalist'), /Unknown task: task-9999/);
- assert.throws(() => nextDispatch(createState(), 'task-9999'), /Unknown task: task-9999/);
- });
- test('a new task is queued in specify and owned by the coordinator', () => {
- const r = createOne({ title: 'new-task' });
- assert.equal(r.task.phase, 'specify');
- assert.equal(r.task.assignedAgent, 'generalist');
- assert.equal(r.task.status, 'queued');
- assert.equal(r.task.lock, null);
- assert.deepEqual(r.task.evidence, []);
- assert.equal(r.state.events.at(-1).type, 'created');
- });
- // ── stage machine ────────────────────────────────────────────────────────────
- const FLOWS = {
- build: { phases: ['specify', 'implement', 'verify', 'evidence', 'done'], actors: ['generalist', 'generalist', 'specialist', 'generalist'] },
- web: { phases: ['specify', 'implement', 'verify', 'evidence', 'done'], actors: ['generalist', 'frontend', 'specialist', 'generalist'] },
- complex: { phases: ['specify', 'implement', 'verify', 'evidence', 'done'], actors: ['generalist', 'specialist', 'generalist', 'generalist'] },
- verify: { phases: ['specify', 'implement', 'verify', 'evidence', 'done'], actors: ['generalist', 'specialist', 'generalist', 'generalist'] },
- environment: { phases: ['specify', 'triage', 'implement', 'verify', 'evidence', 'done'], actors: ['generalist', 'frontend', 'frontend', 'specialist', 'generalist'] },
- };
- test('STAGES lists the pipeline in order', () => {
- assert.deepEqual(STAGES, ['specify', 'triage', 'implement', 'verify', 'domain_review', 'evidence', 'done']);
- });
- for (const type of TYPES) {
- test('stage machine advances ' + type + ' to done', () => {
- const flow = FLOWS[type];
- const created = createOne({ title: type + '-flow', type });
- const id = created.task.id;
- let state = created.state;
- assert.equal(state.tasks[id].phase, 'specify');
- assert.equal(state.tasks[id].assignedAgent, 'generalist');
- for (let i = 0; i < flow.actors.length; i++) {
- const task = state.tasks[id];
- assert.equal(task.phase, flow.phases[i], 'phase before step ' + i);
- assert.equal(task.assignedAgent, flow.actors[i], 'owner of ' + flow.phases[i]);
- state = step(state, id, flow.actors[i], 'step ' + i);
- }
- const done = state.tasks[id];
- assert.equal(done.phase, 'done');
- assert.equal(done.status, 'done');
- assert.equal(done.assignedAgent, null);
- assert.equal(done.lock, null);
- });
- }
- test('the verifier is never the implementer (every type)', () => {
- for (const type of TYPES) {
- const flow = FLOWS[type];
- const implIdx = flow.phases.indexOf('implement');
- const verIdx = flow.phases.indexOf('verify');
- assert.notEqual(flow.actors[implIdx], flow.actors[verIdx], type + ' reuses the implementer as verifier');
- assert.equal(verifierFor(flow.actors[implIdx]), flow.actors[verIdx], type + ' verifier mismatch');
- }
- assert.equal(verifierFor('generalist'), 'specialist');
- // frontend scores 1 for verify, specialist scores 3: the scored ranking wins.
- assert.equal(verifierFor('frontend'), 'specialist');
- assert.equal(verifierFor('specialist'), 'generalist');
- });
- test('completeStage without the claim is rejected', () => {
- const r = createOne({ title: 'unclaimed' });
- assert.throws(() => completeStage(r.state, r.task.id, 'generalist'), /must be claimed by generalist/);
- });
- test('completeStage records evidence supplied by the owner', () => {
- const r = createOne({ title: 'evidence' });
- const claimed = claimTask(r.state, r.task.id, 'generalist').state;
- const out = completeStage(claimed, r.task.id, 'generalist', { evidence: 'exit code 0', spec: 's', acceptance: 'a' });
- assert.equal(out.task.evidence.length, 1);
- assert.equal(out.task.evidence[0].text, 'exit code 0');
- assert.equal(out.task.evidence[0].agent, 'generalist');
- });
- test('completeStage rejects a task stuck on an unknown phase', () => {
- const r = createOne({ title: 'weird-phase' });
- const claimed = claimTask(r.state, r.task.id, 'generalist').state;
- claimed.tasks[r.task.id].phase = 'nonsense';
- assert.throws(() => completeStage(claimed, r.task.id, 'generalist'), /cannot complete unknown phase nonsense/);
- });
- // ── single-writer claim locks ────────────────────────────────────────────────
- test('a held lock rejects a second agent', () => {
- const r = createOne({ title: 'lock' });
- const claimed = claimTask(r.state, r.task.id, 'generalist').state;
- assert.equal(claimed.tasks[r.task.id].lock.owner, 'generalist');
- assert.equal(claimed.tasks[r.task.id].status, 'in_progress');
- assert.throws(() => claimTask(claimed, r.task.id, 'specialist'), /already held by generalist/);
- });
- test('claiming a task assigned to another agent is rejected', () => {
- const r = createOne({ title: 'wrong-agent', type: 'complex' });
- const claimed = step(r.state, r.task.id, 'generalist'); // specify -> implement(specialist)
- assert.equal(claimed.tasks[r.task.id].assignedAgent, 'specialist');
- assert.throws(() => claimTask(claimed, r.task.id, 'generalist'), /is assigned to specialist, not generalist/);
- });
- test('the same agent may re-claim its own lock (idempotent)', () => {
- const r = createOne({ title: 're-claim' });
- const once = claimTask(r.state, r.task.id, 'generalist').state;
- const twice = claimTask(once, r.task.id, 'generalist').state;
- assert.equal(twice.tasks[r.task.id].lock.owner, 'generalist');
- });
- test('claim and complete release the lock between stages', () => {
- const r = createOne({ title: 'release' });
- const after = step(r.state, r.task.id, 'generalist');
- assert.equal(after.tasks[r.task.id].lock, null);
- assert.equal(after.tasks[r.task.id].status, 'queued');
- });
- // ── capability ───────────────────────────────────────────────────────────────
- test('CAPABILITIES is exactly the documented set', () => {
- assert.deepEqual(CAPABILITIES, ['full', 'web', 'complex', 'verify', 'environment']);
- });
- test('capability is derived from the task type', () => {
- const expected = { build: 'full', web: 'web', complex: 'complex', verify: 'verify', environment: 'environment' };
- for (const [type, capability] of Object.entries(expected)) {
- assert.equal(capabilityForType(type), capability);
- assert.equal(createOne({ title: 'cap-' + type, type }).task.capability, capability);
- }
- assert.equal(capabilityForType('unknown-type'), 'full');
- assert.equal(createOne({ title: 'default-cap' }).task.capability, 'full');
- });
- test('capability may be overridden explicitly', () => {
- const r = createOne({ title: 'override', type: 'build', capability: 'web' });
- assert.equal(r.task.capability, 'web');
- });
- test('an invalid capability or type is rejected', () => {
- assert.throws(() => createOne({ title: 'x', capability: 'bogus' }), /Invalid capability: bogus/);
- assert.throws(() => createOne({ title: 'x', type: 'bogus' }), /Invalid type: bogus/);
- });
- test('capability flows into the dispatch record and the dry-run prompt', () => {
- const r = createOne({ title: 'cap-dispatch', type: 'complex' });
- const dispatch = nextDispatch(r.state, r.task.id);
- assert.equal(dispatch.capability, 'complex');
- assert.match(dispatch.command, /Capability: complex/);
- });
- // ── important task start choices ─────────────────────────────────────────────
- test('important tasks record all three start choices', () => {
- const r = createOne({
- title: 'important-task',
- type: 'build',
- important: true,
- executionMode: 'collaborative',
- security: 'planned',
- independentVerify: 'planned',
- });
- assert.equal(r.task.important, true);
- assert.equal(r.task.startChoices.executionMode, 'collaborative');
- assert.equal(r.task.startChoices.security, 'planned');
- assert.equal(r.task.startChoices.independentVerify, 'planned');
- });
- test('important tasks are rejected when a start choice is missing', () => {
- const base = { title: 'x', important: true };
- assert.throws(() => createOne({ ...base, executionMode: 'single', security: 'skip' }), /Important task requires: --independent-verify/);
- assert.throws(() => createOne({ ...base, security: 'skip', independentVerify: 'skip' }), /Important task requires: --execution-mode/);
- assert.throws(() => createOne({ ...base, executionMode: 'single', independentVerify: 'skip' }), /Important task requires: --security/);
- assert.throws(() => createOne(base), /Important task requires: --execution-mode, --security, --independent-verify/);
- });
- test('plain tasks fall back to the documented defaults', () => {
- const r = createOne({ title: 'plain' });
- assert.equal(r.task.important, false);
- assert.deepEqual(r.task.startChoices, {
- executionMode: DEFAULT_START_CHOICES.executionMode,
- security: DEFAULT_START_CHOICES.security,
- independentVerify: DEFAULT_START_CHOICES.independentVerify,
- });
- assert.equal(r.task.startChoices.executionMode, 'single');
- assert.equal(r.task.startChoices.security, 'skip');
- assert.equal(r.task.startChoices.independentVerify, 'skip');
- });
- test('invalid start choice values are rejected even on plain tasks', () => {
- assert.throws(() => createOne({ title: 'x', executionMode: 'parallel' }), /Invalid execution-mode: parallel/);
- assert.throws(() => createOne({ title: 'x', security: 'auto' }), /Invalid security: auto/);
- assert.throws(() => createOne({ title: 'x', independentVerify: 'yes' }), /Invalid independent-verify: yes/);
- });
- test('start choices are injected into the dry-run prompt, with the scan warning', () => {
- const r = createOne({
- title: 'choices-in-prompt',
- important: true,
- executionMode: 'collaborative',
- security: 'planned',
- independentVerify: 'planned',
- });
- const { command } = nextDispatch(r.state, r.task.id);
- assert.match(command, /Execution mode: collaborative/);
- assert.match(command, /Security: planned/);
- assert.match(command, /Independent verify: planned/);
- assert.match(command, /do not start a scan until the user confirms again/);
- });
- // ── dry-run dispatch ─────────────────────────────────────────────────────────
- test('dispatch is a dry run and never claims to execute', () => {
- const r = createOne({ title: 'dry-run' });
- const dispatch = nextDispatch(r.state, r.task.id);
- assert.equal(dispatch.execute, false);
- assert.equal(dispatch.requiresHumanCoordination, false);
- assert.equal(dispatch.taskId, r.task.id);
- assert.equal(dispatch.agent, 'generalist');
- assert.equal(dispatch.phase, 'specify');
- assert.equal(typeof dispatch.command, 'string');
- assert.ok(dispatch.command.length > 0);
- assert.match(dispatch.command, /^agent-run --headless /);
- });
- test('the dispatch record carries an argv array, not only a shell string', () => {
- const r = createOne({ title: 'argv-form' });
- const dispatch = nextDispatch(r.state, r.task.id);
- assert.ok(Array.isArray(dispatch.argv));
- assert.equal(dispatch.argv[0], 'agent-run');
- assert.equal(dispatch.argv[1], '--headless');
- assert.equal(dispatch.argv.length, 3);
- assert.equal(dispatch.command, `agent-run --headless ${shellQuote(dispatch.argv[2])}`);
- // The prompt embeds an apostrophe ("task's"), so the rendered line must escape it
- // rather than end the quoted region early.
- assert.ok(dispatch.command.includes("'\\''"), 'embedded quotes are escaped');
- });
- test('shell metacharacters in free-form task text stay inside a quoted argument', () => {
- // Regression: the dispatch line used to be built with JSON.stringify, which leaves
- // $(...), backticks, ${...} and ! live inside double quotes. The printed line is
- // meant to be pasted into a shell, so that was command execution.
- const r = createOne({ title: 'Fix $(touch pwned) `id` ${HOME} !boom' });
- const dispatch = nextDispatch(r.state, r.task.id);
- const prompt = dispatch.argv[dispatch.argv.length - 1];
- assert.ok(prompt.includes('$(touch pwned)'), 'argv keeps the raw text');
- assert.ok(prompt.includes('`id`'));
- assert.match(dispatch.command, /^agent-run --headless '/);
- const outsideQuoted = dispatch.command.replace(/'[^']*'/g, '');
- assert.ok(!/[$\`!]/.test(outsideQuoted), 'no live substitution outside quotes: ' + outsideQuoted);
- });
- test('shellQuote escapes embedded single quotes instead of breaking out', () => {
- assert.equal(shellQuote("a'b"), "'a'\\''b'");
- assert.equal(shellQuote('safe-token_1.2:3'), 'safe-token_1.2:3');
- assert.equal(shellQuote(''), "''");
- assert.equal(shellQuote('x; rm -rf /'), "'x; rm -rf /'");
- Note: Quoted as data in code: Runs `rm -rf` against a root or home-directory path
- Line 372 on GitHub
- });
- test('the prompt carries task, workspace, boundary and evidence fields', () => {
- const r = createOne({ title: 'prompt-fields', workspace: '<workspace>/client' });
- const { command } = nextDispatch(r.state, r.task.id);
- assert.match(command, /prompt-fields/);
- assert.match(command, /Phase: specify/);
- assert.match(command, /Workspace: <workspace>\/client/);
- assert.match(command, /Capability: full/);
- assert.match(command, /Write boundary/);
- assert.match(command, /Verifier independence/);
- assert.ok(command.includes('Evidence:'));
- assert.ok(command.includes('External actions:'));
- });
- test('each agent identity gets its own dry-run command', () => {
- const created = createOne({ title: 'agent-cmds', type: 'web' });
- const id = created.task.id;
- assert.match(nextDispatch(created.state, id).command, /^agent-run --headless /);
- assert.equal(nextDispatch(created.state, id).requiresCoordinatorCoordination, true);
- const afterSpecify = step(created.state, id, 'generalist'); // specify -> implement(frontend)
- const d = nextDispatch(afterSpecify, id);
- assert.equal(d.agent, 'frontend');
- assert.match(d.command, /^agent-run --profile frontend /);
- assert.equal(d.requiresCoordinatorCoordination, false);
- });
- test('an unapproved external action blocks its own step, not the internal stages', () => {
- const plain = createOne({ title: 'no-action' });
- assert.equal(nextDispatch(plain.state, plain.task.id).externalActionApproved, null);
- const gated = createOne({ title: 'deploy', externalAction: 'deploy' });
- // Internal work may proceed: the approval gates the external STEP, not the whole task.
- assert.equal(nextDispatch(gated.state, gated.task.id).externalActionApproved, false);
- // Drive the task to the stage that performs the action (externalAction.at defaults to implement).
- const atImplement = step(gated.state, gated.task.id, 'generalist');
- assert.equal(atImplement.tasks[gated.task.id].phase, 'implement');
- // Reporting the flag while still handing back a runnable argv was the hole: a host adapter that
- // executed the command would have bypassed the approval gate entirely.
- assert.throws(() => nextDispatch(atImplement, gated.task.id), /is not approved/);
- const approved = approveExternalAction(atImplement, gated.task.id, { approvedBy: 'ops', scope: 'deploy' }).state;
- assert.equal(nextDispatch(approved, gated.task.id).externalActionApproved, true);
- });
- test('an approval scope must EQUAL the action target, not merely appear in it', () => {
- const s = createOne({ title: 'prod-deploy', externalAction: 'deploy', externalTarget: 'production' });
- const atImplement = step(s.state, s.task.id, 'generalist');
- const wrong = approveExternalAction(atImplement, s.task.id, { approvedBy: 'ops', scope: 'staging' }).state;
- assert.throws(() => nextDispatch(wrong, s.task.id), /must equal the action's target exactly/);
- // A substring that merely mentions the target is not an authorisation either.
- const sneaky = approveExternalAction(atImplement, s.task.id, { approvedBy: 'ops', scope: 'not production' }).state;
- assert.throws(() => nextDispatch(sneaky, s.task.id), /must equal the action's target exactly/);
- const right = approveExternalAction(atImplement, s.task.id, { approvedBy: 'ops', scope: 'production' }).state;
- assert.equal(nextDispatch(right, s.task.id).externalActionApproved, true);
- });
- test('dispatch throws for non-dispatchable tasks', () => {
- const done = createOne({ title: 'done-task' });
- const finished = FLOWS.build.actors.reduce((state, actor) => step(state, done.task.id, actor), done.state);
- assert.throws(() => nextDispatch(finished, done.task.id), /is already done/);
- const blocked = createOne({ title: 'blocked-task' });
- blocked.state.tasks[blocked.task.id].status = 'blocked';
- assert.throws(() => nextDispatch(blocked.state, blocked.task.id), /blocked and cannot be dispatched/);
- const locked = createOne({ title: 'locked-task' });
- const held = claimTask(locked.state, locked.task.id, 'generalist').state;
- assert.throws(() => nextDispatch(held, locked.task.id), /already held by generalist/);
- const orphan = createOne({ title: 'no-agent' });
- orphan.state.tasks[orphan.task.id].assignedAgent = null;
- assert.throws(() => nextDispatch(orphan.state, orphan.task.id), /no assigned agent/);
- // Any agent id a roster declares is dispatchable: the generic fallback template
- // covers teams whose agents are not called generalist/frontend/specialist.
- const custom = createOne({ title: 'custom-agent' });
- custom.state.tasks[custom.task.id].assignedAgent = 'ghost';
- const dispatched = nextDispatch(custom.state, custom.task.id);
- assert.deepEqual(dispatched.argv.slice(0, 3), ['agent-run', '--profile', 'ghost']);
- });
- // ── external-action approve gate ─────────────────────────────────────────────
- test('an external action stays blocked until approved', () => {
- const r = createOne({ title: 'publish', externalAction: 'publish' });
- assert.equal(canRunExternalAction(r.state, r.task.id), false);
- assert.equal(r.task.externalAction.approved, false);
- assert.equal(r.task.externalAction.approvedBy, null);
- assert.equal(r.task.externalAction.scope, null);
- });
- test('tasks without an external action report null', () => {
- const r = createOne({ title: 'no-external' });
- assert.equal(canRunExternalAction(r.state, r.task.id), null);
- assert.throws(() => approveExternalAction(r.state, r.task.id, { approvedBy: 'user', scope: 'x' }), /has no external action to approve/);
- });
- test('approval requires both approvedBy and scope', () => {
- const r = createOne({ title: 'gate', externalAction: 'deploy' });
- assert.throws(() => approveExternalAction(r.state, r.task.id, {}), /Approval requires approvedBy and scope/);
- assert.throws(() => approveExternalAction(r.state, r.task.id, { approvedBy: 'user' }), /Approval requires approvedBy and scope/);
- assert.throws(() => approveExternalAction(r.state, r.task.id, { scope: 'deploy main' }), /Approval requires approvedBy and scope/);
- });
- test('an explicit approval unlocks the external action and is recorded', () => {
- const r = createOne({ title: 'approve-me', externalAction: 'deploy' });
- const out = approveExternalAction(r.state, r.task.id, { approvedBy: 'user', scope: 'deploy staging' });
- assert.equal(canRunExternalAction(out.state, r.task.id), true);
- assert.equal(out.task.externalAction.approvedBy, 'user');
- assert.equal(out.task.externalAction.scope, 'deploy staging');
- assert.equal(out.state.events.at(-1).type, 'external_action_approved');
- assert.equal(r.state.tasks[r.task.id].externalAction.approved, false, 'input state must stay unapproved');
- });
- // ── fail-closed on blocked tasks ─────────────────────────────────────────────
- test('a blocked task refuses claim, complete and approve', () => {
- const blocked = createOne({ title: 'blocked-guard', externalAction: 'deploy' });
- blocked.state.tasks[blocked.task.id].status = 'blocked';
- assert.throws(() => claimTask(blocked.state, blocked.task.id, 'generalist'), /blocked and cannot be claimed/);
- assert.throws(() => completeStage(blocked.state, blocked.task.id, 'generalist'), /blocked and cannot complete/);
- assert.throws(() => approveExternalAction(blocked.state, blocked.task.id, { approvedBy: 'user', scope: 'x' }), /blocked and cannot approve/);
- });
- // ── recover ──────────────────────────────────────────────────────────────────
- test('recover clears a stranded lock, keeps the phase and re-queues the task', () => {
- const r = createOne({ title: 'stranded', type: 'complex' });
- const held = claimTask(r.state, r.task.id, 'generalist').state;
- const out = recoverTask(held, r.task.id, { reason: 'worker process ended during probe' });
- assert.equal(out.task.lock, null);
- assert.equal(out.task.status, 'queued');
- assert.equal(out.task.assignedAgent, 'generalist');
- assert.equal(out.task.phase, 'specify');
- assert.equal(out.task.evidence.at(-1).agent, 'recovery');
- assert.match(out.task.evidence.at(-1).text, /^Recovered: worker process ended/);
- const recoveredEvent = out.state.events.at(-1);
- assert.equal(recoveredEvent.type, 'recovered');
- assert.equal(recoveredEvent.reason, 'worker process ended during probe');
- assert.equal(recoveredEvent.previous.status, 'in_progress');
- });
- test('recover also accepts a blocked task', () => {
- const r = createOne({ title: 'blocked-recoverable' });
- r.state.tasks[r.task.id].status = 'blocked';
- const out = recoverTask(r.state, r.task.id, { reason: 'block cleared by coordinator' });
- assert.equal(out.task.status, 'queued');
- });
- test('recover requires a reason and refuses unrecoverable tasks', () => {
- const r = createOne({ title: 'recover-guards' });
- assert.throws(() => recoverTask(r.state, r.task.id, {}), /Recovery requires a reason/);
- assert.throws(() => recoverTask(r.state, r.task.id, { reason: ' ' }), /Recovery requires a reason/);
- assert.throws(() => recoverTask(r.state, r.task.id, { reason: 'nothing stranded' }), /is not recoverable/);
- });
- test('a done task cannot be recovered', () => {
- const r = createOne({ title: 'done-recover' });
- const finished = FLOWS.build.actors.reduce((state, actor) => step(state, r.task.id, actor), r.state);
- assert.equal(finished.tasks[r.task.id].phase, 'done');
- assert.throws(() => recoverTask(finished, r.task.id, { reason: 'late' }), /is not recoverable/);
- });
- // ── sanitisation and dry-run guards on the sources ───────────────────────────
- const RUNTIME_SOURCES = ['lib.mjs', 'orchestrator.mjs'];
- for (const file of RUNTIME_SOURCES) {
- test('sanitisation: ' + file + ' contains no absolute machine path', () => {
- const src = readFileSync(new URL(file, import.meta.url), 'utf8');
- assert.ok(!/[A-Za-z]:[\\/]/.test(src), 'drive-letter absolute path found in ' + file);
- // Assembled from fragments so this guard does not itself look like a machine path.
- assert.ok(!src.includes('/' + 'Users/'), 'user home path found in ' + file);
- assert.ok(!src.includes('\\' + 'Users\\'), 'windows user path found in ' + file);
- assert.ok(!/\bTODO\b/.test(src), 'placeholder left in ' + file);
- });
- test('dry run: ' + file + ' cannot spawn a process', () => {
- const src = readFileSync(new URL(file, import.meta.url), 'utf8');
- assert.ok(!src.includes('child_process'), 'child_process imported in ' + file);
- assert.ok(!/\bspawnSync?\s*\(/.test(src), 'spawn call found in ' + file);
- assert.ok(!/\bexecFile\s*\(/.test(src), 'execFile call found in ' + file);
- // A bare exec( is the process form; RegExp.prototype.exec() is not, and flagging it made this
- // guard cry wolf on ordinary parsing code.
- assert.ok(!/(?<![.\w])exec\s*\(/.test(src), 'exec call found in ' + file);
- });
- }
- // ── the task packet is an artifact, not prose (R3) ───────────────────────────────────
- test('the specify stage cannot end without a spec and an acceptance criterion', () => {
- const state = createState();
- const created = createTask(state, { title: 'no-packet' });
- const claimed = claimTask(created.state, created.task.id, created.task.assignedAgent).state;
- assert.throws(
- () => completeStage(claimed, created.task.id, created.task.assignedAgent, {}),
- /without its work products: --spec, --acceptance/,
- );
- const done = completeStage(claimed, created.task.id, created.task.assignedAgent, {
- spec: 'add a settings page', acceptance: 'npm test exits 0', nonGoals: 'no deploy',
- }).state;
- const task = done.tasks[created.task.id];
- assert.equal(task.spec, 'add a settings page');
- assert.equal(task.acceptance, 'npm test exits 0');
- assert.equal(task.nonGoals, 'no deploy');
- assert.equal(task.phase, 'implement');
- });
- test('the packet may be supplied at create time instead', () => {
- const state = createState();
- const created = createTask(state, { title: 'packet-upfront', spec: 's', acceptance: 'a' });
- const claimed = claimTask(created.state, created.task.id, created.task.assignedAgent).state;
- const done = completeStage(claimed, created.task.id, created.task.assignedAgent, {}).state;
- assert.equal(done.tasks[created.task.id].phase, 'implement');
- });
- // ── the rules an independent review found were only documented ──────────────────────
- test('a resource held by a live task cannot be claimed by another task', () => {
- const state = createState();
- const a = createTask(state, { title: 'A', resources: ['src/foo.ts', 'src/bar.ts'] });
- const b = createTask(a.state, { title: 'B', resources: ['src/foo.ts'] });
- const c = createTask(b.state, { title: 'C', resources: ['src/other.ts'] });
- const held = claimTask(c.state, a.task.id, a.task.assignedAgent).state;
- // overlapping resource -> refused, and the message names who holds it
- assert.throws(() => claimTask(held, b.task.id, b.task.assignedAgent), /resource already held by src\/foo\.ts/);
- // disjoint resource -> fine
- assert.equal(claimTask(held, c.task.id, c.task.assignedAgent).state.tasks[c.task.id].lock.owner, c.task.assignedAgent);
- // once the holder completes the stage, the lock is released and the resource is free
- const released = completeStage(held, a.task.id, a.task.assignedAgent, { spec: 's', acceptance: 'a' }).state;
- assert.equal(released.tasks[a.task.id].lock, null);
- assert.ok(claimTask(released, b.task.id, b.task.assignedAgent).state.tasks[b.task.id].lock);
- });
- test('a verification PASS must carry its witness', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'witness' });
- let s = completeStage(claimTask(created, task.id, created.tasks[task.id].assignedAgent).state, task.id,
- created.tasks[task.id].assignedAgent, { spec: 's', acceptance: 'a' }).state;
- const implAgent = s.tasks[task.id].assignedAgent;
- s = completeStage(claimTask(s, task.id, implAgent).state, task.id, implAgent, {}).state;
- assert.equal(s.tasks[task.id].phase, 'verify');
- const verifier = s.tasks[task.id].assignedAgent;
- s = claimTask(s, task.id, verifier).state;
- assert.throws(() => completeStage(s, task.id, verifier, {}), /cannot PASS without criterion-linked evidence/);
- const passed = completeStage(s, task.id, verifier, { evidence: { grade: 'E2', text: 're-ran it', checkedBy: 'peer-x' } }).state;
- // the verifier's record is a witness on the verification, not evidence authored by the verifier
- assert.equal(passed.tasks[task.id].verification.witness.grade, 'E2');
- assert.equal(passed.tasks[task.id].evidence.length, 0);
- });
- test('an important task may not self-verify, even at L1', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'flagged', rigor: 'L1', important: true,
- executionMode: 'collaborative', security: 'planned', independentVerify: 'planned' });
- const done = driveToDone(created, task.id, 'ran it');
- assert.equal(done.tasks[task.id].verification.selfVerified, false, 'important work needs an independent verifier');
- assert.notEqual(done.tasks[task.id].verification.by, done.tasks[task.id].evidence.at(-1)?.agent);
- });
- test('three failed verifications block the task instead of retrying forever', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'stuck' });
- let s = created;
- for (let round = 0; round < 3; round++) {
- // walk forward until the task is actually in the verify stage again
- let t = s.tasks[task.id];
- while (t.phase !== 'verify') {
- const agent = t.assignedAgent;
- s = claimTask(s, task.id, agent).state;
- s = completeStage(s, task.id, agent, t.phase === 'specify' ? { spec: 's', acceptance: 'a' } : {}).state;
- t = s.tasks[task.id];
- }
- const v = s.tasks[task.id];
- const verifier = v.assignedAgent;
- s = claimTask(s, task.id, verifier).state;
- s = failVerification(s, task.id, verifier, { criteria: 'not met', findings: 'round ' + (round + 1) }).state;
- }
- const final = s.tasks[task.id];
- assert.equal(final.verification.attempts, 3);
- assert.equal(final.status, 'blocked', 'the third failure must block the task');
- assert.throws(() => claimTask(s, task.id, final.assignedAgent), /blocked/);
- });
- // ── a check that exists must also hold on every path into it ────────────────────────
- test('E1 evidence carrying explicit nulls is refused, not just undefined', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'nulls', rigor: 'L1' });
- const claimed = claimTask(created, task.id, task.assignedAgent).state;
- // This is the shape the CLI produced: the flags were absent, so it filled them with null,
- // and a check for `undefined` let a command-less, exit-code-less E1 through to done.
- const nulls = { grade: 'E1', text: 'tests ok', command: null, exitCode: null, revision: null };
- assert.throws(
- () => completeStage(claimed, task.id, task.assignedAgent, { evidence: nulls, spec: 's', acceptance: 'a' }),
- /E1 evidence must carry command, exitCode, revision/,
- );
- });
- test('the CLI cannot record an E1 without its fields (the layer that was broken)', async () => {
- const dir = mkdtempSync(join(tmpdir(), 'e1-cli-'));
- const stateFile = join(dir, 'queue.json');
- const cli = fileURLToPath(new URL('./orchestrator.mjs', import.meta.url));
- const env = { ORCHESTRATOR_LOCK_WAIT_MS: '300' };
- const run = (args) => runCli(cli, stateFile, args, env);
- assert.equal((await run(['init'])).code, 0);
- const created = JSON.parse((await run(['create', '--title', 'e1', '--type', 'build'])).out);
- const id = created.id;
- // walk to the evidence phase, acting as whoever the queue assigns
- for (let i = 0; i < 6; i++) {
- const task = JSON.parse(readFileSync(stateFile, 'utf8')).tasks[id];
- if (task.phase === 'evidence' || task.phase === 'done') break;
- const agent = task.assignedAgent;
- await run(['claim', '--task', id, '--agent', agent]);
- const args = ['complete', '--task', id, '--agent', agent, '--evidence', 'stage done'];
- if (task.phase === 'specify') args.push('--spec', 's', '--acceptance', 'a');
- await run(args);
- }
- const before = JSON.parse(readFileSync(stateFile, 'utf8')).tasks[id];
- assert.equal(before.phase, 'evidence', 'the walk should stop at the evidence phase');
- await run(['claim', '--task', id, '--agent', before.assignedAgent]);
- const bad = await run(['complete', '--task', id, '--agent', before.assignedAgent,
- '--evidence-grade', 'E1', '--evidence', 'tests ok']);
- assert.equal(bad.code, 1, 'an E1 with no command, exit code or revision must be refused');
- assert.match(bad.err, /must carry command, exitCode, revision/);
- const after = JSON.parse(readFileSync(stateFile, 'utf8')).tasks[id];
- assert.notEqual(after.phase, 'done');
- assert.ok(!(after.evidence || []).some((e) => e.grade === 'E1' && e.command == null), 'no null-bearing E1 may be stored');
- });
- test('an L3 task must state the three start choices, even without --important', () => {
- const state = createState();
- // L3 is consequential by definition, so the intake gate applies whether or not the caller says so.
- assert.throws(
- () => createTask(state, { title: 'consequential', rigor: 'L3' }),
- /Important task requires: --execution-mode, --security, --independent-verify/,
- );
- const ok = createTask(state, { title: 'consequential', rigor: 'L3',
- executionMode: 'collaborative', security: 'planned', independentVerify: 'planned' });
- assert.equal(ok.task.important, true, 'an L3 task is important by definition');
- assert.equal(ok.task.startChoicesSource, 'stated');
- });
- // ── the other grades, and the L1 rung of the rigor ladder ────────────────────────────
- test('E2 must name a peer, and the peer must not be the author', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'peer', rigor: 'L1' });
- const claimed = claimTask(created, task.id, task.assignedAgent).state;
- const payload = (ev) => ({ evidence: ev, spec: 's', acceptance: 'a' });
- assert.throws(
- () => completeStage(claimed, task.id, task.assignedAgent, payload({ grade: 'E2', text: 'looks right' })),
- /must name the peer/,
- );
- assert.throws(
- () => completeStage(claimed, task.id, task.assignedAgent, payload({ grade: 'E2', text: 'x', checkedBy: task.assignedAgent })),
- /must differ from the author/,
- );
- const ok = completeStage(claimed, task.id, task.assignedAgent,
- payload({ grade: 'E2', text: 're-ran it', checkedBy: 'someone-else', steps: 'node --test' })).state;
- assert.equal(ok.tasks[task.id].evidence.at(-1).grade, 'E2');
- });
- test('E4 must say what is planned', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'planned', rigor: 'L1' });
- const claimed = claimTask(created, task.id, task.assignedAgent).state;
- assert.throws(
- () => completeStage(claimed, task.id, task.assignedAgent, { evidence: { grade: 'E4', text: 'later' }, spec: 's', acceptance: 'a' }),
- /say what is planned/,
- );
- const ok = completeStage(claimed, task.id, task.assignedAgent,
- { evidence: { grade: 'E4', plan: 'run the suite after the refactor' }, spec: 's', acceptance: 'a' }).state;
- assert.equal(ok.tasks[task.id].evidence.at(-1).grade, 'E4');
- });
- test('L1 may be self-verified, L2 may not', () => {
- const pack = { spec: 's', acceptance: 'a' };
- // L1: the implementer completes the verify stage, and the record says so.
- const l1 = createTask(createState(), { title: 'local', rigor: 'L1' });
- const done = driveToDone(l1.state, l1.task.id, 'ran the unit tests: exit 0');
- assert.equal(done.tasks[l1.task.id].phase, 'done');
- assert.equal(done.tasks[l1.task.id].verification.selfVerified, true);
- assert.equal(
- done.tasks[l1.task.id].verification.by,
- done.tasks[l1.task.id].evidence.at(-1).agent,
- 'for L1 the verifier is the implementer',
- );
- // L2: verification must come from a different agent, so the same walk cannot even reach verify.
- const l2 = createTask(createState(), { title: 'shared', rigor: 'L2' });
- const l2done = driveToDone(l2.state, l2.task.id, 'ran the suite: exit 0');
- assert.equal(l2done.tasks[l2.task.id].phase, 'done');
- const verdict = l2done.tasks[l2.task.id].verification;
- assert.notEqual(verdict.by, undefined);
- assert.ok(!verdict.selfVerified, 'L2 must not be self-verified');
- assert.ok(pack);
- });
- // ── the intake gate must not answer for the user ─────────────────────────────────────
- test('a task with an external action must state all three start choices', () => {
- const state = createState();
- // Refused either because it is not L3 or because the three choices are missing - both are the
- // point: an external action has to be consequential AND explicitly answered for.
- assert.throws(
- () => createTask(state, { title: 'deploy', type: 'build', externalAction: 'deploy' }),
- /must be L3|requires all three start choices/,
- );
- assert.throws(
- () => createTask(state, { title: 'deploy', type: 'build', rigor: 'L3', externalAction: 'deploy' }),
- /Important task requires: --execution-mode, --security, --independent-verify/,
- );
- const ok = createTask(state, { title: 'deploy', type: 'build', rigor: 'L3', externalAction: 'deploy', important: true,
- executionMode: 'collaborative', security: 'planned', independentVerify: 'planned' });
- assert.equal(ok.task.startChoicesSource, 'stated');
- });
- test('a routine task records that its start choices were assumed, not stated', () => {
- const state = createState();
- const created = createTask(state, { title: 'routine', type: 'build' }).task;
- assert.equal(created.startChoicesSource, 'defaults');
- assert.deepEqual(created.startChoices, { executionMode: 'single', security: 'skip', independentVerify: 'skip' });
- });
- // ── rigor decides what is mandatory (R1) and who may verify (R2) ─────────────────────
- test('L2 requires E1 or E2 evidence: a self-report cannot close it', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'shared', rigor: 'L2' });
- const passed = { ...created.tasks[task.id].verification, status: 'passed', lastResult: 'pass' };
- const atEvidence = () => ({
- ...created,
- tasks: {
- ...created.tasks,
- [task.id]: {
- ...created.tasks[task.id],
- phase: 'evidence',
- lock: null,
- status: 'queued',
- assignedAgent: 'specialist',
- verification: passed,
- },
- },
- });
- // An empty record is refused.
- assert.throws(
- () => completeStage(claimTask(atEvidence(), task.id, 'specialist').state, task.id, 'specialist', {}),
- /requires at least one graded evidence record/,
- );
- // A self-report is refused too: at L2 the floor is E1 or E2, and E3 is neither.
- const claimed = claimTask(atEvidence(), task.id, 'specialist').state;
- assert.throws(
- () => completeStage(claimed, task.id, 'specialist', { evidence: { grade: 'E3', text: 'looks fine to me' } }),
- /requires E1 or E2 evidence/,
- );
- // E2 closes it.
- const done = completeStage(claimed, task.id, 'specialist',
- { evidence: { grade: 'E2', text: 'a peer re-ran the suite', checkedBy: 'peer-x' } }).state;
- assert.equal(done.tasks[task.id].phase, 'done');
- });
- test('an L3 verifier may not also be an author of the evidence', () => {
- // One agent owns both verify and evidence, while a different agent implements.
- const roster = {
- agents: {
- impl: { cost: 1, scores: { specify: 3, triage: 1, implement: 3, verify: 1, evidence: 1, environment: 1, domain: 1 } },
- solo: { cost: 2, scores: { specify: 2, triage: 1, implement: 1, verify: 3, evidence: 3, environment: 1, domain: 1 } },
- other: { cost: 3, scores: { specify: 1, triage: 1, implement: 1, verify: 2, evidence: 2, environment: 1, domain: 3 } },
- },
- routes: { specify: 'impl', implement: 'impl', verify: 'solo', evidence: 'solo' },
- };
- const state = createState(roster);
- const { state: created, task } = createTask(state, { title: 'consequential', rigor: 'L3', executionMode: 'collaborative', security: 'planned', independentVerify: 'planned' });
- const evidence = { grade: 'E1', text: 'gate evidence', command: 'node scripts/gate.mjs', exitCode: 0, revision: 'abc123' };
- assert.throws(
- () => driveToDone(created, task.id, evidence),
- /also produced evidence on this task|not independent/,
- );
- });
- // ── concurrency: two writers at once ─────────────────────────────────────────
- // ── rigor: how much verification a change carries ─────────────────────────────────────
- /** Drive a task from its current phase to done, optionally supplying evidence at the last step. */
- function driveToDone(state, id, evidence = DEFAULT_EVIDENCE) {
- let current = state;
- for (let i = 0; i < 10; i++) {
- const task = current.tasks[id];
- if (task.phase === 'done') return current;
- const agent = task.assignedAgent;
- const at = current.tasks[id].phase;
- current = claimTask(current, id, agent).state;
- // A verification PASS and a domain review both carry a record of what they checked.
- const payload = at === 'specify'
- ? { spec: 'the change under test', acceptance: 'the suite exits 0' }
- : ((at === 'evidence' || at === 'verify' || at === 'domain_review') ? { evidence: asEvidence(evidence) } : {});
- current = completeStage(current, id, agent, payload).state;
- }
- throw new Error('task did not reach done: ' + current.tasks[id].phase);
- }
- test('rigor defaults to L2 and must be a level the engine accepts', () => {
- const state = createState();
- assert.equal(createTask(state, { title: 'defaulted' }).task.rigor, 'L2');
- for (const level of RIGOR_LEVELS) {
- // L3 is consequential, so it has to answer the intake questions; L1 and L2 need not.
- const extra = level === 'L3' ? { executionMode: 'collaborative', security: 'planned', independentVerify: 'planned' } : {};
- assert.equal(createTask(state, { title: 'r-' + level, rigor: level, ...extra }).task.rigor, level);
- }
- assert.throws(() => createTask(state, { title: 'bad', rigor: 'L9' }), /Invalid rigor/);
- });
- test('an external action must be L3 - L1 and L2 are both refused', () => {
- const state = createState();
- assert.throws(
- () => createTask(state, { title: 'deploy', rigor: 'L1', externalAction: 'deploy' }),
- /must be L3/,
- );
- assert.throws(
- () => createTask(state, { title: 'deploy', rigor: 'L2', externalAction: 'deploy' }),
- /must be L3/,
- 'L2 only requires a different verifier, which is not enough for something irreversible',
- );
- const ok = createTask(state, { title: 'deploy', rigor: 'L3', externalAction: 'deploy', important: true,
- executionMode: 'collaborative', security: 'planned', independentVerify: 'planned' });
- assert.equal(ok.task.rigor, 'L3');
- });
- test('a bare evidence string is stored as E3 and labelled, never as E1', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'e3', rigor: 'L1' });
- const claimed = claimTask(created, task.id, task.assignedAgent).state;
- const done = completeStage(claimed, task.id, task.assignedAgent, { evidence: 'looks fine to me', spec: 's', acceptance: 'a' }).state;
- const record = done.tasks[task.id].evidence.at(-1);
- assert.equal(record.grade, 'E3');
- assert.match(record.note, /downgraded to E3/);
- assert.equal(record.text, 'looks fine to me');
- });
- test('E1 evidence must carry the command, the exit code and the revision', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'e1', rigor: 'L1' });
- const claimed = claimTask(created, task.id, task.assignedAgent).state;
- const partial = { grade: 'E1', text: 'tests pass', command: 'npm test' }; // no exitCode, no revision
- assert.throws(
- () => completeStage(claimed, task.id, task.assignedAgent, { evidence: partial, spec: 's', acceptance: 'a' }),
- /E1 evidence must carry exitCode, revision/,
- );
- const full = { grade: 'E1', text: 'tests pass', command: 'npm test', exitCode: 0, revision: 'abc123' };
- const done = completeStage(claimed, task.id, task.assignedAgent, { evidence: full, spec: 's', acceptance: 'a' }).state;
- assert.equal(done.tasks[task.id].evidence.at(-1).grade, 'E1');
- assert.equal(done.tasks[task.id].evidence.at(-1).exitCode, 0);
- });
- test('an L3 task requires E1 evidence', () => {
- const state = createState();
- const { state: created, task } = createTask(state, { title: 'consequential', rigor: 'L3', executionMode: 'collaborative', security: 'planned', independentVerify: 'planned' });
- // Self-report only: the last step must refuse.
- assert.throws(() => driveToDone(created, task.id, 'I checked it'), /requires E1 evidence/);
- // With a reproducible artifact it goes through.
- const finished = driveToDone(created, task.id, {
- grade: 'E1', text: 'gate evidence', command: 'node scripts/gate.mjs', exitCode: 0, revision: 'abc123',
- });
- assert.equal(finished.tasks[task.id].phase, 'done');
- assert.ok(EVIDENCE_GRADES.includes(finished.tasks[task.id].evidence.at(-1).grade));
- });
- function runCli(cliPath, stateFile, args, extraEnv = {}) {
- return new Promise((resolve) => {
- const child = spawn(process.execPath, [cliPath, ...args], {
- env: { ...process.env, ORCHESTRATOR_STATE: stateFile, ...extraEnv },
- stdio: ['ignore', 'pipe', 'pipe'],
- });
- let out = '';
- let err = '';
- child.stdout.on('data', (d) => { out += String(d); });
- child.stderr.on('data', (d) => { err += String(d); });
- child.on('close', (code) => resolve({ code, out, err }));
- });
- }
- test('four concurrent creates: every task survives (no lost update)', async () => {
- const dir = mkdtempSync(join(tmpdir(), 'agent-orchestra-'));
- const stateFile = join(dir, 'queue.json');
- const cli = fileURLToPath(new URL('./orchestrator.mjs', import.meta.url));
- const init = await runCli(cli, stateFile, ['init']);
- assert.equal(init.code, 0, init.err);
- // All four processes read the same revision and then try to write. Without the
- // read-modify-write lock they all allocate task-0001 and three writes are lost.
- const results = await Promise.all(
- [1, 2, 3, 4].map((n) => runCli(cli, stateFile, ['create', '--title', 'race-' + n, '--type', 'build'])),
- );
- const failures = results.filter((r) => r.code !== 0).map((r) => r.err.trim());
- assert.deepEqual(failures, [], 'every concurrent create should succeed: ' + JSON.stringify(failures));
- const final = JSON.parse(readFileSync(stateFile, 'utf8'));
- assert.deepEqual(Object.keys(final.tasks).sort(), ['task-0001', 'task-0002', 'task-0003', 'task-0004']);
- assert.equal((final.events ?? []).filter((e) => e.type === 'created').length, 4);
- assert.equal(final.nextTaskNumber, 5);
- // Atomic write + lock leave nothing behind.
- assert.deepEqual(readdirSync(dir).sort(), ['queue.json']);
- });
- test('the lock refuses a second writer, and a stale lock is recovered', async () => {
- const dir = mkdtempSync(join(tmpdir(), 'agent-orchestra-'));
- const stateFile = join(dir, 'queue.json');
- const lockFile = stateFile + '.lock';
- const cli = fileURLToPath(new URL('./orchestrator.mjs', import.meta.url));
- const fast = { ORCHESTRATOR_LOCK_WAIT_MS: '300' };
- await runCli(cli, stateFile, ['init']);
- // A fresh lock (another process may be mid-write) makes this command refuse to write.
- writeFileSync(lockFile, 'someone-else\n', 'utf8');
- const blocked = await runCli(cli, stateFile, ['create', '--title', 'blocked', '--type', 'build'], fast);
- assert.equal(blocked.code, 1);
- assert.match(blocked.err, /locked by another process/);
- assert.deepEqual(Object.keys(JSON.parse(readFileSync(stateFile, 'utf8')).tasks), []);
- // A lock left by a crashed process is stolen once it is stale, so it never wedges.
- const old = new Date(Date.now() - 60_000);
- utimesSync(lockFile, old, old);
- const recovered = await runCli(cli, stateFile, ['create', '--title', 'after-crash', '--type', 'build'], fast);
- assert.equal(recovered.code, 0, recovered.err);
- assert.deepEqual(Object.keys(JSON.parse(readFileSync(stateFile, 'utf8')).tasks), ['task-0001']);
- assert.deepEqual(readdirSync(dir).sort(), ['queue.json']);
- });
- test('the CLI resolves the queue file relative to the module', () => {
- const src = readFileSync(new URL('orchestrator.mjs', import.meta.url), 'utf8');
- assert.match(src, /new URL\('TASK-QUEUE\.json', import\.meta\.url\)/);
- });
- test('the default agent roster is available to callers', () => {
- assert.deepEqual(AGENTS, ['generalist', 'frontend', 'specialist']);
- });
- // ── roster genericity: any size, any agent names ─────────────────────────────
- const SIX_AGENT_ROSTER = {
- agents: {
- planner: { cost: 2, scores: { specify: 3, implement: 1, verify: 1, evidence: 3 } },
- builder1: { cost: 4, specialties: ['full'], scores: { specify: 1, implement: 3, verify: 1, evidence: 1 } },
- builder2: { cost: 5, specialties: ['web'], scores: { specify: 1, implement: 2, verify: 1, evidence: 1 } },
- auditorA: { cost: 6, specialties: ['verify'], scores: { specify: 1, implement: 1, verify: 3, evidence: 2 } },
- auditorB: { cost: 7, specialties: ['verify'], scores: { specify: 1, implement: 1, verify: 3, evidence: 2 } },
- scribe: { cost: 1, scores: { specify: 1, implement: 1, verify: 0, evidence: 3 } },
- },
- routes: { specify: 'planner', evidence: 'scribe', implement: { build: 'builder1', web: 'builder2' } },
- };
- test('a six-agent roster with custom names routes without any code change', () => {
- const created = createTask(createState(SIX_AGENT_ROSTER), { title: 'six-agents', type: 'build' });
- const state = created.state;
- const id = created.task.id;
- assert.equal(Object.keys(state.roster.agents).length, 6);
- assert.equal(implementerForType('build', state.roster), 'builder1');
- assert.equal(implementerForType('web', state.roster), 'builder2');
- assert.equal(ownerFor('specify', { roster: state.roster }), 'planner');
- assert.equal(ownerFor('evidence', { roster: state.roster }), 'scribe');
- // The verifier is scored, never the implementer: auditorA outranks auditorB on cost.
- assert.equal(selectVerifier('builder1', state.roster), 'auditorA');
- assert.equal(selectVerifier('auditorA', state.roster), 'auditorB');
- // And the whole pipeline runs end to end for a custom roster.
- let s = step(state, id, 'planner');
- s = step(s, id, 'builder1');
- assert.equal(s.tasks[id].assignedAgent, 'auditorA');
- s = step(s, id, 'auditorA');
- s = step(s, id, 'scribe');
- assert.equal(s.tasks[id].phase, 'done');
- });
- test('the verifier ranking scales and stays deterministic', () => {
- const roster = {
- agents: {
- a: { cost: 9, scores: { verify: 2 } },
- b: { cost: 1, scores: { verify: 2 } },
- c: { cost: 5, scores: { verify: 1 } },
- impl: { cost: 1, scores: { verify: 3, implement: 3 } },
- },
- };
- // impl has the highest score but is excluded; b beats a on cost at equal score.
- assert.equal(selectVerifier('impl', roster), 'b');
- assert.equal(selectAgent('verify', { roster }).id, 'impl');
- assert.equal(selectAgent('verify', { roster, exclude: ['impl'] }).id, 'b');
- // Capability match outranks cost at equal score.
- const specialised = {
- agents: {
- cheap: { cost: 1, scores: { verify: 2 } },
- webby: { cost: 9, specialties: ['web'], scores: { verify: 2 } },
- },
- };
- assert.equal(selectAgent('verify', { roster: specialised, capability: 'web' }).id, 'webby');
- });
- test('a roster with nobody scored for verification fails closed', () => {
- const roster = { agents: { solo: { cost: 1, scores: { implement: 3 } } } };
- assert.throws(() => selectVerifier('other', roster), /No agent in the roster is scored for verification/);
- });
- test('validateRoster rejects malformed rosters loudly', () => {
- assert.throws(() => validateRoster({}), /must look like/);
- assert.throws(() => validateRoster({ agents: {} }), /at least one agent/);
- assert.throws(() => validateRoster({ agents: { a: { scores: { verify: 4 } } } }), /score for verify must be a number 0-3/);
- assert.throws(() => validateRoster({ agents: { a: { scores: { auditing: 1 } } } }), /unknown role "auditing"/);
- assert.throws(() => validateRoster({ agents: { a: { cost: 'cheap' } } }), /cost must be a number/);
- assert.throws(() => validateRoster({ agents: { a: { specialties: 'web' } } }), /specialties must be an array/);
- assert.doesNotThrow(() => validateRoster(SIX_AGENT_ROSTER));
- });
- // ── the verification gate ────────────────────────────────────────────────────
- function atVerify(title = 'gate-task', type = 'build') {
- const created = createOne({ title, type });
- const id = created.task.id;
- let s = step(created.state, id, 'generalist'); // specify -> implement
- s = step(s, id, implementerForType(type)); // implement -> verify
- const verifier = s.tasks[id].assignedAgent;
- s = claimTask(s, id, verifier).state;
- return { state: s, id, verifier, implementer: implementerForType(type) };
- }
- test('a failed verification blocks the gate and returns the task to implement', () => {
- const g = atVerify();
- const out = failVerification(g.state, g.id, g.verifier, { criteria: 'acceptance test X fails' });
- const task = out.task;
- assert.equal(task.phase, 'implement');
- assert.equal(task.assignedAgent, g.implementer);
- assert.equal(task.verification.status, 'failed');
- assert.equal(task.verification.blocked, true);
- assert.equal(task.verification.lastResult, 'fail');
- assert.equal(task.verification.criteria, 'acceptance test X fails');
- assert.equal(task.verification.by, g.verifier);
- assert.equal(task.lock, null);
- assert.equal(out.state.events.at(-1).type, 'verification_failed');
- });
- test('failVerification demands the verify stage and a stated failure', () => {
- const r = createOne({ title: 'not-verifying' });
- // Task is in specify: not a verify stage.
- assert.throws(() => failVerification(r.state, r.task.id, 'generalist', { findings: 'x' }), /not in a stage that can fail/);
- const g = atVerify();
- assert.throws(() => failVerification(g.state, g.id, g.verifier, {}), /must state the failing criteria or the findings/);
- // Only the lock holder may fail the stage.
- assert.throws(() => failVerification(g.state, g.id, 'someone-else', { findings: 'x' }), /must be claimed by/);
- });
- test('the gate cannot be walked past: no done without a pass or an override', () => {
- const g = atVerify();
- const failed = failVerification(g.state, g.id, g.verifier, { findings: 'flaky' }).state;
- // Simulate a queue edited by hand into the final stage: the gate still refuses.
- const doctored = structuredClone(failed);
- const task = doctored.tasks[g.id];
- task.phase = 'evidence';
- task.assignedAgent = ownerFor('evidence', { roster: doctored.roster });
- task.lock = { owner: task.assignedAgent, claimedAt: new Date().toISOString() };
- task.status = 'in_progress';
- assert.throws(() => completeStage(doctored, g.id, task.assignedAgent), /verification has not passed/);
- // And it is not dispatchable either.
- const unclaimed = structuredClone(doctored);
- unclaimed.tasks[g.id].lock = null;
- unclaimed.tasks[g.id].status = 'queued';
- assert.throws(() => nextDispatch(unclaimed, g.id), /is gated/);
- });
- test('reworking and passing the verify stage opens the gate', () => {
- const g = atVerify();
- let s = failVerification(g.state, g.id, g.verifier, { findings: 'flaky' }).state;
- const blocked = s.tasks[g.id];
- s = claimTask(s, g.id, blocked.assignedAgent).state;
- s = completeStage(s, g.id, blocked.assignedAgent, { evidence: 'fixed the flake' }).state;
- const verifier = s.tasks[g.id].assignedAgent;
- s = claimTask(s, g.id, verifier).state;
- s = completeStage(s, g.id, verifier, { evidence: 'reproduced, exit 0' }).state;
- assert.equal(s.tasks[g.id].verification.status, 'passed');
- assert.equal(s.tasks[g.id].verification.blocked, false);
- assert.equal(s.tasks[g.id].verification.attempts, 2);
- s = claimTask(s, g.id, s.tasks[g.id].assignedAgent).state;
- s = completeStage(s, g.id, s.tasks[g.id].lock.owner).state;
- assert.equal(s.tasks[g.id].phase, 'done');
- });
- test('an override needs an approver, a scope and a reason, and is recorded', () => {
- const g = atVerify();
- const failed = failVerification(g.state, g.id, g.verifier, { findings: 'known flake' }).state;
- assert.throws(() => overrideVerificationGate(failed, g.id, { approvedBy: 'user' }), /requires approvedBy, scope and reason/);
- assert.throws(() => overrideVerificationGate(failed, g.id, { approvedBy: 'user', scope: 'ship it' }), /requires approvedBy, scope and reason/);
- const out = overrideVerificationGate(failed, g.id, { approvedBy: 'user', scope: 'release 2.1', reason: 'flake is pre-existing' });
- assert.equal(out.task.verification.blocked, false);
- // The failure is kept next to the override - an override never erases the record.
- assert.equal(out.task.verification.status, 'failed');
- assert.equal(out.task.verification.override.approvedBy, 'user');
- assert.equal(out.task.verification.override.scope, 'release 2.1');
- assert.equal(out.task.verification.override.reason, 'flake is pre-existing');
- assert.equal(out.state.events.at(-1).type, 'verification_gate_overridden');
- // With the override recorded, the final stage is allowed.
- const doctorable = structuredClone(out.state);
- const task = doctorable.tasks[g.id];
- task.phase = 'evidence';
- task.lock = { owner: task.assignedAgent, claimedAt: new Date().toISOString() };
- task.status = 'in_progress';
- const done = completeStage(doctorable, g.id, task.assignedAgent);
- assert.equal(done.task.phase, 'done');
- });
- test('a clean run passes the gate with no override', () => {
- const created = createOne({ title: 'clean' });
- const id = created.task.id;
- let s = FLOWS.build.actors.reduce((state, actor) => step(state, id, actor), created.state);
- const task = s.tasks[id];
- assert.equal(task.phase, 'done');
- assert.equal(task.verification.status, 'passed');
- assert.equal(task.verification.blocked, false);
- assert.equal(task.verification.override, null);
- assert.equal(task.verification.attempts, 1);
- });
- test('an L3 task refuses to run when no domain-qualified third party exists', () => {
- // Independence is not competence: with nobody scored for the domain role, the domain review must
- // block instead of letting any third-party verifier stand in for it.
- const roster = {
- agents: {
- impl: { cost: 1, scores: { specify: 3, triage: 1, implement: 3, verify: 1, evidence: 1, environment: 1 } },
- ver: { cost: 2, scores: { specify: 1, triage: 1, implement: 1, verify: 3, evidence: 2, environment: 1 } },
- spare: { cost: 3, scores: { specify: 1, triage: 1, implement: 1, verify: 2, evidence: 2, environment: 1 } },
- },
- routes: { specify: 'impl', implement: 'impl', verify: 'ver', evidence: 'spare' },
- };
- const state = createState(roster);
- const created = createTask(state, { title: 'consequential', rigor: 'L3',
- executionMode: 'collaborative', security: 'planned', independentVerify: 'planned' });
- const e1 = { grade: 'E1', text: 'ran it', command: 'npm test', exitCode: 0, revision: 'abc' };
- assert.throws(() => driveToDone(created.state, created.task.id, e1), /domain/);
- });
- test('a route named after a prototype member is ignored, not resolved', () => {
- // `agents['constructor']` resolves through Object.prototype, so a route named after a prototype
- // member used to hand the task to an agent that does not exist.
- const roster = {
- agents: { real: { cost: 1, scores: { specify: 3, implement: 2, verify: 1, evidence: 1, environment: 1 } } },
- routes: { specify: 'constructor', implement: 'toString', verify: '__proto__' },
- };
- const state = createState(roster);
- const created = createTask(state, { title: 'prototype route' });
- const owner = created.task.assignedAgent;
- assert.equal(owner, 'real', 'a prototype member must not become an agent');
- });
- // ── the defects an independent review reproduced (2026-10-02) ────────────────
- const claimBoth = (a, b, workspaceB) => {
- // Chained, not parallel: both tasks have to live in ONE state for a lock to mean anything.
- const one = createTask(createState(), { title: 'one', resources: [a] });
- const two = createTask(one.state, { title: 'two', resources: [b], ...(workspaceB ? { workspace: workspaceB } : {}) });
- const held = claimTask(two.state, one.task.id, one.task.assignedAgent).state;
- return { held, two };
- };
- test('resource identity: ./src/a.ts and src/a.ts are the same file', () => {
- // Raw string comparison let both hold the same file at once.
- const { held, two } = claimBoth('src/a.ts', './src/a.ts');
- assert.throws(() => claimTask(held, two.task.id, two.task.assignedAgent), /already held by/);
- });
- test('resource identity: a backslash spelling is the same file', () => {
- const { held, two } = claimBoth('src/a.ts', 'src\\a.ts');
- assert.throws(() => claimTask(held, two.task.id, two.task.assignedAgent), /already held by/);
- });
- test('resource identity: the same relative path in two workspaces is two files', () => {
- // The opposite error: two projects naming 'src/a.ts' were treated as competing.
- const { held, two } = claimBoth('src/a.ts', 'src/a.ts', 'wsB');
- assert.doesNotThrow(() => claimTask(held, two.task.id, two.task.assignedAgent));
- });
- test('resource identity: a directory covers the files beneath it', () => {
- const { held, two } = claimBoth('dir:src', 'src/deep/a.ts');
- assert.throws(() => claimTask(held, two.task.id, two.task.assignedAgent), /already held by/);
- });
- test('routing: an implicit route needs a positive score, so an all-zero roster is refused', () => {
- const roster = {
- agents: { solo: { cost: 1, scores: { specify: 0, implement: 0, verify: 0, evidence: 0, triage: 0, environment: 0, domain: 0 } } },
- routes: {},
- };
- assert.equal(ownerFor('specify', { roster }), null);
- assert.throws(() => createTask(createState(roster), { title: 'nobody' }), /No agent in the roster can own/);
- });
- test('routing: an explicit route is a declaration and is honoured at score 0', () => {
- // The boundary, stated rather than implied: naming an agent for a role is the roster author
- // vouching for it, and the engine does not second-guess the declaration.
- const roster = {
- agents: { solo: { cost: 1, scores: { specify: 0, implement: 0, verify: 0, evidence: 0, triage: 0, environment: 0, domain: 0 } } },
- routes: { specify: 'solo' },
- };
- assert.equal(ownerFor('specify', { roster }), 'solo');
- });
- test('an unapproved external action cannot be claimed at its external step', () => {
- const created = createOne({ title: 'deploy', externalAction: 'deploy', externalTarget: 'production' });
- const id = created.task.id;
- let state = step(created.state, id, created.task.assignedAgent);
- const implementer = state.tasks[id].assignedAgent;
- assert.equal(implementer, implementerForType('build', state.roster));
- // No dispatch happened, so nothing approved the step.
- assert.throws(() => claimTask(state, id, implementer), /not approved/);
- });
- test('an E1 exit code must be an integer, because NaN serialises to null', () => {
- const created = createOne({ title: 'nan' });
- const state = step(created.state, created.task.id, created.task.assignedAgent);
- const agent = state.tasks[created.task.id].assignedAgent;
- const claimed = claimTask(state, created.task.id, agent).state;
- assert.throws(
- () => completeStage(claimed, created.task.id, agent, {
- evidence: { grade: 'E1', text: 'ran it', command: 'npm test', exitCode: Number('abc'), revision: 'r1' },
- }),
- /integer exit code/,
- );
- });
- test('evidence provenance is the engine record, not the callers', () => {
- const record = normalizeEvidence('agent-a', { grade: 'E3', text: 'x', agent: 'attacker', at: '1999-01-01' });
- assert.equal(record.agent, 'agent-a');
- assert.notEqual(record.at, '1999-01-01');
- });
- test('the dispatch record carries the packet a worker needs, not only an argv array', () => {
- const created = createOne({ title: 'packet', resources: ['src/x.ts'] });
- const id = created.task.id;
- const walked = step(created.state, id, created.task.assignedAgent);
- const dispatch = nextDispatch(walked, id);
- assert.ok(Array.isArray(dispatch.argv));
- assert.equal(dispatch.packet.version, 1);
- assert.equal(dispatch.packet.spec, 'the change under test');
- assert.equal(dispatch.packet.acceptance, 'the suite exits 0');
- assert.deepEqual(dispatch.packet.resources, ['src/x.ts']);
- assert.ok(Array.isArray(dispatch.packet.evidenceFloor));
- });
- // ── the second review round (2026-10-02): handoff vocabulary and override ────
- test('handoff: an English negated precedence claim is refused', async () => {
- const dir = mkdtempSync(join(tmpdir(), 'ao-hop-'));
- for (const f of ['HANDOFF-RULES.md', 'MEMORY-SNAPSHOT.md', 'AGENT-ROLES.md']) {
- writeFileSync(join(dir, f), 'placeholder\n');
- }
- writeFileSync(join(dir, 'CURRENT-TASK.md'), [
- 'Status: in-progress', 'Owner: a', 'Current step: x', 'Next step: y', 'Evidence: z',
- '', 'The record never outranks the queue.', '',
- ].join('\n'));
- const { checkHandoff } = await import('./../check-handoff.mjs');
- const result = checkHandoff(dir);
- assert.equal(result.ok, false, 'a negation must not satisfy the requirement it negates');
- assert.ok(result.problems.some((p) => /outranks the queue/.test(p)));
- });
- test('handoff: the template in references/handoff.md passes the checker', async () => {
- // The documentation and the checker disagreeing was a trap: following the reference record
- // produced a rejected record.
- const doc = readFileSync(new URL('../../references/handoff.md', import.meta.url), 'utf8');
- const block = [...doc.matchAll(/```[a-z]*\n([\s\S]*?)```/g)].map((m) => m[1]).find((b) => /^status:/m.test(b));
- assert.ok(block, 'the reference record template was not found in references/handoff.md');
- const dir = mkdtempSync(join(tmpdir(), 'ao-hop-'));
- for (const f of ['HANDOFF-RULES.md', 'MEMORY-SNAPSHOT.md', 'AGENT-ROLES.md']) {
- writeFileSync(join(dir, f), 'placeholder\n');
- }
- // The block is a SCHEMA: its values are alternatives ('in-progress | handoff-wait | ...'), so it is
- // the FIELD VOCABULARY that has to be accepted, not the block verbatim.
- const keys = block.split('\n').map((l) => (l.match(/^([a-z]+):/) || [])[1]).filter(Boolean);
- for (const key of ['status', 'owner', 'current', 'next', 'evidence']) {
- assert.ok(keys.includes(key), 'the documented field names no longer include: ' + key);
- }
- writeFileSync(join(dir, 'CURRENT-TASK.md'), [
- 'status: in-progress',
- 'owner: someone',
- 'current: the step in flight',
- 'next: the next owner does X',
- 'evidence: criterion -> command, exit code',
- 'The record outranks the queue.',
- '',
- ].join('\n'));
- const { checkHandoff } = await import('./../check-handoff.mjs');
- const result = checkHandoff(dir);
- assert.deepEqual(result.problems, [], 'the documented field vocabulary must satisfy the documented checker');
- });
- test('override: accepting a failure lets the task proceed instead of demanding a second PASS', () => {
- const created = createOne({ title: 'override' });
- const id = created.task.id;
- let state = step(created.state, id, created.task.assignedAgent);
- const implementer = state.tasks[id].assignedAgent;
- state = step(state, id, implementer);
- const verifier = state.tasks[id].assignedAgent;
- state = claimTask(state, id, verifier).state;
- state = failVerification(state, id, verifier, { findings: 'the change does not hold' }).state;
- assert.equal(state.tasks[id].verification.blocked, true);
- const overridden = overrideVerificationGate(state, id, {
- approvedBy: 'coordinator', scope: 'ship it', reason: 'accepted risk, tracked separately',
- }).state;
- const task = overridden.tasks[id];
- assert.equal(task.verification.blocked, false);
- assert.equal(task.phase, 'evidence', 'the override is the decision to proceed');
- assert.equal(task.lock, null);
- });
- // ── landing an earlier queue on the current rules (2026-10-02) ───────────────
- test('migrate: an earlier queue keeps every task and gains the current fields', () => {
- const legacy = JSON.parse(readFileSync(new URL('../../examples/legacy-queue-v3.json', import.meta.url), 'utf8'));
- const { state, report } = migrateState(legacy, { source: 'examples/legacy-queue-v3.json' });
- assert.equal(state.version, STATE_VERSION);
- assert.deepEqual(Object.keys(state.tasks).sort(), ['task-0001', 'task-0002', 'task-0003']);
- assert.equal(report.tasks, 3);
- assert.ok(state.roster && state.policy, 'a migrated queue must carry a roster and a policy');
- assert.equal(state.nextTaskNumber, 4, 'the next id must not collide with the migrated ones');
- for (const task of Object.values(state.tasks)) {
- assert.ok(RIGOR_LEVELS.includes(task.rigor));
- assert.ok(task.verification, 'every task needs a verification record');
- assert.ok(Array.isArray(task.evidence));
- assert.ok(task.legacy, 'a migrated task must say it was migrated');
- }
- });
- test('migrate: a historical outcome is recorded as unrecorded, never as passed', () => {
- // Claiming a task finished in August was verified under rules that did not exist then would be the
- // false qualification this project exists to prevent.
- const legacy = JSON.parse(readFileSync(new URL('../../examples/legacy-queue-v3.json', import.meta.url), 'utf8'));
- const { state } = migrateState(legacy);
- assert.equal(state.tasks['task-0001'].status, 'done');
- assert.equal(state.tasks['task-0001'].verification.status, 'unrecorded');
- assert.ok(state.tasks['task-0001'].verification.note);
- assert.equal(state.tasks['task-0002'].status, 'blocked');
- assert.equal(state.tasks['task-0002'].verification.blocked, true);
- // The last real stage is recovered from the task's own event log rather than invented.
- assert.equal(state.tasks['task-0002'].phase, 'verify');
- assert.equal(state.tasks['task-0003'].verification.status, 'pending');
- });
- test('migrate: a migrated queued task is workable under the current rules', () => {
- const legacy = JSON.parse(readFileSync(new URL('../../examples/legacy-queue-v3.json', import.meta.url), 'utf8'));
- const { state } = migrateState(legacy);
- const dispatch = nextDispatch(state, 'task-0003');
- assert.equal(dispatch.execute, false);
- assert.equal(dispatch.agent, 'agent-c');
- assert.ok(dispatch.packet && dispatch.packet.version === 1);
- const claimed = claimTask(state, 'task-0003', 'agent-c').state;
- assert.equal(claimed.tasks['task-0003'].status, 'in_progress');
- });
- test('migrate: a queue that is already current is refused, not silently rewritten', () => {
- assert.throws(() => migrateState(createState()), /already version/);
- assert.throws(() => migrateState({}), /expected a state object/);
- });
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.