Skip to content
Back to search

gap

MarketplaceWorks with Claude CodeGitHub

gap — graph as policy: compile language instructions into typed, verified robot skill graphs and execute them on simulators or real robots

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/graph-robots--graph-as-policy/badge.svg)](/listings/graph-robots--graph-as-policy)

HTML

<a href="/listings/graph-robots--graph-as-policy"><img src="/api/listings/graph-robots--graph-as-policy/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

**Requirements:** **1× NVIDIA RTX 4090 (≥24 GB VRAM, Linux + EGL)**, an LLM API key (OpenRouter / Vertex), and [uv](https://docs.astral.sh/uv/). First run downloads ~3.5 GB of model weights; the gated SAM3 weights are part of the default perception path, so in practice you also need `HF_TOKEN` (a free [HuggingFace token](https://huggingface.co/settings/tokens) with SAM3 access) before the first run.

git clone --recurse-submodules https://github.com/graph-robots/graph-as-policy.git
#   ^ submodules are required (vendored sim stack) — if you already cloned
#     plain, run: git submodule update --init --recursive
git clone https://github.com/graph-robots/open-robot-skills.git    # sibling, auto-discovered
cd graph-as-policy
uv sync                                  # engine + LIBERO sim (now baseline)
# `uv sync --extra vertex` instead if you'll use --provider vertex for codegen
uv run gap skills install --all          # per-bundle venvs (sam3, cuRobo, vlm, …)
#   --all includes the heavyweight learned-policy bundles; for one example,
#   `gap skills install --workflow <graph-dir>` installs just what it uses

export HF_TOKEN=...                      # for the gated SAM3 weights
uv run gap skills check --download       # weight prefetch (SAM3 + GDINO) + capability gate

# Pick one LLM provider for codegen + the in-graph VLM. openrouter is the
# default; for vertex, see `docs/source/authoring/llm-providers.md`.
export OPENROUTER_API_KEY=...
MUJOCO_GL=egl uv run gap run examples/libero_quickstart/graph \
  --sim libero_object_all_variance/0
uv run gap viz                           # browse the trial at localhost:9432

See the **[15-minute tour](https://graph-robots.github.io/graph-as-policy/getting-started/quickstart.html)** for the full walkthrough (clone → run → generate → trace).

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 29 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "gap",
  3. "owner": { "name": "graph-as-policy authors", "url": "https://github.com/graph-robots" },
  4. "metadata": {
  5. "description": "Agent skills for gap (graph as policy): drive the robot-graph engine from Claude Code — search skill registries, check capabilities, and run, author, and test verified robot task graphs.",
  6. "version": "0.1.0"
  7. },
  8. "plugins": [
  9. {
  10. "name": "gap",
  11. "source": "./agent",
  12. "description": "Operate the gap robot-graph engine: search robot skill registries (open-robot-skills and your own), check tool/model/skill capabilities (gap check), run graphs in LIBERO sim, generate graphs from language, author skill bundles and registries with scaffolded unit tests, debug traces, and execute safety-gated real-robot tasks.",
  13. "author": { "name": "graph-as-policy authors" },
  14. "homepage": "https://github.com/graph-robots/graph-as-policy",
  15. "repository": "https://github.com/graph-robots/graph-as-policy",
  16. "license": "MIT",
  17. "category": "robotics",
  18. "keywords": ["robotics", "robot-manipulation", "robot-skills", "skill-registry", "LIBERO", "simulation", "task-graphs"]
  19. },
  20. {
  21. "name": "open-robot-skills",
  22. "source": { "source": "github", "repo": "graph-robots/open-robot-skills" },
  23. "description": "Optional: the robot skill/tool bundle contracts from the canonical open-robot-skills registry, loadable as agent skills. Same content as that repo's own marketplace — install from one route, not both.",
  24. "skills": ["./tools"],
  25. "license": "MIT",
  26. "category": "robotics"
  27. }
  28. ]
  29. }
README.md · README · 200 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. # Graph-as-Policy
  3. <h3 align="center">
  4. <strong>🧪 BETA CODE RELEASE</strong> | <em>Beta</em> | July 1, 2026
  5. </h3>
  6. **CoRL 2026** · [Paper (arXiv)](https://arxiv.org/abs/2607.05369) · [Project page](https://graph-robots.github.io/gap/)
  7. 🆕 **Nov 2026 — the self-learning pipeline is open source:**
  8. [**agentic-self-learning**](https://github.com/graph-robots/agentic-self-learning)
  9. · [Paper (arXiv)](https://arxiv.org/abs/2611.09030)
  10. · [Project page](https://graph-robots.github.io/agentic-self-learning/)
  11. **The graph is the policy.**
  12. GaP targets *Variational Automation* — tasks a robot must perform persistently
  13. and reliably across many varying instances (objects vary in geometry and pose),
  14. not just solve once. It compiles a natural-language task into a typed, verified
  15. computation graph of modular skills, self-improves it in simulation, and runs
  16. the graph — not a black-box policy — on simulators and real robots.
  17. [![License: Apache 2.0](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE)
  18. [![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](pyproject.toml)
  19. [![Skills: open-robot-skills](https://img.shields.io/badge/skills-open--robot--skills-orange.svg)](https://github.com/graph-robots/open-robot-skills)
  20. [![Docs](https://img.shields.io/badge/docs-quickstart-0f766e.svg)](https://graph-robots.github.io/graph-as-policy/getting-started/quickstart.html)
  21. ![GaP on real robots — grocery packing, popcorn, tool packing, USB-C insertion](docs/assets/real_robots_strip.gif)
  22. <sub>GaP running on real robots — graphs generated from natural-language
  23. tasks, refined in sim, executed on hardware. Left→right: grocery packing
  24. (Franka, 10×), popcorn (long-horizon stove manipulation, 16×), tool & charger
  25. packing into tagged bins, and USB-C insertion (UR5 with force feedback).</sub>
  26. </div>
  27. ```python
  28. import gap
  29. conn = gap.connector.sim("libero", task="libero_object/0")
  30. result = gap.execute("examples/libero_quickstart/graph", conn)
  31. graph = gap.agent.generate_sync("pick the soup can and put it in the basket")
  32. print(graph) # the policy, as a graph
  33. gap.viz.serve("outputs") # browse the recorded trial
  34. ```
  35. Skills live in the sibling
  36. [**open-robot-skills**](https://github.com/graph-robots/open-robot-skills)
  37. repo (Anthropic Agent Skills format, contributable) and are discovered
  38. by path — clone the two repos side by side and every command finds them.
  39. ## Quickstart
  40. **Requirements:** **1× NVIDIA RTX 4090 (≥24 GB VRAM, Linux + EGL)**, an
  41. LLM API key (OpenRouter / Vertex), and
  42. [uv](https://docs.astral.sh/uv/). First run downloads ~3.5 GB of model
  43. weights; the gated SAM3 weights are part of the default perception path,
  44. so in practice you also need `HF_TOKEN` (a free
  45. [HuggingFace token](https://huggingface.co/settings/tokens) with SAM3
  46. access) before the first run.
  47. ```bash
  48. git clone --recurse-submodules https://github.com/graph-robots/graph-as-policy.git
  49. # ^ submodules are required (vendored sim stack) — if you already cloned
  50. # plain, run: git submodule update --init --recursive
  51. git clone https://github.com/graph-robots/open-robot-skills.git # sibling, auto-discovered
  52. cd graph-as-policy
  53. uv sync # engine + LIBERO sim (now baseline)
  54. # `uv sync --extra vertex` instead if you'll use --provider vertex for codegen
  55. uv run gap skills install --all # per-bundle venvs (sam3, cuRobo, vlm, …)
  56. # --all includes the heavyweight learned-policy bundles; for one example,
  57. # `gap skills install --workflow <graph-dir>` installs just what it uses
  58. export HF_TOKEN=... # for the gated SAM3 weights
  59. uv run gap skills check --download # weight prefetch (SAM3 + GDINO) + capability gate
  60. # Pick one LLM provider for codegen + the in-graph VLM. openrouter is the
  61. # default; for vertex, see `docs/source/authoring/llm-providers.md`.
  62. export OPENROUTER_API_KEY=...
  63. MUJOCO_GL=egl uv run gap run examples/libero_quickstart/graph \
  64. --sim libero_object_all_variance/0
  65. uv run gap viz # browse the trial at localhost:9432
  66. ```
  67. See the **[15-minute tour](https://graph-robots.github.io/graph-as-policy/getting-started/quickstart.html)**
  68. for the full walkthrough (clone → run → generate → trace).
  69. ## Examples
  70. | Example | What it shows |
  71. |---|---|
  72. | [libero_quickstart](examples/libero_quickstart/) | Hero: vision → OBB grasp → transport, ground-truth verified |
  73. | [grocery_packing](examples/grocery_packing/) | Pack every item with a loop: a graph with a real backward edge |
  74. | [generate_a_graph](examples/generate_a_graph/) | Instruction → validated workflow dir; all LLM providers |
  75. | [build_a_graph](examples/build_a_graph/) | Full Python authoring with `gap.builder`: checkpoints, recovery |
  76. | [agent_quickstart](examples/agent_quickstart/) | Claude Code + one skill: sentence → graph → sim success |
  77. | [grocery_fulfillment](examples/grocery_fulfillment/) | Flagship acceptance family; graphs LLM-generated per task |
  78. | [benchmark](examples/benchmark/) | Grid harness: smoke → posvar → release gate |
  79. | [steered_policy](examples/steered_policy/) | Perceive + hover, then hand to a learned VLA policy |
  80. | [collect_and_train](examples/collect_and_train/) | Graph as scripted expert → dataset → train → policy node |
  81. | [cable_ur](examples/cable_ur/) | Real UR + ZED perception (motion-disabled) |
  82. | [real_franka_pick_place](examples/real_franka_pick_place/) | Real Franka pick-place via robots_realtime |
  83. Full gallery with install needs and time estimates in
  84. **[examples/README.md](examples/README.md)**. Real-robot examples
  85. — read [docs/safety.md](docs/safety.md) first.
  86. ## Self-learning pipeline
  87. The loop that self-improves a graph in simulation is released as
  88. **[agentic-self-learning](https://github.com/graph-robots/agentic-self-learning)**.
  89. A coding agent (Claude Code or Codex) writes a GaP `workflow.json` policy,
  90. runs it on [RoboSimStudio](https://github.com/robosimstudio/robosimstudio)
  91. tasks, reads structured simulation feedback, and rewrites the graph. Every
  92. feedback condition from the study (state, relations, or visual feedback; how
  93. space is described; per-episode, per-motion, or per-node timing) is a config
  94. setting, and skills from
  95. [open-robot-skills](https://github.com/graph-robots/open-robot-skills) plug in
  96. as the agent's skill library.
  97. It accompanies *A Controlled Study of Models, Skills, Feedback, and Curriculum
  98. in Agentic Robotics* ([arXiv](https://arxiv.org/abs/2611.09030) ·
  99. [project page](https://graph-robots.github.io/agentic-self-learning/)).
  100. ## Documentation
  101. Full docs site: **<https://graph-robots.github.io/graph-as-policy/>**
  102. - [Quickstart (15-min tour)](https://graph-robots.github.io/graph-as-policy/getting-started/quickstart.html) — clone → run → generate → trace
  103. - [Runtime & schema](https://graph-robots.github.io/graph-as-policy/reference/workflow-schema.html) — workflow JSON, executor semantics, checkpoints
  104. - [Skill authoring](https://graph-robots.github.io/graph-as-policy/skills/authoring-bundles.html) — Agent Skills format, `gap.requires:` frontmatter
  105. - [LLM providers](https://graph-robots.github.io/graph-as-policy/authoring/llm-providers.html) — openrouter / vertex
  106. - [Skill registries](https://graph-robots.github.io/graph-as-policy/skills/registries.html) — `--skills`, `$GAP_SKILLS_PATH`, `gap registry …`
  107. - [CLI reference](https://graph-robots.github.io/graph-as-policy/reference/cli.html) — every `gap` verb
  108. - [Safety](https://graph-robots.github.io/graph-as-policy/real-robots/safety.html) — required reading before any real-robot example
  109. ## Use with Claude Code & AI agents
  110. GaP ships an agent skill ([`agent/`](agent/)) that teaches AI coding
  111. agents the workflows above:
  112. ```bash
  113. claude plugin marketplace add graph-robots/graph-as-policy
  114. claude plugin install gap@gap
  115. ```
  116. Then ask: *"what can this robot do right now?"*, *"run the quickstart graph
  117. in sim"*, *"generate a graph that packs the groceries"*, *"why did this
  118. trial fail?"*. Real-robot commands are gated on explicit human
  119. confirmation. Other agents (Cursor, Codex, …) and the no-install path:
  120. [agent/INSTALL.md](agent/INSTALL.md).
  121. ## Citation
  122. If you use GaP in your research, please cite our CoRL 2026 paper:
  123. ```bibtex
  124. @inproceedings{chen2026gap,
  125. title = {GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness
  126. for Variational Automation (VA) Tasks},
  127. author = {Chen, Kaiyuan and Xie, Shuangyu and Fu, Letian and Yu, Justin and
  128. Pacini, William and Bajamahal, Sandeep and Kim, Hudson and
  129. Drake, Jaimyn and Kim, Daehwa and Xue, Haoru and Francis, Jonathan and
  130. Juette, Christian and Schaldenbrand, Peter and Seker, Muhammet Yunus and
  131. Wickramarachchi, Ruwan and Yoo, Uksang and Wang, Guanzhi and
  132. Murali, Adithyavairavan and Sundaralingam, Balakumar and
  133. Sastry, S. Shankar and Huang, Spencer and Zhu, Yuke and
  134. Fan, Linxi and Goldberg, Ken},
  135. booktitle = {Conference on Robot Learning (CoRL)},
  136. year = {2026},
  137. url = {https://arxiv.org/abs/2607.05369}
  138. }
  139. ```
  140. If you use the self-learning pipeline
  141. ([agentic-self-learning](https://github.com/graph-robots/agentic-self-learning)),
  142. please also cite:
  143. ```bibtex
  144. @misc{chen2026controlled,
  145. title = {A Controlled Study of Models, Skills, Feedback, and Curriculum
  146. in Agentic Robotics},
  147. author = {Chen, Kaiyuan and Zhu, Xinyue and Xie, Shuangyu and Yoo, Uksang and
  148. Luong, Thi Thu Phuong and Gualtieri, Marcus and Juette, Christian and
  149. Ren, Liu and Huang, Spencer and Goldberg, Ken},
  150. year = {2026},
  151. eprint = {2611.09030},
  152. archivePrefix = {arXiv},
  153. primaryClass = {cs.RO},
  154. url = {https://arxiv.org/abs/2611.09030}
  155. }
  156. ```
  157. ## License & attribution
  158. graph-as-policy and open-robot-skills are Apache-2.0-licensed; they stand on third-party
  159. work — LIBERO/LIBERO-PRO, Variational-Automation-Benchmark, robosuite,
  160. robots_realtime, pyroki, SAM3, Grounding DINO, and (optionally) NVIDIA
  161. cuRobo. Full attribution table: **[NOTICE.md](NOTICE.md)**.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.