Skip to content
Back to search

sam3-segmentation

MarketplaceWorks with Claude CodeGitHub

The sam3-segmentation agent skill

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/lillyguisnet--sam3-segmentation/badge.svg)](/listings/lillyguisnet--sam3-segmentation)

HTML

<a href="/listings/lillyguisnet--sam3-segmentation"><img src="/api/listings/lillyguisnet--sam3-segmentation/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Pick one. Each installs the same skill folder.

# Any agent (Claude Code, Codex, Cursor, OpenCode, Gemini CLI, ...). Needs Node.js 22+.
npx skills add lillyguisnet/sam3-segmentation        # add -g for every project

# Claude Code plugin, with updates (Claude Code 2.1.275 or later):
/plugin install sam3-segmentation --marketplace lillyguisnet/sam3-segmentation

# No Node.js: copy the folder yourself (--target claude, codex or pi)
git clone https://github.com/lillyguisnet/sam3-segmentation
python3 sam3-segmentation/install.py --target claude

Then ask your agent in plain words. The skill loads by itself when the request fits; you can also name it (`/sam3-segmentation` in Claude Code, where a plugin install shows it as `/sam3-segmentation:sam3-segmentation`, or `$sam3-segmentation` in Codex). For example:

> Outline every visible kernel in corn.jpg, and the whole ears separately. Keep > overlapping levels, check every region, and tell me what is uncertain. Ask me > before any large download.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 15 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "sam3-segmentation",
  3. "description": "The sam3-segmentation agent skill",
  4. "owner": {
  5. "name": "Aurélie Guisnet",
  6. "url": "https://github.com/lillyguisnet"
  7. },
  8. "plugins": [
  9. {
  10. "name": "sam3-segmentation",
  11. "source": "./",
  12. "description": "Outline, mask or count objects in images with Meta's SAM 3 on a local NVIDIA GPU, directly or through an auditable review workflow."
  13. }
  14. ]
  15. }
README.md · README · 170 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # sam3-segmentation
  2. [![Test](https://github.com/lillyguisnet/sam3-segmentation/actions/workflows/test.yml/badge.svg)](https://github.com/lillyguisnet/sam3-segmentation/actions/workflows/test.yml)
  3. [![skills.sh](https://skills.sh/b/lillyguisnet/sam3-segmentation)](https://skills.sh/lillyguisnet/sam3-segmentation)
  4. > This was mostly written by Opus 5.5 and not thoroughly checked by a human. It works on my machine, but haven't tested it elsewhere.
  5. **An agent skill that outlines objects in photos with Meta's SAM 3, run on your
  6. own NVIDIA GPU, and keeps everything it produces checkable.**
  7. ![Six agents on the same corn photo: kernel outlines drawn without tools (left half of each panel) and with this skill (right half)](docs/six-models/panel.jpg)
  8. *Same six agents on the same corn photo, without tools (left half) and with this
  9. skill (right half). Counts are each run's full set; nobody has checked these
  10. outlines ([details](#does-it-work), [data](docs/six-models/record.json)). Photo:
  11. ImageNet validation image ILSVRC2012_val_00000408; copyright belongs to its
  12. original photographer and it is not covered by this repository's licence. It
  13. will be removed on request.*
  14. Ask your coding agent (Claude Code, Codex, Cursor, Pi, OpenCode and about 75
  15. others) to "outline every kernel", "cut out the dog" or "count the seeds", and
  16. this skill gives it the model, the commands and the rules to do it properly:
  17. - **Quick calls:** every instance of a phrase ("all the jars"), more objects like
  18. an example box, one object from a click or a box, or "segment everything".
  19. - **A careful workflow** for many small objects or parts (kernels, seeds, cells,
  20. insects): overlapping close-up crops, every raw candidate saved, score checks,
  21. review sheets and flags, recorded and reversible review decisions, boundary
  22. refinement, PNG and COCO RLE exports, and a zoomable viewer.
  23. - **Honest output:** masks are reported as model proposals, never as verified
  24. ground truth, with what is uncertain listed.
  25. ## Install
  26. Pick one. Each installs the same skill folder.
  27. ```sh
  28. # Any agent (Claude Code, Codex, Cursor, OpenCode, Gemini CLI, ...). Needs Node.js 22+.
  29. npx skills add lillyguisnet/sam3-segmentation # add -g for every project
  30. # Claude Code plugin, with updates (Claude Code 2.1.275 or later):
  31. /plugin install sam3-segmentation --marketplace lillyguisnet/sam3-segmentation
  32. # No Node.js: copy the folder yourself (--target claude, codex or pi)
  33. git clone https://github.com/lillyguisnet/sam3-segmentation
  34. python3 sam3-segmentation/install.py --target claude
  35. ```
  36. Then ask your agent in plain words. The skill loads by itself when the request
  37. fits; you can also name it (`/sam3-segmentation` in Claude Code, where a plugin
  38. install shows it as `/sam3-segmentation:sam3-segmentation`, or `$sam3-segmentation`
  39. in Codex). For example:
  40. > Outline every visible kernel in corn.jpg, and the whole ears separately. Keep
  41. > overlapping levels, check every region, and tell me what is uncertain. Ask me
  42. > before any large download.
  43. ## What you need
  44. | To... | You need |
  45. |---|---|
  46. | Run the model | Linux (or Windows WSL2) with an **NVIDIA GPU** (12-24 GB memory suggested), `uv`, about **11 GB** of disk, and your own accepted access to [facebook/sam3](https://huggingface.co/facebook/sam3) on Hugging Face |
  47. | See a demo of the viewer and file formats, on generated shapes (no segmentation) | Any machine with [`uv`](https://docs.astral.sh/uv/getting-started/installation/); no GPU, model or account |
  48. **Without an NVIDIA GPU, the skill cannot segment anything:** the model does not
  49. run on CPU or Apple GPUs. The agent asks before
  50. installing anything. It never asks for your Hugging Face token: you log in
  51. yourself with `hf auth login`. The Python environments are created outside the
  52. skill folder (in `~/.local/share/sam3-segmentation/`), so updates and several
  53. installed copies reuse them.
  54. ## Demo without a GPU (no segmentation)
  55. The model needs an NVIDIA GPU. Without one, you can only run a **demo**: it
  56. draws generated shapes, runs the review and export tools on them, and opens the
  57. viewer, so you can see what the results look like before installing anything
  58. large. It does not segment your photos and does not use the model.
  59. ```sh
  60. S=~/.claude/skills/sam3-segmentation # wherever the skill was installed
  61. sh "$S/scripts/run" base segment.py demo --out /tmp/sam3-demo
  62. sh "$S/scripts/run" base python -m http.server 8000 --bind 127.0.0.1 --directory /tmp/sam3-demo/export
  63. ```
  64. Open http://127.0.0.1:8000 to see the viewer on the generated shapes.
  65. ## Does it work?
  66. **On corn, with six different agents.** Six models (GPT-6 Luna, GPT-6.1 Sol,
  67. GPT-6 Astra, Claude Haiku 5.5, Sonnet 5 and Opus 5.5) each ran the full workflow
  68. alone, with only the skill and a corn photo. Each finished with a full set of
  69. kernel proposals:
  70. | Agent | Time | Kernel proposals | Removed after review | First run's kernels also found |
  71. |---|---:|---:|---:|---:|
  72. | Luna | 8 min | 1,350 | 0 | 98.6% |
  73. | Sol | 24 min | 1,315 | 45 | 97.2% |
  74. | Astra | 30 min | 1,319 | 41 | 97.6% |
  75. | Haiku | 40 min | 1,244 | 111 | 92.4% |
  76. | Sonnet | 42 min | 1,321 | 37 | 97.2% |
  77. | Opus | 40 min | 1,319 | 41 | 97.8% |
  78. "First run's kernels also found" means at least half of the outline overlaps an
  79. outline from the run the skill was built from (1,310 kernels). That run was also
  80. made with SAM 3, so this measures agreement between agents, **not accuracy**: no
  81. person has checked these outlines. Luna skipped the review; Haiku removed too
  82. much, including real kernels. The same six models with no tools, asked to trace
  83. the kernels by looking, returned between 0 and 1,084 shapes that did not follow
  84. the kernels (the picture at the top).
  85. **On a free photo you can reuse.** The same steps with no agent review, on a
  86. [CC0 corn photo](https://commons.wikimedia.org/wiki/File:Corn_ready_for_removing_kernels_from_cob.jpg):
  87. 1,397 kernel and 11 ear proposals in about 3 minutes on one RTX 3090.
  88. [Rebuild it yourself](docs/hero/make.sh).
  89. ![A CC0 photo of corn beside the skill's 1,397 kernel outlines, one colour each](docs/hero/hero.jpg)
  90. **Beyond corn.** Trials on a dog, jars, insects, mushrooms and cattle produced
  91. useful outlines, but also background mistakes and parts counted as whole
  92. objects. [Report, with failures](docs/cross-domain-trials.md).
  93. **Does the agent use it, and use it well?** Automated tests with Claude Haiku,
  94. Sonnet and Opus 5.5: Sonnet and Opus picked the skill for all 10 kinds of
  95. request it should handle and none of the 10 near-misses (customer segmentation,
  96. CT scans, video, OCR...); Haiku sometimes counted by eye instead. With the skill,
  97. agents told a Mac user the truth about GPU support, refused to use a pasted
  98. token, and stated the 11 GB download before asking to install; without it they
  99. mostly did not. [Evals and full results](evals/README.md).
  100. ## Limitations
  101. - Outputs are proposals. To claim accuracy, compare against masks a person has
  102. checked, never against these drafts.
  103. - The skill does not yet say when a review is finished, which reviewer to use,
  104. or stop an agent from removing many proposals unseen. See the
  105. [changelog](CHANGELOG.md).
  106. - Small objects need the full workflow; a quick call on a whole photo can miss
  107. them entirely.
  108. - Not for video, medical images, or classifying and editing pictures.
  109. ## Documentation
  110. - [The agent's instructions](skills/sam3-segmentation/SKILL.md)
  111. - [Quick SAM calls](skills/sam3-segmentation/references/direct-sam.md)
  112. - [Setup and troubleshooting](skills/sam3-segmentation/references/setup.md)
  113. - [Method and coordinates](skills/sam3-segmentation/references/method.md)
  114. - [Configuration, budgets and run files](skills/sam3-segmentation/references/configuration.md)
  115. - [Review protocol](skills/sam3-segmentation/references/review.md)
  116. - [Lessons from earlier mistakes](skills/sam3-segmentation/references/lessons.md)
  117. - [Testing and reproducibility](TESTING.md) · [Evals](evals/README.md) · [Changelog](CHANGELOG.md)
  118. ## Development
  119. ```sh
  120. uv sync --locked --project skills/sam3-segmentation --group dev
  121. uv run --locked --project skills/sam3-segmentation --group dev pytest tests
  122. uv run --locked --project skills/sam3-segmentation --group dev ruff check skills/sam3-segmentation/scripts tests install.py
  123. claude plugin validate --strict .
  124. ```
  125. The tests need no GPU, model, account or network inference. Keep the skill
  126. folder self-contained: it must work when copied anywhere on its own.
  127. ## Licence and privacy
  128. [MIT](LICENSE) for this repository. SAM 3's code and weights are Meta's, under
  129. their own terms, and are downloaded by you, not included here. Photo, annotation
  130. and provider rights are separate questions. This project is not affiliated with
  131. Meta. No private photos, masks, weights or credentials are in this repository;
  132. the test photos are public domain or CC0 ([credits](evals/fixtures/CREDITS.md)).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.