Skip to content
Back to search

harbor

Framework for evaluating and improving agents

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/harbor-framework/harbor with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/harbor-framework/harbor'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/harbor-framework-harbor/badge.svg)](/listings/harbor-framework-harbor)

HTML

<a href="/listings/harbor-framework-harbor"><img src="/api/listings/harbor-framework-harbor/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

uv tool install harbor

or

pip install harbor

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 85 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # Harbor
  2. [![](https://dcbadge.limes.pink/api/server/https://discord.gg/QVvyhRw5UQ)](https://discord.gg/QVvyhRw5UQ)
  3. [![Docs](https://img.shields.io/badge/Docs-000000?style=for-the-badge&logo=mdbook&color=105864)](https://harborframework.com/docs)
  4. [![Cookbook](https://img.shields.io/badge/Cookbook-000000?style=for-the-badge&logo=mdbook&color=105864)](https://github.com/harbor-framework/harbor-cookbook)
  5. [![DOI](https://zenodo.org/badge/1032170083.svg)](https://doi.org/10.5281/zenodo.20953922)
  6. Harbor is a framework from the creators of [Terminal-Bench](https://www.tbench.ai) for evaluating and optimizing agents and language models. You can use Harbor to:
  7. - Evaluate arbitrary agents like Claude Code, OpenHands, Codex CLI, and more.
  8. - Build and share your own benchmarks and environments.
  9. - Conduct experiments in thousands of environments in parallel through providers like Daytona, Modal, LangSmith, Blaxel, Novita Sandbox, Tensorlake, Runta, and Sail.
  10. - Generate rollouts for RL optimization.
  11. Check out the [Harbor Cookbook](https://github.com/harbor-framework/harbor-cookbook) for end-to-end examples and guides.
  12. ## Installation
  13. ```bash tab="uv"
  14. uv tool install harbor
  15. ```
  16. or
  17. ```bash tab="pip"
  18. pip install harbor
  19. ```
  20. ## Example: Running Terminal-Bench-2.0
  21. Harbor is the official harness for [Terminal-Bench-2.0](https://github.com/laude-institute/terminal-bench-2):
  22. ```bash
  23. export ANTHROPIC_API_KEY=<YOUR-KEY>
  24. harbor run --dataset terminal-bench@2.0 \
  25. --agent claude-code \
  26. --model anthropic/claude-opus-5-5 \
  27. --n-concurrent 4
  28. ```
  29. This will launch the benchmark locally using Docker. To run it on a cloud provider (like Daytona) pass the `--env` flag as below:
  30. ```bash
  31. export ANTHROPIC_API_KEY=<YOUR-KEY>
  32. export DAYTONA_API_KEY=<YOUR-KEY>
  33. harbor run --dataset terminal-bench@2.0 \
  34. --agent claude-code \
  35. --model anthropic/claude-opus-5-5 \
  36. --n-concurrent 100 \
  37. --env daytona
  38. ```
  39. To see all supported agents, and other options run:
  40. ```bash
  41. harbor run --help
  42. ```
  43. To explore all supported third party benchmarks (like SWE-Bench and Aider Polyglot) run:
  44. ```bash
  45. harbor datasets list
  46. ```
  47. To evaluate an agent and model one of these datasets, you can use the following command:
  48. ```bash
  49. harbor run -d "<dataset@version>" -m "<model>" -a "<agent>"
  50. ```
  51. ## Citation
  52. If you use **Harbor** in academic work, please cite it using the “Cite this repository” button on GitHub or the following BibTeX entry:
  53. ```bibtex
  54. @software{Harbor_Framework,
  55. author = {{Harbor Framework Team}},
  56. title = {{Harbor: A framework for evaluating and optimizing agents and models in container environments}},
  57. year = {2026},
  58. doi = {10.5281/zenodo.20953922},
  59. url = {https://doi.org/10.5281/zenodo.20953922}
  60. }
  61. ```
  62. The DOI above is the **concept DOI**, which always resolves to the latest release and aggregates citations across all versions. To cite a specific version instead, use that version's DOI from the [Zenodo record](https://doi.org/10.5281/zenodo.20953922).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.