harbor
Framework for evaluating and improving agents
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
It has changed since its scan on 11 Nov 2026.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/harbor-framework/harbor with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/harbor-framework/harbor'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/harbor-framework-harbor)HTML
<a href="/listings/harbor-framework-harbor"><img src="/api/listings/harbor-framework-harbor/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
uv tool install harbor
or
pip install harbor
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 85 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- # Harbor
- [](https://discord.gg/QVvyhRw5UQ)
- [](https://harborframework.com/docs)
- [](https://github.com/harbor-framework/harbor-cookbook)
- [](https://doi.org/10.5281/zenodo.20953922)
- Harbor is a framework from the creators of [Terminal-Bench](https://www.tbench.ai) for evaluating and optimizing agents and language models. You can use Harbor to:
- - Evaluate arbitrary agents like Claude Code, OpenHands, Codex CLI, and more.
- - Build and share your own benchmarks and environments.
- - Conduct experiments in thousands of environments in parallel through providers like Daytona, Modal, LangSmith, Blaxel, Novita Sandbox, Tensorlake, Runta, and Sail.
- - Generate rollouts for RL optimization.
- Check out the [Harbor Cookbook](https://github.com/harbor-framework/harbor-cookbook) for end-to-end examples and guides.
- ## Installation
- ```bash tab="uv"
- uv tool install harbor
- ```
- or
- ```bash tab="pip"
- pip install harbor
- ```
- ## Example: Running Terminal-Bench-2.0
- Harbor is the official harness for [Terminal-Bench-2.0](https://github.com/laude-institute/terminal-bench-2):
- ```bash
- export ANTHROPIC_API_KEY=<YOUR-KEY>
- harbor run --dataset terminal-bench@2.0 \
- --agent claude-code \
- --model anthropic/claude-opus-5-5 \
- --n-concurrent 4
- ```
- This will launch the benchmark locally using Docker. To run it on a cloud provider (like Daytona) pass the `--env` flag as below:
- ```bash
- export ANTHROPIC_API_KEY=<YOUR-KEY>
- export DAYTONA_API_KEY=<YOUR-KEY>
- harbor run --dataset terminal-bench@2.0 \
- --agent claude-code \
- --model anthropic/claude-opus-5-5 \
- --n-concurrent 100 \
- --env daytona
- ```
- To see all supported agents, and other options run:
- ```bash
- harbor run --help
- ```
- To explore all supported third party benchmarks (like SWE-Bench and Aider Polyglot) run:
- ```bash
- harbor datasets list
- ```
- To evaluate an agent and model one of these datasets, you can use the following command:
- ```bash
- harbor run -d "<dataset@version>" -m "<model>" -a "<agent>"
- ```
- ## Citation
- If you use **Harbor** in academic work, please cite it using the “Cite this repository” button on GitHub or the following BibTeX entry:
- ```bibtex
- @software{Harbor_Framework,
- author = {{Harbor Framework Team}},
- title = {{Harbor: A framework for evaluating and optimizing agents and models in container environments}},
- year = {2026},
- doi = {10.5281/zenodo.20953922},
- url = {https://doi.org/10.5281/zenodo.20953922}
- }
- ```
- The DOI above is the **concept DOI**, which always resolves to the latest release and aggregates citations across all versions. To cite a specific version instead, use that version's DOI from the [Zenodo record](https://doi.org/10.5281/zenodo.20953922).
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.