Skip to content
Back to search

bencher

MarketplaceWorks with Claude CodeGitHub

Bencher continuous benchmarking CLI

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/bencherdev/bencher with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/bencherdev/bencher'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/bencherdev--bencher/badge.svg)](/listings/bencherdev--bencher)

HTML

<a href="/listings/bencherdev--bencher"><img src="/api/listings/bencherdev--bencher/badge.svg" alt="findsafeskills safety rating" /></a>

Install

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 15 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "bencher",
  3. "owner": { "name": "Bencher" },
  4. "description": "Bencher continuous benchmarking CLI",
  5. "plugins": [
  6. {
  7. "name": "bencher",
  8. "source": "./plugins/bencher",
  9. "description": "Catch performance regressions before they merge",
  10. "author": { "name": "Bencher" },
  11. "homepage": "https://bencher.dev",
  12. "repository": "https://github.com/bencherdev/bencher"
  13. }
  14. ]
  15. }
README.md · README · 484 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <h1>
  2. <a href="https://bencher.dev">
  3. <img
  4. src="https://bencher.dev/favicon.svg"
  5. aria-label="🐰 Bencher"
  6. width=32
  7. />
  8. </a>
  9. Bencher
  10. </h1>
  11. [Bencher](https://bencher.dev) is a [continuous benchmarking](https://bencher.dev/docs/explanation/continuous-benchmarking/) platform that helps software engineers and agents catch performance regressions before they merge. [Bencher Bare Metal](https://bencher.dev/docs/explanation/bare-metal/) runs your benchmarks on the exact same bare metal, whether launched from your local terminal, an agent sandbox, or in CI.
  12. <p align="center">
  13. <a href="https://bencher.dev">
  14. <img
  15. src="https://s3.amazonaws.com/public.bencher.dev/github/regression-chart.svg"
  16. alt="Animated chart showing benchmark results with a performance regression spike that triggers an alert, followed by a fix back to baseline"
  17. width="800"
  18. />
  19. </a>
  20. </p>
  21. Built for teams where performance matters:
  22. - Databases
  23. - Compilers
  24. - Browsers
  25. - Runtimes
  26. - Networking stacks
  27. - Cryptographic libraries
  28. - Operating Systems
  29. Most teams start on GitHub Actions runners with a benchmark comparison script.
  30. That approach breaks down when noisy shared CI runners hide real performance regressions.
  31. - Typical CI runners: **>30% variance**
  32. - Bencher Bare Metal runners: **<2% variance**
  33. When a number moves, it means something.
  34. <p>
  35. <a href="https://bencher.dev/auth/signup"><b>Benchmark for free →</b></a>
  36. &nbsp;&nbsp;<a href="https://bencher.dev/docs/tutorial/bare-metal/">Bare Metal Quickstart</a>
  37. </p>
  38. <p>
  39. Used by the teams behind
  40. <a href="#showcase">Microsoft snmalloc</a>,
  41. <a href="#showcase">Google Sedpack</a>,
  42. <a href="#showcase">GitLab Git</a>,
  43. <a href="#showcase">pnpm</a>,
  44. <a href="#showcase">conda</a>,
  45. <a href="#showcase">Firezone</a>,
  46. and <a href="#showcase">Servo</a>.
  47. </p>
  48. > 🐰 [Use the GitHub Action with your project](#github-actions)
  49. ## The Problem
  50. Shipping a performance regression is expensive:
  51. - A database regression on a hot path pages someone at 2am
  52. - A compiler regression silently makes every downstream build slower
  53. - A browser engine ships a 10% paint regression and users notice
  54. - A crypto library that adds a microsecond to a handshake breaks an SLA
  55. Without trustworthy benchmarks in the PR workflow, you find out when users do.
  56. Local benchmarks aren't reproducible. Every check means stopping work to pull the baseline branch and wait on a comparison. Most engineers skip it.
  57. CI runners are shared and noisy. Noisy benchmarks train engineers to ignore alerts. Performance regressions silently ship.
  58. If you can't tell a real regression from noise, the results are worthless.
  59. So teams stop looking.
  60. ## How Bencher Works
  61. 1. **Run**: Run your benchmarks locally or in CI using the _exact same_ bare metal runners and your favorite benchmarking tools. The [`bencher` CLI](https://bencher.dev/docs/how-to/install-cli/) orchestrates running your benchmarks on bare metal and stores the results.
  62. 2. **Track**: Track the results of your benchmarks over time. Monitor, query, and graph the results using the Bencher web console based on the source branch, testbed, benchmark, and measure.
  63. 3. **Catch**: Catch performance regressions locally or in CI using the _exact same_ bare metal hardware. Bencher uses state of the art, customizable analytics to detect performance regressions before they merge.
  64. For the same reasons that unit tests are run to prevent feature regressions, benchmarks should be run with Bencher to prevent performance regressions.
  65. Performance bugs are bugs!
  66. ## Example PR Comment
  67. <h2><a href="https://bencher.dev/perf/bencher/reports/36a1eeff-57f5-4b99-b058-8c9c240a9f2c?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher"><img src="https://bencher.dev/favicon.svg" width="24" height="24" alt="🐰" /> Bencher Report</a></h2><table><tr><td>Project</td><td><a href="https://bencher.dev/perf/bencher?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">Bencher</a></td></tr><tr><td>Branch</td><td><a href="https://bencher.dev/perf/bencher/branches/254-merge?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">254/merge</a></td></tr><tr><td>Testbed</td><td><a href="https://bencher.dev/perf/bencher/testbeds/ubuntu-latest?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">ubuntu-latest</a></td></tr></table><blockquote><b>🚨 1 ALERT:</b> Threshold Boundary Limit exceeded!</blockquote><table><thead><tr><th>Benchmark</th><th>Measure<br/>Units</th><th>View</th><th>Benchmark Result<br/>(Result Δ%)</th><th>Upper Boundary<br/>(Limit %)</th></tr></thead><tbody><tr><td><a href="https://bencher.dev/perf/bencher/benchmarks/e93b3d71-8499-4fae-bb7c-4e540b775714?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">Adapter::Json</a></td><td><a href="https://bencher.dev/perf/bencher/measures/latency?utm_medium=referral&utm_source=cli&utm_content=comment&utm_campaign=pr+comments&utm_term=bencher">Latency<br/>microseconds (µs)</a></td><td>📈 <a href="https://bencher.dev/perf/bencher?branches=bdcbbf3c-9073-4006-b194-b11aff2f94c1&testbeds=0d991aac-b241-493a-8b0f-8d41419455d2&benchmarks=e93b3d71-8499-4fae-bb7c-4e540b775714&measures=4358146b-b647-4869-9d24-bd22bb0c49b5&start_time=1699143413000&end_time=1701735487000&upper_boundary=true">plot</a><br/>🚨 <a href="https://bencher.dev/perf/bencher/alerts/91ee27a7-2aee-41fe-b037-80b786f26cd5?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">alert</a><br/>🚷 <a href="https://bencher.dev/perf/bencher/thresholds/f6ade42d-ef45-4533-b6fe-588c1f3e9405?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">threshold</a></td><td><b>3.45<br/>(+1.52%)</b></td><td><b>3.36<br/>(102.48%)</b></td></tr></tbody></table><details><summary>Click to view all benchmark results</summary><br/><table><tr><th>Benchmark</th><th><a href="https://bencher.dev/perf/bencher/measures/latency?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">Latency</a></th><th>Benchmark Results<br/>microseconds (µs)<br/>(Result Δ%)</th><th>Upper Boundary<br/>microseconds (µs)<br/>(Limit %)</th></tr><tr><td><a href="https://bencher.dev/perf/bencher/benchmarks/e93b3d71-8499-4fae-bb7c-4e540b775714?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">Adapter::Json</a></td><td>📈 <a href="https://bencher.dev/perf/bencher?branches=bdcbbf3c-9073-4006-b194-b11aff2f94c1&testbeds=0d991aac-b241-493a-8b0f-8d41419455d2&benchmarks=e93b3d71-8499-4fae-bb7c-4e540b775714&measures=4358146b-b647-4869-9d24-bd22bb0c49b5&start_time=1699143413000&end_time=1701735487000&upper_boundary=true">view plot</a><br/>🚨 <a href="https://bencher.dev/perf/bencher/alerts/91ee27a7-2aee-41fe-b037-80b786f26cd5?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">view alert</a><br/>🚷 <a href="https://bencher.dev/perf/bencher/thresholds/f6ade42d-ef45-4533-b6fe-588c1f3e9405?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">view threshold</a></td><td><b>3.45<br/>(+1.52%)</b></td><td><b>3.36<br/>(102.48%)</b></td></tr><tr><td><a href="https://bencher.dev/perf/bencher/benchmarks/3bfd5887-83ec-4e62-8690-02855a38fbc9?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">Adapter::Magic (JSON)</a></td><td>📈 <a href="https://bencher.dev/perf/bencher?branches=bdcbbf3c-9073-4006-b194-b11aff2f94c1&testbeds=0d991aac-b241-493a-8b0f-8d41419455d2&benchmarks=3bfd5887-83ec-4e62-8690-02855a38fbc9&measures=4358146b-b647-4869-9d24-bd22bb0c49b5&start_time=1699143413000&end_time=1701735487000&upper_boundary=true">view plot</a><br/>🚷 <a href="https://bencher.dev/perf/bencher/thresholds/f6ade42d-ef45-4533-b6fe-588c1f3e9405?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">view threshold</a></td><td>3.43<br/>(+0.69%)</td><td>3.60<br/>(95.40%)</td></tr><tr><td><a href="https://bencher.dev/perf/bencher/benchmarks/3525f177-fc8f-4a92-bd2f-dda7c4e15699?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">Adapter::Magic (Rust)</a></td><td>📈 <a href="https://bencher.dev/perf/bencher?branches=bdcbbf3c-9073-4006-b194-b11aff2f94c1&testbeds=0d991aac-b241-493a-8b0f-8d41419455d2&benchmarks=3525f177-fc8f-4a92-bd2f-dda7c4e15699&measures=4358146b-b647-4869-9d24-bd22bb0c49b5&start_time=1699143413000&end_time=1701735487000&upper_boundary=true">view plot</a><br/>🚷 <a href="https://bencher.dev/perf/bencher/thresholds/f6ade42d-ef45-4533-b6fe-588c1f3e9405?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">view threshold</a></td><td>22.10<br/>(-0.83%)</td><td>24.73<br/>(89.33%)</td></tr><tr><td><a href="https://bencher.dev/perf/bencher/benchmarks/5655ed2a-3e45-4622-bdbd-39cdd9837af8?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">Adapter::Rust</a></td><td>📈 <a href="https://bencher.dev/perf/bencher?branches=bdcbbf3c-9073-4006-b194-b11aff2f94c1&testbeds=0d991aac-b241-493a-8b0f-8d41419455d2&benchmarks=5655ed2a-3e45-4622-bdbd-39cdd9837af8&measures=4358146b-b647-4869-9d24-bd22bb0c49b5&start_time=1699143413000&end_time=1701735487000&upper_boundary=true">view plot</a><br/>🚷 <a href="https://bencher.dev/perf/bencher/thresholds/f6ade42d-ef45-4533-b6fe-588c1f3e9405?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">view threshold</a></td><td>2.31<br/>(-2.76%)</td><td>2.50<br/>(92.21%)</td></tr><tr><td><a href="https://bencher.dev/perf/bencher/benchmarks/1db23e93-f909-40aa-bf42-838cc7ae05f5?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">Adapter::RustBench</a></td><td>📈 <a href="https://bencher.dev/perf/bencher?branches=bdcbbf3c-9073-4006-b194-b11aff2f94c1&testbeds=0d991aac-b241-493a-8b0f-8d41419455d2&benchmarks=1db23e93-f909-40aa-bf42-838cc7ae05f5&measures=4358146b-b647-4869-9d24-bd22bb0c49b5&start_time=1699143413000&end_time=1701735487000&upper_boundary=true">view plot</a><br/>🚷 <a href="https://bencher.dev/perf/bencher/thresholds/f6ade42d-ef45-4533-b6fe-588c1f3e9405?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">view threshold</a></td><td>2.30<br/>(-3.11%)</td><td>2.50<br/>(91.87%)</td></tr></table></details><a href="https://bencher.dev/perf/bencher/reports/36a1eeff-57f5-4b99-b058-8c9c240a9f2c?utm_medium=referral&utm_source=github&utm_content=readme&utm_campaign=readme&utm_term=bencher">🐰 View full continuous benchmarking report in Bencher</a>
  68. ## The Bencher Suite
  69. Bencher is a suite of bare metal continuous benchmarking tools:
  70. - [`bencher` CLI](https://bencher.dev/docs/how-to/install-cli/): run benchmarks and publish results
  71. - Bencher API Server: store, query, and alert on results
  72. - Bencher Console: web UI for tracking and graphing
  73. - Bencher Bare Metal [`runner`](https://bencher.dev/docs/tutorial/bare-metal/): bare metal benchmark execution
  74. The best place to start is the [Bare Metal Quickstart](https://bencher.dev/docs/tutorial/bare-metal/).
  75. For on-prem deployments, check out the [Bencher Self-Hosted Quickstart](https://bencher.dev/docs/tutorial/self-hosted/).
  76. ## Documentation
  77. - Tutorial
  78. - [Quickstart](https://bencher.dev/docs/tutorial/quickstart/)
  79. - [Bare Metal Quickstart](https://bencher.dev/docs/tutorial/bare-metal/)
  80. - [Self-Hosted Quickstart](https://bencher.dev/docs/tutorial/self-hosted/)
  81. - How To
  82. - [Install CLI](https://bencher.dev/docs/how-to/install-cli/)
  83. - [Claim Benchmark Results](https://bencher.dev/docs/how-to/claim/)
  84. - [Track Benchmarks in CI](https://bencher.dev/docs/how-to/track-benchmarks/)
  85. - [GitHub Actions](https://bencher.dev/docs/how-to/github-actions/)
  86. - [GitLab CI/CD](https://bencher.dev/docs/how-to/gitlab-ci-cd/)
  87. - [Track Custom Benchmarks](https://bencher.dev/docs/how-to/track-custom-benchmarks/)
  88. - [Track Build Time](https://bencher.dev/docs/how-to/track-build-time/)
  89. - [Track File Size](https://bencher.dev/docs/how-to/track-file-size/)
  90. - [Self-Hosted GitHub App](https://bencher.dev/docs/how-to/github-app/)
  91. - [Self-Hosted Google OAuth](https://bencher.dev/docs/how-to/google-oauth/)
  92. - Explanation
  93. - [Benchmarking Overview](https://bencher.dev/docs/explanation/benchmarking/)
  94. - [Bare Metal Overview](https://bencher.dev/docs/explanation/bare-metal/)
  95. - [`bencher run`](https://bencher.dev/docs/explanation/bencher-run/)
  96. - [Bare Metal Images](https://bencher.dev/docs/explanation/images/)
  97. - [Branches & Start Points](https://bencher.dev/docs/explanation/branches/)
  98. - [Testbeds & Specs](https://bencher.dev/docs/explanation/testbeds/)
  99. - [Thresholds & Alerts](https://bencher.dev/docs/explanation/thresholds/)
  100. - [Benchmark Adapters](https://bencher.dev/docs/explanation/adapters/)
  101. - [Continuous Benchmarking](https://bencher.dev/docs/explanation/continuous-benchmarking/)
  102. - [Bencher Self-Hosted](https://bencher.dev/docs/explanation/bencher-self-hosted/)
  103. - [Bencher Talks](https://bencher.dev/docs/explanation/talks/)
  104. - Reference
  105. - [REST API](https://bencher.dev/docs/api/)
  106. - [Bencher Metric Format](https://bencher.dev/docs/reference/bencher-metric-format/)
  107. - [Bencher Compose](https://bencher.dev/docs/reference/bencher-compose/)
  108. - [`bencher noise`](https://bencher.dev/docs/reference/bencher-noise/)
  109. - [Console Server Config](https://bencher.dev/docs/reference/console-config/)
  110. - [API Server Config](https://bencher.dev/docs/reference/server-config/)
  111. - [Changelog](https://bencher.dev/docs/reference/changelog/)
  112. - [Roadmap](https://bencher.dev/docs/reference/roadmap/)
  113. - [Prior Art](https://bencher.dev/docs/reference/prior-art/)
  114. - [Architecture](https://bencher.dev/docs/reference/architecture/)
  115. - [Database Schema](https://bencher.dev/docs/reference/schema/)
  116. 🌐 Also available in:
  117. - [简体中文](https://bencher.dev/zh/docs/)
  118. - [Español](https://bencher.dev/es/docs/)
  119. - [Português do Brasil](https://bencher.dev/pt/docs/)
  120. - [Русский](https://bencher.dev/ru/docs/)
  121. - [日本語](https://bencher.dev/ja/docs/)
  122. - [Français](https://bencher.dev/fr/docs/)
  123. - [Deutsch](https://bencher.dev/de/docs/)
  124. - [한국어](https://bencher.dev/ko/docs/)
  125. ## Supported Benchmark Harnesses
  126. - {...} JSON
  127. - [Custom benchmark harness support](https://bencher.dev/docs/explanation/adapters/#-json)
  128. - [Build Time (Compile Time)](https://bencher.dev/docs/explanation/adapters/#%EF%B8%8F-build-time)
  129. - [File Size (Binary Size)](https://bencher.dev/docs/explanation/adapters/#%EF%B8%8F-file-size)
  130. - #️⃣ C#
  131. - [BenchmarkDotNet](https://bencher.dev/docs/explanation/adapters/#%EF%B8%8F%E2%83%A3-c-dotnet)
  132. - ➕ C++
  133. - [Catch2](https://bencher.dev/docs/explanation/adapters/#-c-catch2)
  134. - [Google Benchmark](https://bencher.dev/docs/explanation/adapters/#-c-google)
  135. - 🎯 Dart
  136. - [benchmark_harness](https://bencher.dev/docs/explanation/adapters/#-dart-benchmark_harness)
  137. - 🕳 Go
  138. - [go test -bench](https://bencher.dev/docs/explanation/adapters/#-go-bench)
  139. - ☕️ Java
  140. - [Java Microbenchmark Harness (JMH)](https://bencher.dev/docs/explanation/adapters/#%EF%B8%8F-java-jmh)
  141. - 🕸 JavaScript
  142. - [Benchmark.js](https://bencher.dev/docs/explanation/adapters/#-javascript-benchmark)
  143. - [console.time/console.timeEnd](https://bencher.dev/docs/explanation/adapters/#-javascript-time)
  144. - [Vitest](https://bencher.dev/docs/explanation/adapters/#-javascript-vitest)
  145. - 🐍 Python
  146. - [airspeed velocity](https://bencher.dev/docs/explanation/adapters/#-python-asv)
  147. - [pytest-benchmark](https://bencher.dev/docs/explanation/adapters/#-python-pytest)
  148. - ♦️ Ruby
  149. - [Benchmark](https://bencher.dev/docs/explanation/adapters/#%EF%B8%8F-ruby-benchmark)
  150. - 🦀 Rust
  151. - [libtest bench](https://bencher.dev/docs/explanation/adapters/#-rust-bench)
  152. - [Criterion](https://bencher.dev/docs/explanation/adapters/#-rust-criterion)
  153. - [Iai](https://bencher.dev/docs/explanation/adapters/#-rust-iai)
  154. - [Gungraun](https://bencher.dev/docs/explanation/adapters/#-rust-gungraun) (formerly [Iai-Callgrind](https://bencher.dev/docs/explanation/adapters/#-rust-iai-callgrind))
  155. - ❯_ Shell
  156. - [Hyperfine](https://bencher.dev/docs/explanation/adapters/#_%EF%B8%8F-shell-hyperfine)
  157. 👉 For more details see the [explanation of benchmark harness adapters](https://bencher.dev/docs/explanation/adapters/).
  158. Don't see your harness? [Open an issue →](https://github.com/bencherdev/bencher/issues/new?labels=adapter&title=Adapter%20request%3A%20)
  159. ## Showcase
  160. <table>
  161. <tr>
  162. <td>
  163. <p align="center">
  164. <a href="https://bencher.dev/perf/snmalloc?key=true&reports_per_page=4&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&plots_per_page=8&reports_page=1&branches_page=1&testbeds_page=1&benchmarks_page=1&plots_page=1&branches=dc041689-5eaf-4b70-b1c8-9da23594c030&heads=3caf3a2d-f6aa-4e58-b7de-92219e9b4616&testbeds=c7457113-7c66-42a7-ac72-ee4e74d9f300&benchmarks=e6fdc724-f1e4-477e-81eb-19185eaef02d%2Cf99f94fb-805f-4338-a5ce-13d261f91afc%2C24eb178e-d9d4-45d7-90ec-1c82dfb58ea2&measures=44c8d1f2-5a3e-4573-bc7d-79f80461103c&start_time=1781395200000&end_time=1789430400000&clear=true&tab=benchmarks&x_axis=version">
  165. <img
  166. src="https://s3.amazonaws.com/public.bencher.dev/case-study/microsoft.png"
  167. alt="Microsoft"
  168. width="300px"
  169. />
  170. </a>
  171. </p>
  172. <p align="center">Microsoft snmalloc</p>
  173. </td>
  174. <td>
  175. <p align="center">
  176. <a href="https://bencher.dev/perf/sedpack?branches=e27f4617-5c19-4a91-a3f5-ca006bde2dd8&heads=e0f3701a-7886-4317-bf5c-ff04e2d0ccd1&testbeds=c83cc96a-a3b8-4c8e-88d3-d86c49caa12e&benchmarks=2fed029b-b64d-40ac-9d37-e4582ac6ad6b%2C7c8dfdfe-cc70-4928-8d09-841d7864984b&measures=37d645e6-8e9a-4731-8f16-28f12c22bd1c&upper_boundary=true&end_time=1754265600000&key=true&reports_per_page=4&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&plots_per_page=8&reports_page=3&branches_page=1&testbeds_page=1&benchmarks_page=1&plots_page=1&start_time=1741996800000&lower_boundary=false&upper_value=false&lower_value=false&tab=branches&clear=true&branches_search=main">
  177. <img
  178. src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/case-study/google.svg"
  179. alt="Google"
  180. width="300px"
  181. />
  182. </a>
  183. </p>
  184. <p align="center">Google Sedpack</p>
  185. </td>
  186. <td>
  187. <p align="center">
  188. <a href="https://bencher.dev/perf/git?lower_value=false&upper_value=false&lower_boundary=false&upper_boundary=true&x_axis=date_time&branches=595859eb-071c-48e9-97cf-195e0a3d6ed1&testbeds=02dcb8ad-6873-494c-aabc-9a6237601308&benchmarks=5e5c6ae1-ec8e-4c25-b27d-dcf773d33a51%2C0eb509fd-c4a8-45f3-baca-2e7e4a89b0e8&measures=63dafffb-98c4-4c27-ba43-7112cae627fc&tab=plots&plots_search=0d7f6186-f80a-4fbe-9022-75b6caf5164e&key=true&reports_per_page=4&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&plots_per_page=8&reports_page=1&branches_page=1&testbeds_page=1&benchmarks_page=1&plots_page=1&end_time=1745971200000&start_time=1740787200000&utm_medium=share&utm_source=bencher&utm_content=img&utm_campaign=perf%2Bimg&utm_term=git">
  189. <img
  190. src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/case-study/gitlab.svg"
  191. alt="GitLab"
  192. width="300px"
  193. />
  194. </a>
  195. </p>
  196. <p align="center">GitLab Git</p>
  197. </td>
  198. </tr>
  199. <tr>
  200. <td>
  201. <p align="center">
  202. <a href="https://bencher.dev/perf/conda-tdj8rt90?key=true&reports_per_page=4&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&plots_per_page=8&reports_page=1&branches_page=1&testbeds_page=1&benchmarks_page=1&plots_page=1&branches=67634f05-6858-4e0b-8c77-9222afaa78a3&heads=3075025a-cf38-4fbc-851c-fcbf223d35ee&testbeds=d32fdfd6-9631-4a16-b71c-da1e838eaf0f&benchmarks=fbbe288f-2b29-4395-977a-1254429d3ca5%2C08b608e7-8ebb-41ea-a579-9d812b05135c%2C0bad22f1-5340-45fd-9229-7f6cd114c48d&measures=d40e34c0-83a2-4f3e-926c-3f5870705694&start_time=1781395200000&end_time=1789430400000&upper_boundary=true&clear=true&tab=benchmarks">
  203. <img
  204. src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/case-study/conda.svg"
  205. alt="conda"
  206. width="300px"
  207. />
  208. </a>
  209. </p>
  210. <p align="center">conda</p>
  211. </td>
  212. <td>
  213. <p align="center">
  214. <a href="https://bencher.dev/perf/firezone-1l75jv1z?key=true&reports_per_page=4&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&plots_per_page=8&reports_page=1&branches_page=1&testbeds_page=1&benchmarks_page=1&plots_page=1&branches=455d0566-7315-404c-93e1-877e2732cc97&heads=455d0566-7315-404c-93e1-877e2732cc97&testbeds=258ac4fc-9cc9-4281-a45b-666cd8264371&benchmarks=f0008569-a7be-4f17-8f53-37abe7cbc31b%2Cb35d8c57-795d-4eac-9105-6911d81320a7%2Cdbebadcd-b0cf-4abd-a4f8-7274f064f6da&measures=a777fd3c-5f6c-4884-82e8-39cd8b2d30f1&start_time=1781395200000&end_time=1789430400000&clear=true&tab=benchmarks">
  215. <img
  216. src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/case-study/firezone.svg"
  217. alt="Firezone"
  218. width="300px"
  219. />
  220. </a>
  221. </p>
  222. <p align="center">Firezone</p>
  223. </td>
  224. <td>
  225. <p align="center">
  226. <a href="https://bencher.dev/perf/pnpm?key=true&reports_per_page=4&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&plots_per_page=8&reports_page=1&branches_page=1&testbeds_page=1&benchmarks_page=1&plots_page=1&branches=b06c9054-8025-4f4d-973a-36e92a38ba01&heads=751ebac5-6f89-4ed7-889d-74b2b27c8bc8&testbeds=e76d4447-6105-4421-bd5f-092ab7013510&benchmarks=d9e5dbb5-fb8b-42b4-b5bd-6c3afc491616%2Ccbb1df50-ee25-4018-9aa3-9dd41b4d125f%2C94a7b145-0268-4341-904c-6caa15e85686&measures=d54343e2-b40b-4dec-a120-2fc6cb336362&start_time=1781395200000&end_time=1789430400000&clear=true&tab=benchmarks">
  227. <img
  228. src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/case-study/pnpm.svg"
  229. alt="pnpm"
  230. width="300px"
  231. />
  232. </a>
  233. </p>
  234. <p align="center">pnpm</p>
  235. </td>
  236. </tr>
  237. <tr>
  238. <td>
  239. <p align="center">
  240. <a href="https://bencher.dev/perf/servo?key=true&reports_per_page=4&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&plots_per_page=8&reports_page=1&branches_page=1&testbeds_page=1&benchmarks_page=1&plots_page=1&branches=52e1e9bb-959c-4171-a53d-e06bd694a6c1&heads=3dbe3681-11b1-4e30-b482-4ee72dc0960c&testbeds=d742c702-3842-4108-9d0c-2db74e57599a&measures=678e4118-c8a5-494d-8799-08abc3021cd5&start_time=1734048000000&end_time=1735236203000&lower_boundary=false&upper_boundary=false&clear=true&tab=benchmarks&benchmarks=c4da10d8-9539-4943-95ca-5e08df0cd6f9&benchmarks_search=servo">
  241. <img
  242. src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/case-study/servo-tlf.svg"
  243. alt="Servo"
  244. width="300px"
  245. />
  246. </a>
  247. </p>
  248. <p align="center">Servo</p>
  249. </td>
  250. <td>
  251. <p align="center">
  252. <a href="https://bencher.dev/perf/rustls-821705769?key=true&reports_per_page=8&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&reports_page=1&branches_page=1&testbeds_page=1&benchmarks_page=1&clear=true&tab=branches&measures=013468de-9c37-4605-b363-aebbbf63268d&branches=28fae530-2b53-4482-acd4-47e16030d54f&testbeds=62ed31c3-8a58-479c-b828-52521ed67bee&benchmarks=bd25f73c-b2b9-4188-91b4-f632287c0a1b%2C8d443816-7a23-40a1-a54c-59de911eb517%2C42edb37f-ca91-4984-8835-445514575c85&start_time=1704067200000&notify_kind=alert&notify_text=Learn%20more%20about%20continuous%20benchmarking%20for%20the%20Rustls%20project.&notify_timeout=2147483647&notify_link_url=https%3A%2F%2Fbencher.dev%2Flearn%2Fcase-study%2Frustls%2F&notify_link_text=Read%20the%20case%20study">
  253. <img
  254. src="https://s3.amazonaws.com/public.bencher.dev/case-study/rustls-rust-tls.png"
  255. alt="Rustls TLS Library"
  256. width="300px"
  257. />
  258. </a>
  259. </p>
  260. <p align="center">Rustls</p>
  261. </td>
  262. <td>
  263. <p align="center">
  264. <a href="https://bencher.dev/perf/clap-rs-clap?key=true&reports_per_page=4&branches_per_page=8&testbeds_per_page=8&benchmarks_per_page=8&plots_per_page=8&reports_page=1&branches_page=1&testbeds_page=1&benchmarks_page=1&plots_page=1&branches=b920383c-b9ee-4bd6-94ea-8d101b55286a&heads=5eeccfee-4fdd-405a-8554-90cd945ee1c1&testbeds=551ebdbf-b50a-4813-9064-286d2e66888f&benchmarks=b0a8ca01-4418-485e-9446-81d2a9c62774&measures=04ff075b-dc09-4c77-909a-634352fd5b02&end_time=1767052800000&lower_boundary=false&upper_boundary=false&clear=true&start_time=1748908800000&tab=branches&branches_search=master">
  265. <img
  266. src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/case-study/clap.png"
  267. alt="clap"
  268. width="300px"
  269. />
  270. </a>
  271. </p>
  272. <p align="center">clap</p>
  273. </td>
  274. </tr>
  275. </table>
  276. 👉 Checkout [all public projects](https://bencher.dev/perf).
  277. ## What Engineers Say
  278. <table>
  279. <tr>
  280. <td>
  281. <p>Bencher is like CodeCov for performance metrics.</p>
  282. <br />
  283. <p align="center">
  284. <a href="https://github.com/JonathanWoollett-Light">
  285. <img src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/customers/JonathanWoollett-Light.jpg" width="48" height="48" alt="Jonathan Woollett-Light" />
  286. </a>
  287. <br />
  288. Jonathan Woollett-Light
  289. <br />
  290. <a href="https://github.com/JonathanWoollett-Light">
  291. @JonathanWoollett-Light
  292. </a>
  293. </p>
  294. </td>
  295. <td>
  296. <p>I think I'm in heaven. Now that I'm starting to see graphs of performance over time automatically from tests I'm running in CI. It's like this whole branch of errors can be caught and noticed sooner.</p>
  297. <br />
  298. <p align="center">
  299. <a href="https://github.com/gpwclark">
  300. <img src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/customers/gpwclark.jpg" width="48" height="48" alt="Price Clark" />
  301. </a>
  302. <br />
  303. Price Clark
  304. <br />
  305. <a href="https://github.com/gpwclark">
  306. @gpwclark
  307. </a>
  308. </p>
  309. </td>
  310. </tr>
  311. <tr>
  312. <td>
  313. <p>95% of the time I don't want to think about my benchmarks. But when I need to, Bencher ensures that I have the detailed historical record waiting there for me. It's fire-and-forget.</p>
  314. <br />
  315. <p align="center">
  316. <a href="https://github.com/jneem">
  317. <img src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/customers/jneem.jpg" width="48" height="48" alt="Joe Neeman" />
  318. </a>
  319. <br />
  320. Joe Neeman
  321. <br />
  322. <a href="https://github.com/jneem">
  323. @jneem
  324. </a>
  325. </p>
  326. </td>
  327. <td>
  328. <p>I've been looking for a public service like Bencher for about 10 years.</p>
  329. <br />
  330. <p align="center">
  331. <a href="https://github.com/jaqx0r">
  332. <img src="https://s3.us-east-1.amazonaws.com/public.bencher.dev/customers/jaqx0r.png" width="48" height="48" alt="Jamie Wilkinson" />
  333. </a>
  334. <br />
  335. Jamie Wilkinson
  336. <br />
  337. <a href="https://github.com/jaqx0r">
  338. @jaqx0r
  339. </a>
  340. </p>
  341. </td>
  342. </tr>
  343. </table>
  344. ## Hosting
  345. - **Bencher Cloud**: Zero infrastructure to manage. On-demand bare metal runners, billed by the minute. Pay for your benchmark runs, not idle servers. [Benchmark for free →](https://bencher.dev/auth/signup)
  346. - **Bencher Self-Hosted**: Deploy Bencher on your own infrastructure. Bare metal, Docker, or Kubernetes. Full control, no data leaving your environment. [Deploy in 60 seconds →](https://bencher.dev/docs/tutorial/self-hosted/)
  347. ## GitHub Actions
  348. Install the Bencher CLI using the [GitHub Action](https://github.com/marketplace/actions/bencher-cli),
  349. and use it for [continuous benchmarking](https://bencher.dev/docs/explanation/continuous-benchmarking/) in your project.
  350. ```yaml
  351. name: Continuous Benchmarking with Bencher
  352. on:
  353. push:
  354. branches: main
  355. jobs:
  356. benchmark_with_bencher:
  357. name: Benchmark with Bencher
  358. runs-on: ubuntu-latest
  359. env:
  360. BENCHER_PROJECT: my-project-slug
  361. BENCHER_API_TOKEN: ${{ secrets.BENCHER_API_TOKEN }}
  362. steps:
  363. - uses: actions/checkout@v6
  364. - uses: bencherdev/bencher@main
  365. - run: bencher run "bencher mock"
  366. ```
  367. Supported Operating Systems:
  368. - Linux (x86_64 & ARM64)
  369. - macOS (x86_64 & ARM64)
  370. - Windows (x86_64 & ARM64)
  371. <br />
  372. 👉 For more details see the [explanation of how to use GitHub Actions](https://bencher.dev/docs/how-to/github-actions/).
  373. ### Repository Secrets
  374. Add `BENCHER_API_TOKEN` to you **Repository** secrets (ex: `Repo -> Settings -> Secrets and variables -> Actions -> New repository secret`). You can find your API tokens by running `bencher token list my-user-slug` or [view them in the Bencher Console](https://bencher.dev/console/users/tokens).
  375. ### Error on Alert
  376. You can set the `bencher run` CLI subcommand to error
  377. if [an Alert is generated](https://bencher.dev/docs/explanation/thresholds/) with the `--error-on-alert` flag.
  378. ```bash
  379. bencher run --error-on-alert "bencher mock"
  380. ```
  381. 👉 For more details see the [explanation of `bencher run`](https://bencher.dev/docs/explanation/bencher-run/#--error-on-alert).
  382. ### Comment on PRs
  383. You can set the `bencher run` CLI subcommand to comment on a PR with the `--github-actions` argument.
  384. ```bash
  385. bencher run --github-actions "${{ secrets.GITHUB_TOKEN }}" "bencher mock"
  386. ```
  387. 👉 For more details see the [explanation of `bencher run`](https://bencher.dev/docs/explanation/bencher-run/#--github-actions/).
  388. 👉 See the [example PR comment above](#example-pr-comment).
  389. ### Specify CLI Version
  390. There is also an optional `version` argument to specify an exact version of the Bencher CLI to use.
  391. Otherwise, it will default to using the latest CLI version.
  392. ```yaml
  393. - uses: bencherdev/bencher@main
  394. with:
  395. version: 0.6.17
  396. ```
  397. Specify an exact version if using [Bencher _Self-Hosted_](https://bencher.dev/docs/explanation/bencher-self-hosted/).
  398. Do **not** specify an exact version if using Bencher _Cloud_ as there are still occasional breaking changes.
  399. ## Contributing
  400. To set up a development environment, see the [Development Getting Started](DEVELOPMENT.md) guide.
  401. #### 🐰 All pull requests should target the `devel` branch
  402. <br />
  403. <p align="center">
  404. <a href="https://github.dev/bencherdev/bencher">
  405. <img
  406. src="https://img.shields.io/static/v1?label=GitHub%20Codespaces&message=Open&color=orange&logo=github&style=for-the-badge"
  407. alt="Bencher GitHub Codespaces"
  408. />
  409. </a>
  410. </p>
  411. ## License
  412. All content that resides under any directory or [feature](https://doc.rust-lang.org/cargo/reference/features.html) named "plus" is licensed under the [Bencher Plus License](license/LICENSE-PLUS).
  413. All other content is licensed under the [Apache License, Version 2.0](license/LICENSE-APACHE) or [MIT License](license/LICENSE-MIT) at your discretion.
  414. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Bencher by you, as defined in the Apache-2.0 license, shall be licensed as above, without any additional terms or conditions.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.