Skip to content
Back to search

web-eval-agent

An MCP server that autonomously evaluates web applications.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/refreshdotdev--web-eval-agent/badge.svg)](/listings/refreshdotdev--web-eval-agent)

HTML

<a href="/listings/refreshdotdev--web-eval-agent"><img src="/api/listings/refreshdotdev--web-eval-agent/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Easy Setup with One-Click Integration

1. [Get your API key (free)](https://www.operative.sh/mcp) - when you create your API key, you'll see: - **"Add to Cursor"** button with a deeplink for instant Cursor installation - **Prefilled Claude Code command** with your API key automatically included

Manual Setup (macOS/Linux)

1. Pre-requisites (typically not needed): - brew: `/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"` - npm: (`brew install npm`) - jq: `brew install jq` 2. Run the installer after [getting an api key (free)](https://www.operative.sh/mcp) - Installs [playwright](https://github.com/microsoft/playwright) - [Installs uv](https://astral.sh/) - Inserts JSON into your code editor (Cursor/Cline/Windsurf) for you!

curl -LSf https://operative.sh/install.sh -o install.sh && bash install.sh && rm install.sh

3. Visit your favorite IDE and restart to apply the changes 4. Send a prompt in chat mode to call the web eval agent tool! e.g.

Test my app on http://localhost:3000. Use web-eval-agent.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 201 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # ⚠️ PROJECT HAS BEEN SUNSET ⚠️
  2. ## This project has been discontinued. We're building something new at [withrefresh.com](https://withrefresh.com)
  3. ---
  4. # 🚀 operative.sh web-eval-agent MCP Server
  5. > *Let the coding agent debug itself, you've got better things to do.*
  6. ![Demo](./demo.gif)
  7. ## 🔥 Supercharge Your Debugging
  8. [operative.sh](https://www.operative.sh/mcp)'s MCP Server launches a browser-use powered agent to autonomously execute and debug web apps directly in your code editor.
  9. ## ⚡ Features
  10. - 🌐 **Navigate your webapp** using BrowserUse (2x faster with operative backend)
  11. - 📊 **Capture network traffic** - requests are intelligently filtered and returned into the context window
  12. - 🚨 **Collect console errors** - captures logs & errors
  13. - 🤖 **Autonomous debugging** - the Cursor agent calls the web QA agent mcp server to test if the code it wrote works as epected end-to-end.
  14. ## 🧰 MCP Tool Reference
  15. | Tool | Purpose |
  16. |------|---------|
  17. | `web_eval_agent` | 🤖 Automated UX evaluator that drives the browser, captures screenshots, console & network logs, and returns a rich UX report. |
  18. | `setup_browser_state` | 🔒 Opens an interactive (non-headless) browser so you can sign in once; the saved cookies/local-storage are reused by subsequent `web_eval_agent` runs. |
  19. **Key arguments**
  20. * `web_eval_agent`
  21. * `url` **(required)** – address of the running app (e.g. `http://localhost:3000`)
  22. * `task` **(required)** – natural-language description of what to test ("run through the signup flow and note any UX issues")
  23. * `headless_browser` *(optional, default `false`)* – set to `true` to hide the browser window
  24. * `setup_browser_state`
  25. * `url` *(optional)* – page to open first (handy to land directly on a login screen)
  26. You can trigger these tools straight from your IDE chat, for example:
  27. ```bash
  28. Evaluate my app at http://localhost:3000 – run web_eval_agent with the task "Try the full signup flow and report UX issues".
  29. ```
  30. ## 🏁 Quick Start
  31. ### Easy Setup with One-Click Integration
  32. 1. [Get your API key (free)](https://www.operative.sh/mcp) - when you create your API key, you'll see:
  33. - **"Add to Cursor"** button with a deeplink for instant Cursor installation
  34. - **Prefilled Claude Code command** with your API key automatically included
  35. ### Manual Setup (macOS/Linux)
  36. 1. Pre-requisites (typically not needed):
  37. - brew: `/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"`
  38. - npm: (`brew install npm`)
  39. - jq: `brew install jq`
  40. 2. Run the installer after [getting an api key (free)](https://www.operative.sh/mcp)
  41. - Installs [playwright](https://github.com/microsoft/playwright)
  42. - [Installs uv](https://astral.sh/)
  43. - Inserts JSON into your code editor (Cursor/Cline/Windsurf) for you!
  44. ```bash
  45. curl -LSf https://operative.sh/install.sh -o install.sh && bash install.sh && rm install.sh
  46. ```
  47. 3. Visit your favorite IDE and restart to apply the changes
  48. 4. Send a prompt in chat mode to call the web eval agent tool! e.g.
  49. ```bash
  50. Test my app on http://localhost:3000. Use web-eval-agent.
  51. ```
  52. ## 🛠️ Manual Installation
  53. 1. Get your API key at operative.sh/mcp
  54. 2. [Install uv](https://docs.astral.sh/uv/#highlights)
  55. ```bash
  56. curl -LsSf https://astral.sh/uv/install.sh | sh
  57. ```
  58. 3. Source environment variables after installing UV
  59. Mac
  60. ```
  61. source ~/.zshrc
  62. ```
  63. Linux
  64. ```
  65. source ~/.bashrc
  66. ```
  67. 4. Install playwright:
  68. ```bash
  69. npm install -g chromium playwright && uvx --with playwright playwright install --with-deps
  70. ```
  71. 5. Add below JSON to your relevant code editor with api key
  72. 6. Restart your code editor
  73. ## 🔃 Updating
  74. - `uv cache clean`
  75. - refresh MCP server
  76. ```json
  77. "web-eval-agent": {
  78. "command": "uvx",
  79. "args": [
  80. "--refresh-package",
  81. "webEvalAgent",
  82. "--from",
  83. "git+https://github.com/Operative-Sh/web-eval-agent.git",
  84. "webEvalAgent"
  85. ],
  86. "env": {
  87. "OPERATIVE_API_KEY": "<YOUR_KEY>"
  88. }
  89. }
  90. ```
  91. ## [Operative Discord Server](https://discord.gg/ryjCnf9myb)
  92. ## 🛠️ Manual Installation (Mac + Cursor/Cline/Windsurf)
  93. 1. Get your API key at operative.sh/mcp
  94. 2. [Install uv](https://docs.astral.sh/uv/#highlights)
  95. ```bash
  96. curl -LsSf https://astral.sh/uv/install.sh | sh)
  97. ```
  98. 3. Install playwright:
  99. ```bash
  100. npm install -g chromium playwright && uvx --with playwright playwright install --with-deps
  101. ```
  102. 4. Add below JSON to your relevant code editor with api key
  103. 5. Restart your code editor
  104. ## Manual Installation (Windows + Cursor/Cline/Windsurf)
  105. We're refining this, please open an issue if you have any issues!
  106. 1. Do all this in your code editor terminal
  107. 2. `curl -LSf https://operative.sh/install.sh -o install.sh && bash install.sh && rm install.sh`
  108. 3. Get your API key at operative.sh/mcp
  109. 4. Install uv `(curl -LsSf https://astral.sh/uv/install.sh | sh)`
  110. 5. `uvx --from git+https://github.com/Operative-Sh/web-eval-agent.git playwright install`
  111. 6. Restart code editor
  112. ## 🚨 Issues
  113. - Updates aren't being received in code editors, update or reinstall for latest version: Run `uv cache clean` for latest
  114. - Any issues feel free to open an Issue on this repo or in the discord!
  115. - 5/5 - static apps without changes weren't screencasting, fixed! `uv clean` + restart to get fix
  116. ## Changelog
  117. - 4/29 - Agent overlay update - pause/play/stop agent run in the browser
  118. ## 📋 Example MCP Server Output Report
  119. ```text
  120. 📊 Web Evaluation Report for http://localhost:5173 complete!
  121. 📝 Task: Test the API-key deletion flow by navigating to the API Keys section, deleting a key, and judging the UX.
  122. 🔍 Agent Steps
  123. 📍 1. Navigate → http://localhost:5173
  124. 📍 2. Click "Login" (button index 2)
  125. 📍 3. Click "API Keys" (button index 4)
  126. 📍 4. Click "Create Key" (button index 9)
  127. 📍 5. Type "Test API Key" (input index 2)
  128. 📍 6. Click "Done" (button index 3)
  129. 📍 7. Click "Delete" (button index 10)
  130. 📍 8. Click "Delete" (confirm index 3)
  131. 🏁 Flow tested successfully – UX felt smooth and intuitive.
  132. 🖥️ Console Logs (10)
  133. 1. [debug] [vite] connecting…
  134. 2. [debug] [vite] connected.
  135. 3. [info] Download the React DevTools …
  136. …
  137. 🌐 Network Requests (10)
  138. 1. GET /src/pages/SleepingMasks.tsx 304
  139. 2. GET /src/pages/MCPRegistryRegistry.tsx 304
  140. …
  141. ⏱️ Chronological Timeline
  142. 01:16:23.293 🖥️ Console [debug] [vite] connecting…
  143. 01:16:23.303 🖥️ Console [debug] [vite] connected.
  144. 01:16:23.312 ➡️ GET /src/pages/SleepingMasks.tsx
  145. 01:16:23.318 ⬅️ 304 /src/pages/SleepingMasks.tsx
  146. …
  147. 01:17:45.038 🤖 🏁 Flow finished – deletion verified
  148. 01:17:47.038 🤖 📋 Conclusion repeated above
  149. 👁️ See the "Operative Control Center" dashboard for live logs.
  150. ```
  151. ## Star History
  152. [![Star History Chart](https://api.star-history.com/svg?repos=Operative-Sh/web-eval-agent&type=Date)](https://www.star-history.com/#Operative-Sh/web-eval-agent&Date)
  153. ---
  154. Built with <3 @ [operative.sh](https://www.operative.sh)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.