Skip to content
Back to search

cuawright

MarketplaceWorks with Claude CodeGitHub

A simple SWE-style browser+desktop agent framework that achieves SOTA results on long-horizon CUA tasks

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/microsoft--cuawright/badge.svg)](/listings/microsoft--cuawright)

HTML

<a href="/listings/microsoft--cuawright"><img src="/api/listings/microsoft--cuawright/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Webwright lets a coding model launch browsers, inspect pages, and debug its work from a terminal. Script-based runs produce a reusable Python script. Live-browser mode keeps a page open across steps and returns an answer without creating a script.

Prerequisites

- Python 3.10+ - Chromium installed through Playwright - An API key for your chosen backend (OpenAI, Anthropic, or OpenRouter)

Install

pip install -e .
playwright install chromium

Script-based run

Export credentials for the configured backend (for example, `OPENAI_API_KEY` with `model_openai.yaml` or `ANTHROPIC_API_KEY` with `model_claude.yaml`). Then:

python -m cuawright.webwright.run.cli main \
    -c base.yaml -c model_openai.yaml \
    -t "Report the page title and the destination of its main link." \
    --start-url https://example.com \
    --task-id demo_openai \
    -o outputs/default

The `image_qa` and `self_reflection` tools use the run's configured model. When running a tool separately, pass `--model-config` with an absolute path to a private YAML file containing your `model:` settings. Keep that file outside the checkout and results directory.

Live-browser mode

For incremental browsing with native OpenAI `run_command` tool calls, use:

cuawright-web main -c best_default_judge_json_persistent_cli.yaml -c model_openai.yaml \
  -t "<task>" --start-url "<url>" --task-id example -o outputs/example

This config keeps a Browserbase cloud session across commands. Set `OPENAI_API_KEY`, `BROWSERBASE_API_KEY`, and `BROWSERBASE_PROJECT_ID` before running. The agent runs shell commands, inspects the results, and checks its work before finishing. See the [browser workflow guide](docs/browser.md) for image reading and trajectory verification.

🚩 Flags

| Flag | Description | |------|-------------| | `-c` | Config file(s) from `src/cuawright/webwright/config/` (stackable). | | `-t` | Task instruction. | | `--start-url` | Initial page. | | `--task-id` | Output subfolder name. | | `-o` | Output directory. |

---

<a id="desktop-quick-start"></a>

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json Β· Manifest Β· 26 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "cuawright",
  3. "owner": {
  4. "name": "Microsoft Research",
  5. "url": "https://github.com/microsoft/Webwright"
  6. },
  7. "plugins": [
  8. {
  9. "name": "cuawright",
  10. "source": {
  11. "source": "github",
  12. "repo": "microsoft/Webwright"
  13. },
  14. "description": "CUAWright browser automation skill; desktop runtime available through cuawright-desktop.",
  15. "version": "0.2.0",
  16. "category": "automation",
  17. "keywords": [
  18. "browser",
  19. "playwright",
  20. "web-agent",
  21. "automation",
  22. "rpa"
  23. ]
  24. }
  25. ]
  26. }
README.md Β· README Β· 583 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # CUAWright
  2. > **Webwright is now CUAWright.** Alongside browser automation, you can now run
  3. > desktop tasks in an Ubuntu VM. Use `cuawright web` for browser tasks and
  4. > `cuawright desktop` for desktop tasks. Existing `webwright` commands and Python
  5. > imports still work.
  6. <p align="center">
  7. <img src="assets/cuawright_logo.svg" alt="CUAWright logo" width="320">
  8. </p>
  9. <p align="center"><b>Turn coding models into browser and desktop agents</b></p>
  10. <p align="center">
  11. <img src="https://img.shields.io/badge/python-%E2%89%A53.10-blue?logo=python&logoColor=white" alt="Python">
  12. <img src="https://img.shields.io/badge/playwright-chromium-green" alt="Playwright">
  13. <img src="https://img.shields.io/badge/backends-OpenAI%20%7C%20Anthropic%20%7C%20OpenRouter-orange" alt="Backends">
  14. </p>
  15. - πŸ“„ **Paper:** [CUAWright: A Minimal Unified Interface for Digital Agents](https://arxiv.org/abs/2610.04116)
  16. - πŸ“ **Blog:** [Webwright: A Terminal Is All You Need For Web Agents](https://www.microsoft.com/en-us/research/articles/webwright-a-terminal-is-all-you-need-for-web-agents/)
  17. - 🌐 **Project Page:** [microsoft.github.io/CUAWright](https://microsoft.github.io/CUAWright/)
  18. CUAWright gives coding models a terminal to automate browsers and desktop applications.
  19. Use Webwright to browse the web and build reusable Playwright scripts, or run your
  20. own desktop task and reproduce OSWorld-V2 benchmarks in an Ubuntu VM.
  21. Start with the [browser quick start](#-browser-quick-start) or
  22. [desktop quick start](#desktop-quick-start).
  23. Already got your favorite agents, and wonder how to make Claude Code, Codex, Hermes more capable in browser tasks? Consider adding [CUAWright browser plugin/skill](#-use-as-a-plugin)!
  24. ---
  25. ## πŸ“° News
  26. - **2026-10-03** β€” **Webwright β†’ CUAWright:** renamed the project to bring browser and desktop agents together. The Webwright browser runtime is now `cuawright.webwright`, and OSWorld desktop support lives at `cuawright.desktop`. Existing `webwright` commands and imports remain compatible. See [desktop setup](docs/desktop.md).
  27. - **2026-09-01** β€” Persistent step-by-step browsing and native `run_command` tool calls improve performance to **88.1% on Online-Mind2Web** and **77.5% on Odysseys**.
  28. - **2026-07-21** β€” Skill Factory: every solve leaves a script behind, distilled into reusable, verified, parameterized code skills that rerun standalone with no model (~40 s, zero tokens). On WebArena, reuse lifts held-out accuracy 55% β†’ 70% (+15 pp). See the optional [Skill Factory](#-skill-factory-turn-solved-tasks-into-runnable-code-skills).
  29. - **2026-05-11** β€” Support Task2UI mode: Webwright completes the task and renders task results into an HTML-based web app you can easily view and reuse.
  30. - **2026-05-06** β€” Codex and Claude Code plugin manifests added; install via `/plugin install cuawright@cuawright`. Hermes Agent integration shipped; the same `skills/cuawright-web/` folder now loads across Claude Code, Codex, and Hermes.
  31. - **2026-05-04** β€” Initial public release: ~1.5k LoC, OpenAI / Anthropic / OpenRouter backends, Playwright environment.
  32. ---
  33. <details>
  34. <summary><strong>πŸ’‘ Motivation: Beyond Step-by-Step Web Interaction in a Stateful Browser</strong></summary>
  35. Most web agents today treat the browser session itself as the workspace: at each step the model receives the current page state and predicts a single next operation β€” a click, a type, a DOM selector, or a short tool call. Whatever the format, the agent is locked into predicting one web action at a time inside a predefined interaction loop. That harness was useful when LLMs were weaker. As models get stronger at writing and debugging code, the same harness becomes a bottleneck.
  36. Webwright takes a different stance: **separate the agent from the browser**, and treat the browser as something the agent can launch, inspect, and discard while developing a program. The persistent artifact is not the browser session β€” it's the **code and logs in the local workspace**.
  37. - 🧱 **Robust, reusable interaction with web environments** β€” instead of fragile pixel-level actions, a coding agent with a terminal queries elements, waits for conditions, and handles dynamic behaviors like lazy loading or re-rendering. The resulting scripts can be rerun, adapted, and shared across tasks rather than rediscovered from scratch.
  38. - ⚑ **Efficient composition of complex workflows** β€” multi-step interactions like selecting a date or filling a form become a compact program. Loops, functions, and abstractions let the agent generalize across similar tasks (e.g. different dates) without re-predicting the same low-level sequences. Fewer interaction rounds, faster execution, less error accumulation on long horizons.
  39. - πŸ§ͺ **Workspace-as-state, not browser-as-state** β€” the agent can write exploratory scripts, spawn fresh browser sessions, and decide for itself when to capture screenshots and inspect failures, much like a human engineer iterating on an RPA script.
  40. - πŸͺ„ **Surprisingly effective despite being minimal** β€” this stripped-down setup turns out to handle complex and especially long-horizon web tasks well (see [Performance](#-performance)).
  41. </details>
  42. ---
  43. <details>
  44. <summary><strong>🌟 Why Webwright</strong></summary>
  45. Most web agent frameworks bury the actual agent loop under layers of abstractions. Webwright takes the opposite stance:
  46. - πŸͺΆ **Readable core** β€” browser agent loop, environments, models, and CLI are separate modules under `src/cuawright/webwright/`.
  47. - 🧩 **Pluggable model backends** β€” OpenAI, Anthropic, and OpenRouter, each ~150–200 lines.
  48. - πŸ” **Zero hidden frameworks** β€” just `httpx`, `pydantic`, `playwright`, and `typer`.
  49. - πŸ” **Flat prompt β†’ observe β†’ execute script loop** β€” readable end-to-end, easy to debug, easy to fork.
  50. - πŸ§ͺ **Run-artifact first** β€” every run writes trajectories and screenshots to disk for inspection.
  51. If you want a minimal, easy-to-debug starting point for browser-using agents instead of another heavyweight platform, this is it.
  52. </details>
  53. ---
  54. <details>
  55. <summary><strong>πŸ†š How Webwright Differs From Other Browser-Agent Repos</strong></summary>
  56. How they differ at the architectural level:
  57. | | **Stagehand (Browserbase)** | **agent-browser (Vercel)** | **browser-use** | **Webwright** |
  58. | ------------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------- |
  59. | **Paradigm** | Hybrid: code + NL primitives (`act` / `extract` / `agent`) | CLI tool that *another* agent (Claude Code, Codex, etc.) calls | Autonomous LLM agent loop over DOM/AX snapshots | **Coding agent with a terminal**; browser is just an environment it spawns |
  60. | **Action space** | Playwright code, or NL β†’ LLM-translated Playwright | Discrete subcommands (`open`, `click @e2`, `snapshot`, `eval`) | Indexed click/type actions selected by the LLM | **Free-form Python (writes Playwright scripts itself)** |
  61. | **What is "state"?**| The browser session | The browser session (held by daemon across CLI calls) | The browser session | **The local workspace β€” code, screenshots, logs.** Browser is disposable. |
  62. | **Loop shape** | Imperative; `agent()` does multi-step when needed | One CLI invocation per micro-step | observe β†’ predict next action β†’ execute β†’ repeat | write code β†’ execute β†’ inspect screenshots β†’ repair (code-as-action) |
  63. </details>
  64. ---
  65. ## πŸŽ₯ Demo
  66. **CUAWright demo**
  67. https://github.com/user-attachments/assets/2063be64-4d0a-40a2-9d1f-7906be43f2c9
  68. <details>
  69. <summary><strong>Webwright demo</strong></summary>
  70. https://github.com/user-attachments/assets/4ed94cd5-11be-4daa-b2d7-1260a803baca
  71. </details>
  72. ---
  73. ## πŸ“Š Performance
  74. CUAWright beats the same model running in a different harness on every benchmark
  75. below, from desktop and CAD to long-horizon web tasks. See the
  76. [paper](https://arxiv.org/abs/2610.04116) for full details.
  77. <p align="center">
  78. <img src="assets/main_results.png" alt="CUAWright results on OSWorld-V2, CADGenBench, BenchCAD, Online-Mind2Web, Odysseys, and WeaveBench" width="49%">
  79. <img src="assets/osworld_v2_cost.png" alt="OSWorld-V2 partial score versus API cost per task" width="49%">
  80. </p>
  81. - πŸ–₯️ **OSWorld-V2:** **67.9%** partial score with GPT-5.6 Sol (+5.2 over GPT-5.6 Sol alone) and **63.2%** with GPT-5.5 (+15.7 over the official GPT-5.5 OSWorld agent).
  82. - 🌐 **Web:** **88.1%** on Online-Mind2Web and **77.5%** on Odysseys with GPT-5.4, against 83.4% and 33.5% for a GPT-5.4 GUI agent.
  83. - πŸ“ **CAD:** **79.0%** Vision2Code mean IoU on BenchCAD with GPT-5.5, and **55.7%** aggregate score on CADGenBench with GPT-5.6 Sol.
  84. - πŸ’° **Cost:** on OSWorld-V2, CUAWright raises the score while cutting API cost by $9.5 per task with GPT-5.6 Sol and $6.0 per task with GPT-5.5.
  85. ---
  86. ## πŸ—ΊοΈ Project Map
  87. ```text
  88. CUAWright/
  89. β”œβ”€β”€ pyproject.toml # package, dependency extras, CLI commands
  90. β”œβ”€β”€ setup.py # desktop provenance in wheels and source archives
  91. β”œβ”€β”€ src/
  92. β”‚ β”œβ”€β”€ cuawright/
  93. β”‚ β”‚ β”œβ”€β”€ core/ # shared API helpers
  94. β”‚ β”‚ β”œβ”€β”€ webwright/ # browser runtime
  95. β”‚ β”‚ β”‚ β”œβ”€β”€ agents/ # browser agent loop
  96. β”‚ β”‚ β”‚ β”œβ”€β”€ environments/ # browser and terminal workspaces
  97. β”‚ β”‚ β”‚ β”œβ”€β”€ models/ # OpenAI, Anthropic, OpenRouter backends
  98. β”‚ β”‚ β”‚ β”œβ”€β”€ tools/ # browser sessions, images, self-reflection
  99. β”‚ β”‚ β”‚ β”œβ”€β”€ config/ # stackable browser YAML configs
  100. β”‚ β”‚ β”‚ β”œβ”€β”€ run/ # cuawright-web CLI and doctor
  101. β”‚ β”‚ β”‚ └── utils/ # evidence, logging, serialization
  102. β”‚ β”‚ └── desktop/ # persistent desktop runtime
  103. β”‚ β”‚ β”œβ”€β”€ agents/ # desktop agent loop and call budget
  104. β”‚ β”‚ β”œβ”€β”€ environments/osworld/ # VM commands, screenshots, and submission
  105. β”‚ β”‚ β”œβ”€β”€ models/ # standard Responses API transport and tools
  106. β”‚ β”‚ β”œβ”€β”€ config/prompts.py # desktop prompts
  107. β”‚ β”‚ β”œβ”€β”€ setup.py # pinned OSWorld setup and verification
  108. β”‚ β”‚ β”œβ”€β”€ run/cli.py # custom and official desktop workflows
  109. β”‚ β”‚ β”œβ”€β”€ run/benchmarks/ # official task setup and evaluation
  110. β”‚ β”‚ └── utils/ # artifacts and provenance
  111. β”‚ └── webwright/__init__.py # legacy Python import compatibility
  112. β”œβ”€β”€ extensions/skill-factory/ # optional cuawright-skill-factory distribution
  113. β”‚ β”œβ”€β”€ pyproject.toml # separate wheel; depends on the base package
  114. β”‚ └── src/cuawright/webwright/ # skill_factory/ and tools/skill_use.py
  115. β”œβ”€β”€ skills/cuawright-web/ # browser skill, commands, reference guides
  116. β”œβ”€β”€ .claude-plugin/ # Claude Code plugin and marketplace manifests
  117. β”œβ”€β”€ .codex-plugin/ # Codex plugin manifest
  118. β”œβ”€β”€ docs/ # desktop setup and Skill Factory guides
  119. β”œβ”€β”€ tests/ # browser, Skill Factory, compatibility tests
  120. β”œβ”€β”€ release/osworld/tests/ # desktop request, lifecycle, privacy tests
  121. β”œβ”€β”€ .github/workflows/ # runtime and Skill Factory CI
  122. β”œβ”€β”€ assets/ # showcase, trajectory viewer, figures, logos
  123. β”œβ”€β”€ LICENSE # MIT browser license
  124. β”œβ”€β”€ licenses/ # Apache-2.0 desktop runtime license
  125. └── NOTICE # imported runtime attribution
  126. ```
  127. ---
  128. ## πŸš€ Browser Quick Start
  129. Webwright lets a coding model launch browsers, inspect pages, and debug its work
  130. from a terminal. Script-based runs produce a reusable Python script. Live-browser
  131. mode keeps a page open across steps and returns an answer without creating a script.
  132. ### Prerequisites
  133. - Python 3.10+
  134. - Chromium installed through Playwright
  135. - An API key for your chosen backend (OpenAI, Anthropic, or OpenRouter)
  136. ### Install
  137. ```bash
  138. pip install -e .
  139. playwright install chromium
  140. ```
  141. ### Script-based run
  142. Export credentials for the configured backend (for example, `OPENAI_API_KEY`
  143. with `model_openai.yaml` or `ANTHROPIC_API_KEY` with `model_claude.yaml`). Then:
  144. ```bash
  145. python -m cuawright.webwright.run.cli main \
  146. -c base.yaml -c model_openai.yaml \
  147. -t "Report the page title and the destination of its main link." \
  148. --start-url https://example.com \
  149. --task-id demo_openai \
  150. -o outputs/default
  151. ```
  152. The `image_qa` and `self_reflection` tools use the run's configured model.
  153. When running a tool separately, pass `--model-config` with an absolute path to a
  154. private YAML file containing your `model:` settings. Keep that file outside the
  155. checkout and results directory.
  156. ### Live-browser mode
  157. For incremental browsing with native OpenAI `run_command` tool calls, use:
  158. ```bash
  159. cuawright-web main -c best_default_judge_json_persistent_cli.yaml -c model_openai.yaml \
  160. -t "<task>" --start-url "<url>" --task-id example -o outputs/example
  161. ```
  162. This config keeps a Browserbase cloud session across commands. Set
  163. `OPENAI_API_KEY`, `BROWSERBASE_API_KEY`, and `BROWSERBASE_PROJECT_ID` before running.
  164. The agent runs shell commands, inspects the results, and checks its work before
  165. finishing. See the [browser workflow guide](docs/browser.md) for image reading and
  166. trajectory verification.
  167. ### 🚩 Flags
  168. | Flag | Description |
  169. |------|-------------|
  170. | `-c` | Config file(s) from `src/cuawright/webwright/config/` (stackable). |
  171. | `-t` | Task instruction. |
  172. | `--start-url` | Initial page. |
  173. | `--task-id` | Output subfolder name. |
  174. | `-o` | Output directory. |
  175. ---
  176. <a id="desktop-quick-start"></a>
  177. ## πŸ–₯️ Desktop Quick Start
  178. `cuawright desktop` runs tasks in an OSWorld-V2 Ubuntu VM. The agent uses a
  179. terminal to inspect applications, run commands, view screenshots, and save its work.
  180. You can give it your own task or run an official benchmark task with a score.
  181. ### Requirements and installation
  182. - **Python 3.12+** and a Responses-compatible model API key.
  183. - Docker and access to the gated official task and asset snapshots.
  184. ```bash
  185. pip install -e ".[desktop]"
  186. ```
  187. Before setup, accept access to the [tasks](https://huggingface.co/datasets/xlangai/osworld_v2_tasks)
  188. and [assets](https://huggingface.co/datasets/xlangai/osworld_v2_assets_gated) on
  189. Hugging Face, then log in:
  190. ```bash
  191. pip install huggingface_hub
  192. hf auth login
  193. cuawright setup osworld --root ~/.cache/cuawright/osworld-v2-2026.08.08
  194. cuawright verify osworld \
  195. --setup ~/.cache/cuawright/osworld-v2-2026.08.08/setup.json
  196. ```
  197. Setup downloads the pinned source, task classes, assets, and VM by default; existing
  198. resources can be registered without copying. Put the API key in a
  199. user-owned file with mode `0600`, outside source, task, asset, and result directories.
  200. Choose a new result directory whose parent already exists.
  201. ### Run an arbitrary desktop task
  202. ```bash
  203. cuawright desktop run \
  204. --setup /absolute/path/to/setup.json \
  205. --credentials /absolute/path/to/api-key \
  206. --results /absolute/path/to/results/custom \
  207. --model YOUR_MODEL \
  208. --instruction "<task>"
  209. ```
  210. This workflow has no evaluator or score. For an official scored task:
  211. ```bash
  212. cuawright desktop reproduce osworld \
  213. --setup /absolute/path/to/setup.json \
  214. --credentials /absolute/path/to/api-key \
  215. --results /absolute/path/to/results/task-001 \
  216. --task-id 001 --model YOUR_MODEL --reasoning xhigh
  217. ```
  218. For another Responses-compatible provider, pass its full API URL with
  219. `--responses-url`. See [task selection and proxies](REPRODUCE.md#choose-an-official-task)
  220. for official task IDs and `--proxy-config` setup.
  221. Each run saves its settings in `manifest.json`, the agent's actions in `trace.jsonl`,
  222. and the outcome in `result.json`. Images viewed by the agent are embedded in the
  223. trace; desktop runs do not export separate screenshot files. Official benchmark
  224. runs also include the evaluator score. Files created in the VM are not automatically
  225. copied to the host; see [custom tasks](REPRODUCE.md#smoke-and-custom-task).
  226. See [REPRODUCE.md](REPRODUCE.md) for setup, smoke tests, and reproduction commands,
  227. or [desktop setup and execution](docs/desktop.md) for more detail.
  228. ---
  229. ## πŸ”Œ Use as a Plugin
  230. CUAWright ships plugin manifests for both [Claude Code](https://docs.claude.com/en/docs/claude-code/plugins) ([`.claude-plugin/plugin.json`](.claude-plugin/plugin.json)) and [OpenAI Codex](https://developers.openai.com/codex/plugins) ([`.codex-plugin/plugin.json`](.codex-plugin/plugin.json)), with the shared skill at [`skills/cuawright-web/`](skills/cuawright-web/) and slash commands at [`skills/cuawright-web/commands/`](skills/cuawright-web/commands/). The host agent drives the Webwright loop natively β€” no extra LLM API key or cost beyond your host subscription. Hosts that read PNG screenshots natively skip the `image_qa` / `self_reflection` tools.
  231. The bundled `cuawright-web` skill handles browser tasks. Desktop tasks use
  232. `cuawright-desktop`; see [desktop setup](docs/desktop.md).
  233. Common runtime deps (install once after either path):
  234. ```bash
  235. pip install -e .
  236. playwright install chromium
  237. ```
  238. <details>
  239. <summary><b>Claude Code</b></summary>
  240. ### Install
  241. Install through the bundled marketplace inside Claude Code:
  242. ```text
  243. # 1. Add this repo as a Claude Code plugin marketplace
  244. /plugin marketplace add microsoft/Webwright
  245. # 2. Install the plugin from that marketplace
  246. /plugin install cuawright@cuawright
  247. ```
  248. Prefer a local checkout? Point the marketplace command at the cloned repo instead:
  249. ```text
  250. /plugin marketplace add /absolute/path/to/Webwright
  251. /plugin install cuawright@cuawright
  252. ```
  253. ### Use
  254. **Start a new Claude Code session** after installing β€” plugins are loaded at session start and won't appear until you restart.
  255. You can either ask Claude Code in plain English (the skill auto-activates from its description), or use one of the slash commands:
  256. ```
  257. /cuawright:run search Google Flights for flights from SEA to JFK on 2026-08-15 to 2026-08-20
  258. /cuawright:craft search a ticket on Google Flights from LAX to SFO depart June 7 return June 14
  259. ```
  260. - `/cuawright:run` (or any plain prompt) produces a **one-shot** `final_script.py` for the literal task values.
  261. - `/cuawright:craft` produces a **reusable CLI tool**: `final_script.py` becomes one parameterized function with a Google-style `Args:` docstring and an `argparse` wrapper whose flags default to the concrete task values, so you can rerun it later with different arguments β€” e.g. `python final_script.py --origin JFK --destination LAX --depart-date 2026-07-01`.
  262. In both modes Claude Code scaffolds a workspace with `plan.md`, runs instrumented Playwright scripts under `final_runs/run_<id>/`, and visually self-verifies each critical point against the saved screenshots.
  263. </details>
  264. <details>
  265. <summary><b>OpenAI Codex</b></summary>
  266. ### Install
  267. Codex reads Claude-style marketplaces, so the same repo works as a Codex plugin marketplace. From the Codex CLI:
  268. ```bash
  269. # 1. Add this repo as a Codex plugin marketplace
  270. codex plugin marketplace add microsoft/Webwright
  271. # 2. Open the plugin browser and install CUAWright
  272. codex
  273. /plugins
  274. ```
  275. Prefer a local checkout?
  276. ```bash
  277. codex plugin marketplace add /absolute/path/to/Webwright
  278. ```
  279. Then restart Codex so the new marketplace and plugin are picked up.
  280. ### Use
  281. In a new Codex thread, either ask in plain English (the skill auto-activates from its description) or invoke the bundled skill explicitly with `@cuawright-web`:
  282. ```
  283. @cuawright-web search Google Flights for flights from SEA to JFK on 2026-08-15 to 2026-08-20
  284. ```
  285. Codex scaffolds a workspace with `plan.md`, runs instrumented Playwright scripts under `final_runs/run_<id>/`, and visually self-verifies each critical point against the saved screenshots.
  286. To turn the plugin off without uninstalling, set its entry in `~/.codex/config.toml` to `enabled = false` and restart Codex.
  287. </details>
  288. <details>
  289. <summary><b>Hermes Agent</b></summary>
  290. ### Install
  291. [Hermes Agent](https://github.com/NousResearch/hermes-agent) is a [skills-compatible client](https://agentskills.io), so the same `skills/cuawright-web/` folder loads as a Hermes skill. Symlink it into your Hermes user-skills directory:
  292. ```bash
  293. mkdir -p ~/.hermes/skills
  294. ln -sfn /absolute/path/to/Webwright/skills/cuawright-web ~/.hermes/skills/cuawright-web
  295. ```
  296. No Hermes-specific manifest is needed; only `SKILL.md` is loaded.
  297. ### Use
  298. Start Hermes (`hermes`) and ask it to drive a web task in natural language β€” the skill auto-activates from its description. You can also invoke it explicitly with `/cuawright-web`.
  299. Note: the named subcommands shipped under [`skills/cuawright-web/commands/`](skills/cuawright-web/commands/) (`/cuawright:run`, `/cuawright:craft`) are a Claude Code / Codex convention and are inert in Hermes; the skill itself still works end-to-end.
  300. </details>
  301. <a id="-task-showcase-repeatable-runs-as-a-dashboard"></a>
  302. <details>
  303. <summary><b>πŸ“° Task Showcase (repeatable runs as a dashboard)</b></summary>
  304. A tiny Flask app under [`assets/task_showcase/`](assets/task_showcase/README.md) consolidates
  305. Webwright runs for **repeatable** odyssey tasks (deals, inventory, listings,
  306. job boards, weather, etc.) into a single dashboard. Each task ships only two
  307. files β€” `task.json` (metadata) and `report.json` (curated, structured output:
  308. sources + result sections like tables, lists, summaries) β€” and the templates
  309. render them generically, so adding a new task is just dropping a new folder
  310. in `assets/task_showcase/tasks/`.
  311. ```bash
  312. pip install flask
  313. python assets/task_showcase/app.py # http://127.0.0.1:5005
  314. ```
  315. To have Webwright produce a renderer-ready task folder at runtime, stack the
  316. Task Showcase overlay:
  317. ```bash
  318. python -m cuawright.webwright.run.cli main \
  319. -c base.yaml -c model_openai.yaml -c task_showcase.yaml \
  320. -t "<repeatable web task>" \
  321. --task-id my_repeatable_task \
  322. -o outputs/default
  323. ```
  324. > **Note:** `report.json` is only generated when `-c task_showcase.yaml` is
  325. > included. A plain `base.yaml` run produces `trajectory.json` and debug
  326. > artifacts but no `report.json`.
  327. The run writes `task_showcase/tasks/<short_id>/task.json` and `report.json`
  328. inside the output workspace. Render those generated files without copying them
  329. back into the repo:
  330. ```bash
  331. python assets/task_showcase/app.py \
  332. --tasks-dir outputs/default/<run>/task_showcase/tasks
  333. ```
  334. </details>
  335. <a id="-skill-factory-turn-solved-tasks-into-runnable-code-skills"></a>
  336. <details>
  337. <summary><b>🧠 Skill Factory (turn solved tasks into runnable code skills)</b></summary>
  338. Skill Factory is an optional extension for learning and reusing browser automation
  339. scripts. Install it from this checkout:
  340. ```bash
  341. pip install -e ".[skill-factory]" -e extensions/skill-factory
  342. cuawright-skill-factory --help
  343. ```
  344. For desktop execution and Skill Factory together, use
  345. `pip install -e ".[desktop,skill-factory]" -e extensions/skill-factory`.
  346. **Most agent skills are context the model reads. Ours are programs.**
  347. [`cuawright.webwright.skill_factory`](extensions/skill-factory/src/cuawright/webwright/skill_factory/) distills the script every solve leaves
  348. behind into a growing library of **reusable, verified, parameterized skills** β€” code you can run
  349. without a model and compose into the next task instead of re-exploring the site. Plugs in with
  350. **no change to the agent loop**:
  351. - **Reuse** β€” before a solve starts, the library is checked *out of the agent loop* (`recommend`):
  352. `route` either runs a matching skill directly (no model) or injects it into the prompt as a prior
  353. (`{verdict: run|adapt|skip, skill_id, source_path}`); the agent reuses the hint without ever
  354. querying the library itself.
  355. - **Grow** β€” afterwards, `python -m cuawright.webwright.skill_factory learn outputs/ --library ./library`
  356. groups solves of the same template and distills one parameterized skill (`build` does solve→learn
  357. in one shot; `update` is manual-manifest mode).
  358. Before adding a skill, the input gate compares the result with gold answers when
  359. available. Otherwise it checks the output's shape and the agent's reported success;
  360. this fallback can admit an incorrect answer. The generated script is then replayed
  361. without a model, and updates are checked against previous examples.
  362. Once learned, a skill **runs standalone in ~40 s with zero tokens**. On WebArena (10 retrieve-type
  363. templates, 3 self-hosted sites, gpt-5.4) reuse lifts held-out accuracy **55% β†’ 70% (+15 pp)** while
  364. cutting steps. See [`extensions/skill-factory/src/cuawright/webwright/skill_factory/README.md`](extensions/skill-factory/src/cuawright/webwright/skill_factory/README.md).
  365. </details>
  366. <a id="-trajectory-comparison--viewer"></a>
  367. <details>
  368. <summary><b>πŸ“ƒ Trajectory Comparison &amp; Viewer</b></summary>
  369. You can run the same tasks using the Webwright harness and its Codex / GitHub Copilot skill variant, and see how token usage and trajectories stack up between different harnesses. The trajectory viewer supports Codex, GitHub Copilot and Webwright harness traces.
  370. ![Trajectory comparison](assets/trajectory-compare.png)
  371. ### How to use
  372. ```bash
  373. cd assets/compare_trajectory/
  374. python3 -m http.server
  375. ```
  376. Open the webpage in your browser and upload the Webwright `raw_responses.jsonl` and attach `trajectory.json` to view. Then on the other side you can upload your Codex or GitHub Copilot trace.
  377. ### Obtaining Codex traces:
  378. ```
  379. ls ~/.codex/sessions/2026/MONTH/DAY/SESSION_ID.jsonl
  380. ```
  381. ### Obtaining GitHub Copilot traces:
  382. ```
  383. /export file session
  384. -> session.md is the uploadable trace
  385. ```
  386. ### Quick Comparison
  387. #### "Find the cheapest used 8-cylinder bmw made between 2005-2015 and priced from 25,000 to 50,000 dollars with mileage less than 50,000 miles or less."
  388. | Tokens | Webwright Harness (Local Browser Mode) | Codex Webwright Skill |
  389. | --- | ---: | ---: |
  390. | Input | 420,433 | 3,271,143 |
  391. | Output | 3,593 | 20,040 |
  392. | Reasoning | 0 | 4,410 |
  393. | Cached | 217,216 | β€” |
  394. | Total | 424,026 | 3,291,183 |
  395. The Codex cached-token count is omitted because the recorded figure needs verification.
  396. Individual runs and results may vary.
  397. ---
  398. </details>
  399. ## Credits
  400. - [SWE-agent/mini-swe-agent](https://github.com/SWE-agent/mini-swe-agent/tree/main) β€” design inspiration for the minimal agent loop.
  401. - [Playwright](https://playwright.dev/) β€” browser automation.
  402. ## Citation
  403. If you use CUAWright or Webwright in your research or build on it, please cite:
  404. ```bibtex
  405. @misc{lu2026cuawright,
  406. title = {CUAWright: A Minimal Unified Interface for Digital Agents},
  407. author = {Lu, Yadong and Lee, Theodore and Li, Yifei and Jang, Lawrence Keunho and Xue, Tianci and Su, Yu and Sun, Huan and Awadallah, Ahmed Hassan},
  408. year = {2026},
  409. eprint = {2610.04116},
  410. archivePrefix = {arXiv},
  411. primaryClass = {cs.AI},
  412. url = {https://arxiv.org/abs/2610.04116}
  413. }
  414. @misc{webwright2026,
  415. title = {Webwright: A terminal is all you need for web agents},
  416. author = {Lu, Yadong and Xu, Lingrui and Huang, Chao and Awadallah, Ahmed},
  417. year = {2026},
  418. howpublished = {\url{https://github.com/microsoft/Webwright}},
  419. note = {GitHub repository}
  420. }
  421. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.