Skip to content
Back to search

ghostdriver

MCP server that drives desktop applications without stealing the desktop from the person using it

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/steiner385--ghostdriver/badge.svg)](/listings/steiner385--ghostdriver)

HTML

<a href="/listings/steiner385--ghostdriver"><img src="/api/listings/steiner385--ghostdriver/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Python 3.11 or newer. Install a tagged release straight from GitHub; pip needs no GitHub account and no git:

pip install "ghostdriver[server] @ https://github.com/steiner385/ghostdriver/archive/refs/tags/v0.2.0.zip"
ghostdriver-mcp          # the MCP server, on stdio
ghostdriver --help       # the reconnaissance CLI

Do this inside a virtual environment (`python -m venv .venv`) so the console scripts land somewhere you can name in a client config. On Windows, pip also pulls in `pywin32`, `comtypes` and `pywinauto`; on Linux, see the system packages below.

For contributors

git clone https://github.com/steiner385/ghostdriver
cd ghostdriver
pip install -e ".[dev,server]"
pytest

On Linux, the backend reaches AT-SPI through PyGObject, and pip cannot install that. It comes from the system. Install the typelibs, then build the virtual environment with `--system-site-packages` so it can see `gi`:

sudo apt install python3-gi gir1.2-atspi-2.0 gir1.2-gtk-4.0 at-spi2-core xclip xvfb
python3 -m venv --system-site-packages .venv

Without the typelibs, the AT-SPI tests do not fail. They skip, and the run still comes out green. GTK 4 is for the test fixture; the backend itself does not need it.

The quick gate for everyday changes leaves out the tests that launch an engine, a hidden desktop or a server, or wait on a watchdog:

pytest -m "not slow and not clipboard"

Run the whole suite before a push, or after touching a backend, the session gate or the actuation ladder. Both include the lint and type tests and the focus guard.

That one command is the whole gate: the suite includes a test that runs `ruff check src tests` and another that runs `mypy`, both configured in `pyproject.toml`, so lint and type regressions fail the tests rather than waiting for someone to remember a second command.

The `server` extra pulls in the MCP SDK. The CLI and the library work without it.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 405 lines
  1. # ghostdriver
  2. An MCP server for driving desktop applications — for automated testing and
  3. unattended tasks — without contending with a human working on the same
  4. machine.
  5. ## Status
  6. **Usable on Windows, and on Linux under X11.** Twenty-three tools over a session manager, in three
  7. capability tiers (`read_only`, `interactive`, `full`), with session-scoped
  8. consent, a denylist that names controls by caption or by identity, a
  9. tiered actuation ladder that never activates a window, two isolation
  10. strategies (`minimized`, with a watchdog; `hidden_desktop`, launch-only),
  11. locators for windows and for elements that are not windows, embedded web
  12. documents driven through their accessibility tree, and an operator
  13. directory the tool reads profiles and workflows from and refuses to read
  14. from any repository. The Linux backend wires everything except
  15. assessment, so `surfaces` and `probe` refuse there (see *Platforms*).
  16. See `docs/superpowers/specs/2026-09-01-desktop-automation-mcp-design.md` for
  17. the design, `docs/DECISIONS.md` for what was decided along the way and why,
  18. and `docs/VALIDATION.md` for the Iteration 1 runbook.
  19. ## Why
  20. Most desktop automation assumes UI Automation works. For a large class of
  21. enterprise line-of-business software it does not: applications built on
  22. pre-UIA toolkits expose a tree in which every element is a `Pane` and no
  23. element supports any pattern. Those applications remain drivable through
  24. window messages, and that tier is what this tool implements.
  25. Validation against a real target found the picture is not uniform even
  26. within one application. Its native shell exposed 29 accessibility elements
  27. with 2 usable patterns; an embedded web view inside the same window exposed
  28. 231 with 92. Which mechanism reads a region is a per-region question, so the
  29. tool answers it per region.
  30. ## Install
  31. Python 3.11 or newer. Install a tagged release straight from GitHub; pip
  32. needs no GitHub account and no git:
  33. ```bash
  34. pip install "ghostdriver[server] @ https://github.com/steiner385/ghostdriver/archive/refs/tags/v0.2.0.zip"
  35. ghostdriver-mcp # the MCP server, on stdio
  36. ghostdriver --help # the reconnaissance CLI
  37. ```
  38. Do this inside a virtual environment (`python -m venv .venv`) so the
  39. console scripts land somewhere you can name in a client config. On Windows,
  40. pip also pulls in `pywin32`, `comtypes` and `pywinauto`; on Linux, see the
  41. system packages below.
  42. ### For contributors
  43. ```bash
  44. git clone https://github.com/steiner385/ghostdriver
  45. cd ghostdriver
  46. pip install -e ".[dev,server]"
  47. pytest
  48. ```
  49. On Linux, the backend reaches AT-SPI through PyGObject, and pip cannot
  50. install that. It comes from the system. Install the typelibs, then build
  51. the virtual environment with `--system-site-packages` so it can see `gi`:
  52. ```bash
  53. sudo apt install python3-gi gir1.2-atspi-2.0 gir1.2-gtk-4.0 at-spi2-core xclip xvfb
  54. python3 -m venv --system-site-packages .venv
  55. ```
  56. Without the typelibs, the AT-SPI tests do not fail. They skip, and the
  57. run still comes out green. GTK 4 is for the test fixture; the backend
  58. itself does not need it.
  59. The quick gate for everyday changes leaves out the tests that launch an
  60. engine, a hidden desktop or a server, or wait on a watchdog:
  61. ```bash
  62. pytest -m "not slow and not clipboard"
  63. ```
  64. Run the whole suite before a push, or after touching a backend, the
  65. session gate or the actuation ladder. Both include the lint and type
  66. tests and the focus guard.
  67. That one command is the whole gate: the suite includes a test that runs
  68. `ruff check src tests` and another that runs `mypy`, both configured in
  69. `pyproject.toml`, so lint and type regressions fail the tests rather than
  70. waiting for someone to remember a second command.
  71. The `server` extra pulls in the MCP SDK. The CLI and the library work
  72. without it.
  73. ## Use — MCP
  74. ```bash
  75. ghostdriver-mcp # stdio transport
  76. ```
  77. Register it with a Claude client as a stdio server, with the operator
  78. directory in its environment. For Claude Code (and the desktop app's Code
  79. tab), from any directory:
  80. ```bash
  81. claude mcp add --scope user --env GHOSTDRIVER_OPERATOR_DIR=C:\Users\you\ghostdriver-operator ghostdriver -- C:\path\to\ghostdriver-mcp\.venv\Scripts\ghostdriver-mcp.exe
  82. ```
  83. The same server in JSON form, for a `.mcp.json` or the desktop chat
  84. app's `claude_desktop_config.json`:
  85. ```json
  86. {
  87. "mcpServers": {
  88. "ghostdriver": {
  89. "command": "C:\\path\\to\\ghostdriver-mcp\\.venv\\Scripts\\ghostdriver-mcp.exe",
  90. "env": { "GHOSTDRIVER_OPERATOR_DIR": "C:\\Users\\you\\ghostdriver-operator" }
  91. }
  92. }
  93. }
  94. ```
  95. Use the virtual environment's console script, not a bare `python`: the
  96. server needs the platform's bindings installed there: COM and Win32 on
  97. Windows, `gi` and `python-xlib` on Linux. On Linux the same registration
  98. reads:
  99. ```bash
  100. claude mcp add --scope user --env GHOSTDRIVER_OPERATOR_DIR=$HOME/ghostdriver-operator ghostdriver -- /path/to/ghostdriver-mcp/.venv/bin/ghostdriver-mcp
  101. ```
  102. The server must run inside the person's graphical session, with `DISPLAY`
  103. and the session's D-Bus address set. From a bare SSH shell it finds no
  104. accessibility bus and no display. With the
  105. operator directory configured, `attach` accepts a profile by bare name
  106. (`profile_path="sandbox"` finds `profiles/sandbox.json` there), so no
  107. path that names a machine crosses the wire.
  108. Fifteen tools at the `read_only` tier: `list_targets`, `attach`,
  109. `release`, `grant_targets`, `grant_launch`, `describe`, `find`, `read`,
  110. `capture`, `wait_for`, `probe`, `surfaces`, `modals`, `restore`,
  111. `await_human`. `interactive` adds `click`, `set_value`, `type_text`,
  112. `send_keys` and `record_input_sensitivity`; `full` adds `probe_phase2`,
  113. `probe_grid` and `launch_isolated`. The tier is chosen at `attach` and is
  114. immutable for the session.
  115. **A value gets there whole, or the tool says it did not.** `set_value`
  116. types first, reads the field back, and if characters went missing pastes
  117. the value whole as the next rung; the result names the `route` that
  118. delivered. `send_keys` takes the editing actions `select-all`, `paste`,
  119. `copy`, `cut` and `undo` -- what Ctrl+A/V/C/X/Z become inside an edit
  120. control, since a posted keystroke carries no modifier state. `capture`
  121. renders the window that holds what you named, and says its frame.
  122. **Consent lasts as long as the application.** With an operator directory
  123. configured, a decision about a target is kept for as long as that process
  124. runs -- across relaunches of the operator app and redeploys of this server
  125. -- in `consent/` under the directory, fingerprinted by process start time.
  126. A restriction is kept the same way. Without an operator directory a
  127. decision lives for one server run.
  128. **A standing approval, if you want one, is declared by hand.** An
  129. approval still ends when the target itself restarts -- that is what
  130. "that application, running" means. For an executable you have already
  131. decided about for this machine, write
  132. `standing-consent.json` in the operator directory:
  133. ```json
  134. {
  135. "allow": [{"path": "C:\\Program Files\\Vendor\\VendorApp.exe",
  136. "note": "the line-of-business client"}],
  137. "restrict": ["C:\\Windows\\System32\\mmc.exe"]
  138. }
  139. ```
  140. **This server reads that file and never writes it, and no tool can** --
  141. that is the only thing separating an approval you gave from one an agent
  142. would like you to have given (D#125). It answers *last*: a decision about
  143. a running pid, or about its parent, always wins, so a declaration cannot
  144. un-restrict something you ruled out a minute ago. A target approved this
  145. way says so wherever its grant is shown, because nobody was asked about
  146. it. Matching is on the executable path alone, so a declaration covers
  147. every instance of that program -- including one pointed somewhere you
  148. would not have approved. The path to copy is in the `needs_consent`
  149. result as `target.exe`.
  150. **Developing it: hot reload.** Set `GHOSTDRIVER_DEV=1` in the server's
  151. environment and an edit to `tools/` or `backends/` takes effect on the next
  152. tool call, with open sessions kept. The next result says what reloaded.
  153. Edits to `session`, `policy`, `consent`, `model` or the server itself are
  154. reported as needing a reconnect (`/mcp` in Claude Code); a new tool or
  155. parameter always does, since the host caches the tool list at connect.
  156. An edit that fails to import keeps the previous generation running.
  157. **A click can name the surface changing.** When no control's presence
  158. proves an action worked -- an owner-drawn bar whose click docks a
  159. palette seconds later -- `expect={"changed": true}` verifies against the
  160. surface shape instead, and tells "delivered and something changed" from
  161. "delivered and nothing did". The delivered-inactive caveat is dropped
  162. when the click's own disturbances show a window opening or closing.
  163. **A window can be brought forward, on request.** `foreground` is the
  164. deliberate pair to `isolation=minimized`: everything works from the
  165. background, but when a step needs the window frontmost -- handing the
  166. click to a tool that uses real input for a control that ignores posted
  167. messages -- this raises it, reports whether it took the foreground, and
  168. warns that it interrupts the operator. Never automatic.
  169. **The watches follow the process tree.** A session's consent covers the target's helper processes, and so do its watches: under `isolation=minimized`, a helper's window coming on screen is caught, photographed and put away, and a workflow's interference watch reports a helper taking the foreground as the target taking it.
  170. **The clipboard is the person's.** Setting text never touches it: the whole-value rung inserts through the control's own edit path. The one route that must write it, a grid's own Copy, runs only in a session attached with `clipboard=true`, and puts every format back afterwards.
  171. **The effect may arrive after the action.** A profile can name how the target shows it is busy -- `busy_indicator`, a locator present while busy or a region that renders ink while busy. Every actuation result then says `still_arriving` while it shows, and `wait_idle` waits for it to clear before you read a grid that is still filling.
  172. **A result does not say ok for nothing.** A click delivered to a window
  173. that is not active carries a `warning` and reads `delivered_inactive`,
  174. since a control that checks focus would ignore it; `require_active` refuses
  175. rather than delivering blind. `await_human` refuses a hand-off with no
  176. condition and takes `until_change` to watch the surface move when you
  177. cannot name what will appear. `hit_targets` lists a container's child
  178. windows so a point click aims at a real target.
  179. **A click goes to the window under the pointer.** A click posted to a panel over a windowed child is dropped by the panel; this tool posts to the innermost window under the point and says so (`delivered_to`). A control that answers empty and renders content is flagged `read_unreliable`; a window the system reports hung is reported hung; the notice a person sees during `await_human` has a Later that returns the wait as `deferred`.
  180. **Context menus are readable.** A popup menu's items come from the menu
  181. itself: `find` lists them with rectangles and captions, an owner-drawn
  182. item by position, and `click` takes either. `read_clipboard` reads back
  183. what a target's own Copy command put there.
  184. **One server, several chats.** The operator app runs one server process
  185. for every chat, so approvals and restrictions are shared across them. A
  186. restriction recorded in one chat stops that target's sessions in all of
  187. them; a target nobody allowed is never automated, so leaving one alone
  188. needs no record. A refused session says which decision refused it and
  189. that `allow=[pid]` reverses it.
  190. **When it needs a person, it says so on screen.** `await_human` shows its
  191. ask in a small always-on-top notice in the corner of the operator's screen
  192. for as long as it waits -- the one window this tool ever shows, and it
  193. never takes focus. `click` takes `button` (left, right, middle) and
  194. `modifiers` (ctrl, shift), so a Ctrl+right-click that opens a menu is the
  195. tool's to do rather than the person's.
  196. **Consent is decided in the chat, not in configuration.** `list_targets`
  197. shows each candidate with the endpoint read from its own command line —
  198. `the client (pid 62088) -- http://devhost:8183/devenv/gateway` — so the
  199. person can see what they are approving. `attach` then asks them directly if
  200. their client supports elicitation; otherwise it returns `needs_consent` and
  201. the agent asks in chat and records the answer with `grant_targets`.
  202. **How long an approval lasts depends on the configuration, so the tool
  203. says which case it is in rather than leaving anyone to assume** (#62):
  204. the elicitation prompt and every `needs_consent` result carry the same
  205. `lifetime` sentence, naming either "while that process keeps running,
  206. across restarts of this server" or "this server run only" with the
  207. environment variable that changes it. A decision can be revised at any
  208. time, and revoking a target stops a session already using it. A decision
  209. covers the application's whole process tree: the file dialog a browser
  210. opens in a helper process is the application the operator approved, not a
  211. new question, and a session sees that dialog's windows with
  212. `all_windows=true` without a second attach. A profile
  213. document is optional (every key is described in `docs/PROFILE.md`, and
  214. `ghostdriver check-profile <name>` reads one back); its denylist names controls by caption or identity,
  215. and, since D72, by *place*: `{"within": locator, "region": {left, top,
  216. right, bottom}, "note"}` refuses any actuation aimed inside that rectangle
  217. of the named window -- a grid cell, a painted button.
  218. ## Use — CLI
  219. ```bash
  220. ghostdriver list-targets
  221. ghostdriver probe --pid 1234 --out profile.json
  222. ghostdriver describe --pid 1234 --depth 5
  223. ghostdriver describe --pid 1234 --depth 5 --all-windows
  224. ghostdriver find --pid 1234 --locator 'class=VendorButton text="Expand All"'
  225. ghostdriver find --pid 1234 --locator 'type=TabItem' # elements that are not windows
  226. ghostdriver find --pid 1234 --locator 'type=MenuItem index=1 within=(class=ToolbarWindow32)'
  227. ghostdriver find --pid 1234 --locator 'type=Button name="Save" within=(type=Document)' # inside an embedded web document
  228. ghostdriver read --pid 1234 --locator 'class=VendorEdit visible=true'
  229. ghostdriver surfaces --pid 1234
  230. ghostdriver modals --pid 1234
  231. ghostdriver restore --pid 1234
  232. ```
  233. ## The operator directory
  234. Everything that names a target -- profile documents with real endpoints,
  235. curated denylists, workflow files, run reports -- lives in one directory
  236. outside any repository, and the tool reads such things from there and
  237. nowhere else:
  238. ```bash
  239. set GHOSTDRIVER_OPERATOR_DIR=C:\Users\you\ghostdriver-operator
  240. ghostdriver list-workflows
  241. ghostdriver run-workflow walk-tabs --pid 1234
  242. ```
  243. The directory holds `profiles/`, `workflows/`, `reports/`, `consent/`,
  244. the optional `standing-consent.json` described above, and `logs/`,
  245. where every session's server writes a trace of its tool calls (one JSON
  246. line per call: session, tool, arguments, outcome, elapsed; pixels left
  247. out). `ghostdriver trace` prints the recent ones, so what another session
  248. did can be read after the fact; `GHOSTDRIVER_TRACE=0` turns it off. A path
  249. inside a git checkout is refused, and so is a packaged application's
  250. virtualized AppData (a process inside one cannot write anywhere other
  251. processes will see). A workflow is a JSON list of tool calls with
  252. expectations; it names no target -- the pid is given at run time -- and
  253. attaches at the tier and isolation it declares. The runner stops at the
  254. first failed step, fails the run if the target ever takes the foreground,
  255. and writes a report of every step and event back to `reports/`.
  256. ## Platforms
  257. There are two backends, and `ghostdriver.backends.current()` picks one by
  258. `sys.platform`. Everything above that boundary is platform-neutral, and a
  259. test keeps it that way.
  260. **Windows** is complete. It reads the tree with window messages and UI
  261. Automation, acts by posted messages, and isolates by minimizing the target
  262. or by putting it on a hidden desktop.
  263. **Linux** targets X11. It reads the tree and acts through AT-SPI, reads
  264. window-manager state through EWMH, and gets a hidden desktop from a headless
  265. GNOME Shell that has its own D-Bus and AT-SPI buses. `docs/LINUX.md` lists each
  266. contract name with its Linux mechanism and the decision that measured it.
  267. The Linux backend differs from Windows in these ways:
  268. - **No tier 3.** GTK drops synthetic X events (D#11). An action that needs
  269. real input is tier 4, sent with XTest, and only under the live-actuation
  270. consent (D#111).
  271. - **No assessment.** `region_evidence`, `resolve_hosted_app`, `assess_uia`
  272. and `assess_msaa` are not wired yet, so `surfaces` and `probe` return a
  273. `denied_by_environment` error that names what is missing. The embedded-document case has been measured
  274. (D#15); the wiring has not been done.
  275. - **On a Wayland desktop, reading and tier-1 acting work.** Window state
  276. comes from AT-SPI there, not X (D#221): focus from `STATE_ACTIVE`, modals
  277. from `STATE_MODAL`. So `attach`, `describe`, `find`, `read`, `click` and
  278. `set_value` work, with focus left where the operator had it, which was
  279. measured on GNOME 50. Macro replay and workflow runs work too: their
  280. watch reads focus from AT-SPI, and their result says that pointer
  281. movement was not watched (D#225). Still refused there: `capture`,
  282. `restore`, the minimized strategy and live actuation, because Wayland
  283. cannot report minimized state or the pointer (D#215, D#221). A modal blocks the whole
  284. application, since Wayland does not say which window it holds. The plan
  285. for the rest is `docs/superpowers/plans/2026-10-09-wayland-support.md`.
  286. - **The hidden desktop is a headless GNOME Shell.** GNOME 50 removed
  287. `gnome-shell --x11`, so the nested desktop runs `gnome-shell --headless
  288. --wayland` and the target is an X client of its Xwayland (D#217). It needs
  289. `gnome-shell`, `dbus-daemon` and `dbus-send`; it no longer starts an
  290. Xvfb (some measurement tests still do).
  291. ## What it does not do
  292. - **Un-minimize anything on its own.** Measured: every way of restoring a
  293. minimized window from a process allowed to set the foreground -- which any
  294. process launched from the operator's own application is -- brings it to
  295. the front. So a session under `isolation="minimized"` leaves the target
  296. minimized at release and says so, and `restore` reports `took_foreground`
  297. honestly rather than promising it stayed put. The operator clicks the
  298. taskbar.
  299. - **Launch onto a hidden desktop anything but a classic Win32 process.**
  300. Store-activated applications are activated by the shell on the input
  301. desktop and ignore the desktop they were launched onto. And a target that
  302. needs a person to sign in cannot use a hidden desktop at all; `minimized`
  303. is the strategy for a target that is already running.
  304. - **Give exact text out of an owner-drawn grid by itself.** `probe_grid`
  305. finds which route does -- a focus-revealed accessibility surface, or the
  306. control's own clipboard copy -- and records it; `capture` remains the way
  307. to look.
  308. - **Put a value into an owner-drawn grid cell.** A grid that edits through
  309. a window -- an edit appearing under it, or a popup -- hands `set_value` an
  310. ordinary control, and that case needs nothing new. A grid that paints its
  311. editing the way it paints its rows offers no window to address, and there
  312. is no primitive here that sets such a cell. Which one a grid is has not
  313. been measured on a real target; the one field report is of a cell that
  314. took neither a posted key nor a real one. So the tool says how to tell
  315. the two apart (post `f2` or a double-click, then `find` again) instead of
  316. promising the first, and says plainly what the second means: the value
  317. goes in through the application's own interface, or a person does it
  318. (D#199).
  319. - **No role mapping.** `Locator` carries a `role` field for profiles to
  320. populate later, but nothing resolves it, so `Locator.parse` rejects
  321. `role=` outright rather than accepting a query that would match nothing.
  322. ## Design notes
  323. - **Locators, not handles.** The OS recycles window handles and voids them
  324. when a process exits, so elements are re-resolved from a declarative
  325. query on every call. Results still report a handle, marked ephemeral,
  326. because it is the only thing that distinguishes two identically-captioned
  327. controls within a single result.
  328. - **Input-shaped messages, not semantic ones.** Semantic messages such as
  329. `BM_CLICK` are implemented by a specific window class and are discarded
  330. by anything else. Input-shaped messages only assume the toolkit handles
  331. mouse and keyboard input.
  332. - **Pure logic separated from platform calls.** The Windows backend
  333. produces plain `WindowNode` trees; matching, summarization, inference,
  334. policy and waiting operate on those and are testable on any platform.
  335. - **The protocol layer holds no decisions.** the `tools` package returns plain data
  336. and carries every rule; `mcp_server.py` is an adapter over it. Anything
  337. that can be wrong is wrong somewhere testable without a protocol client.
  338. - **Menus, dropdowns and dialogs are separate top-level windows.** On a
  339. measured target, 48% of the process's windows sat outside its main
  340. window. Tools default to the main window and say so; `all_windows` reaches
  341. the rest.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.