Skip to content
Back to search

FlaUI-MCP

MCP server for Windows desktop automation using FlaUI and UI Automation APIs

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/shanselman--flaui-mcp/badge.svg)](/listings/shanselman--flaui-mcp)

HTML

<a href="/listings/shanselman--flaui-mcp"><img src="/api/listings/shanselman--flaui-mcp/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Prerequisites

- Windows 10/11 - .NET 8.0 Runtime

Download Release

Download the latest release from [Releases](https://github.com/shanselman/FlaUI-MCP/releases) and extract to a folder.

Choose the ZIP that matches your machine:

| Asset | Use when | |-------|----------| | `FlaUI-MCP-win-x64-*-self-contained.zip` | 64-bit Windows, no .NET runtime required | | `FlaUI-MCP-win-x64-*.zip` | 64-bit Windows with .NET 8 Runtime already installed | | `FlaUI-MCP-win-arm64-*-self-contained.zip` | Windows on ARM64, no .NET runtime required | | `FlaUI-MCP-win-arm64-*.zip` | Windows on ARM64 with .NET 8 Runtime already installed |

Configure MCP Client

Add to your MCP configuration (e.g., `~/.copilot/mcp-config.json`):

{
  "mcpServers": {
    "windows": {
      "type": "local",
      "command": "C:\\path\\to\\FlaUI-MCP.exe",
      "tools": ["*"]
    }
  }
}

Or using `dotnet run`:

{
  "mcpServers": {
    "windows": {
      "type": "local",
      "command": "dotnet",
      "args": ["run", "--project", "C:\\path\\to\\src\\FlaUI.Mcp"]
    }
  }
}

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 293 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # FlaUI-MCP
  2. An MCP (Model Context Protocol) server that enables AI agents to automate Windows desktop applications using accessibility APIs - the same way Playwright automates browsers.
  3. [![Build](https://github.com/shanselman/FlaUI-MCP/actions/workflows/build.yml/badge.svg)](https://github.com/shanselman/FlaUI-MCP/actions/workflows/build.yml)
  4. [![GitHub release](https://img.shields.io/github/v/release/shanselman/FlaUI-MCP)](https://github.com/shanselman/FlaUI-MCP/releases)
  5. [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
  6. ## Why This Exists
  7. When Playwright's MCP server automates browsers, it provides:
  8. - `browser_snapshot` → Structured accessibility tree with element refs
  9. - `browser_click ref="..."` → Click by ref, not coordinates
  10. **FlaUI-MCP brings the same pattern to Windows desktop apps:**
  11. - `windows_snapshot` → Accessibility tree with refs like `w1e5`
  12. - `windows_click ref="w1e5"` → Click element by ref
  13. No screenshot parsing. No coordinate guessing. Just semantic element references.
  14. ## Quick Demo
  15. ```
  16. Agent: Calculate 3 × 3
  17. 1. windows_launch { "app": "calc.exe" }
  18. → Window handle: w1
  19. 2. windows_snapshot { "handle": "w1" }
  20. → - window "Calculator" [ref=w1]
  21. - button "Three" [ref=w1e43]
  22. - button "Multiply by" [ref=w1e35]
  23. - button "Equals" [ref=w1e38]
  24. - text "Display is 0" [ref=w1e15]
  25. 3. windows_batch { "actions": [
  26. {"action": "click", "ref": "w1e43"},
  27. {"action": "click", "ref": "w1e35"},
  28. {"action": "click", "ref": "w1e43"},
  29. {"action": "click", "ref": "w1e38"},
  30. {"action": "snapshot", "handle": "w1"}
  31. ]}
  32. → 1. click: Invoked Three
  33. 2. click: Invoked Multiply by
  34. 3. click: Invoked Three
  35. 4. click: Invoked Equals
  36. 5. snapshot: ... "Display is 9" ...
  37. ```
  38. ## Installation
  39. ### Prerequisites
  40. - Windows 10/11
  41. - .NET 8.0 Runtime
  42. ### Download Release
  43. Download the latest release from [Releases](https://github.com/shanselman/FlaUI-MCP/releases) and extract to a folder.
  44. Choose the ZIP that matches your machine:
  45. | Asset | Use when |
  46. |-------|----------|
  47. | `FlaUI-MCP-win-x64-*-self-contained.zip` | 64-bit Windows, no .NET runtime required |
  48. | `FlaUI-MCP-win-x64-*.zip` | 64-bit Windows with .NET 8 Runtime already installed |
  49. | `FlaUI-MCP-win-arm64-*-self-contained.zip` | Windows on ARM64, no .NET runtime required |
  50. | `FlaUI-MCP-win-arm64-*.zip` | Windows on ARM64 with .NET 8 Runtime already installed |
  51. ### Configure MCP Client
  52. Add to your MCP configuration (e.g., `~/.copilot/mcp-config.json`):
  53. ```json
  54. {
  55. "mcpServers": {
  56. "windows": {
  57. "type": "local",
  58. "command": "C:\\path\\to\\FlaUI-MCP.exe",
  59. "tools": ["*"]
  60. }
  61. }
  62. }
  63. ```
  64. Or using `dotnet run`:
  65. ```json
  66. {
  67. "mcpServers": {
  68. "windows": {
  69. "type": "local",
  70. "command": "dotnet",
  71. "args": ["run", "--project", "C:\\path\\to\\src\\FlaUI.Mcp"]
  72. }
  73. }
  74. }
  75. ```
  76. ## Available Tools
  77. | Tool | Description |
  78. |------|-------------|
  79. | `windows_launch` | Launch a Windows application |
  80. | `windows_snapshot` | Get accessibility tree with element refs |
  81. | `windows_click` | Click an element by ref |
  82. | `windows_type` | Type text into an element |
  83. | `windows_send_keys` | Send key presses or key chords (for example `Ctrl+A`) |
  84. | `windows_fill` | Clear and fill a text field |
  85. | `windows_get_text` | Get text content of an element |
  86. | `windows_screenshot` | Capture window/element as PNG |
  87. | `windows_list_windows` | List all open windows |
  88. | `windows_focus` | Bring a window to foreground |
  89. | `windows_close` | Close a window |
  90. | `windows_batch` | Execute multiple actions in one call |
  91. `windows_snapshot` accepts an optional `maxDepth` argument to include more or
  92. fewer levels of the accessibility tree. It defaults to `10`; `windows_batch`
  93. snapshot actions accept the same argument.
  94. `windows_screenshot` supports an optional `background: true` argument when a
  95. window `handle` is provided. This uses native background capture when available
  96. and falls back to the normal screenshot path if Windows returns a blank frame.
  97. It can also save screenshots with `savePath`, which must be an absolute local
  98. `.png` path. Existing files are not replaced unless `overwrite: true` is set.
  99. Tool calls have a 30-second timeout so a blocked UI Automation provider or modal
  100. dialog returns an actionable error instead of hanging the MCP server forever.
  101. ### Tool Examples
  102. Send a keyboard chord to a target element:
  103. ```json
  104. {
  105. "ref": "w1e5",
  106. "chord": "Ctrl+A"
  107. }
  108. ```
  109. Send a sequence of key presses or chords:
  110. ```json
  111. {
  112. "keys": ["Ctrl+A", "Delete", "Enter"]
  113. }
  114. ```
  115. Capture a window using opt-in background capture:
  116. ```json
  117. {
  118. "handle": "w1",
  119. "background": true
  120. }
  121. ```
  122. Save a screenshot to disk without replacing existing files:
  123. ```json
  124. {
  125. "handle": "w1",
  126. "savePath": "C:\\Temp\\capture.png"
  127. }
  128. ```
  129. Replace an existing screenshot file explicitly:
  130. ```json
  131. {
  132. "handle": "w1",
  133. "savePath": "C:\\Temp\\capture.png",
  134. "overwrite": true
  135. }
  136. ```
  137. ### Safety and Limitations
  138. - Keyboard input is focus-dependent. When you use `windows_send_keys`, the tool
  139. focuses the supplied `ref` first when possible, but Windows still sends keys to
  140. the active keyboard focus.
  141. - `windows_screenshot` `background: true` is only valid with a window `handle`.
  142. If native background capture returns a blank frame, FlaUI-MCP falls back to the
  143. normal capture path.
  144. - `savePath` accepts absolute local `.png` paths only. UNC paths, device paths,
  145. non-PNG extensions, and existing files without `overwrite: true` are rejected.
  146. - Desktop integration tests require an interactive Windows session because they
  147. launch real WinForms and WPF windows.
  148. - A timeout error means the MCP request returned, but a blocked Windows UI
  149. Automation provider or modal dialog may still need to be dismissed before
  150. retrying the operation.
  151. ## How It Works
  152. ### The Accessibility Snapshot
  153. When you call `windows_snapshot`, you get a structured text tree:
  154. ```
  155. - window "Calculator" [ref=w1e1]
  156. - group "Number pad" [ref=w1e39]
  157. - button "Seven" [ref=w1e47]
  158. - button "Eight" [ref=w1e48]
  159. - button "Nine" [ref=w1e49]
  160. - text "Display is 0" [ref=w1e15]
  161. ```
  162. This comes from **Windows UI Automation** - the same API screen readers use. Each element has:
  163. - **Role** (button, text, group, textbox)
  164. - **Name** ("Seven", "Display is 0")
  165. - **Ref** (w1e47) - a handle for interaction
  166. - **State** ([disabled], [readonly], [checked])
  167. ### Why Not Screenshots?
  168. | Approach | Pros | Cons |
  169. |----------|------|------|
  170. | **Accessibility Tree** | Semantic, precise, fast, works at any resolution | Requires UI Automation support |
  171. | **Screenshot + Vision** | Works with any app | Slow, expensive, imprecise, resolution-dependent |
  172. FlaUI-MCP uses accessibility because it's what screen readers use - it's designed for programmatic UI interaction.
  173. ## Building from Source
  174. ```powershell
  175. # Clone
  176. git clone https://github.com/shanselman/FlaUI-MCP.git
  177. cd FlaUI-MCP
  178. # Build
  179. dotnet build src/FlaUI.Mcp
  180. # Run
  181. dotnet run --project src/FlaUI.Mcp
  182. ```
  183. ### Testing
  184. ```powershell
  185. # Unit tests
  186. dotnet test tests\FlaUI.Mcp.Tests
  187. # Desktop integration tests; requires an interactive Windows session
  188. dotnet test tests\FlaUI.Mcp.IntegrationTests
  189. ```
  190. ## Architecture
  191. ```
  192. ┌─────────────────────────────────────────────────────────────────┐
  193. │ AI Agent (GitHub Copilot, Claude, etc.) │
  194. │ - Calls MCP tools: windows_snapshot, windows_click, etc. │
  195. └─────────────────────────────────────────────────────────────────┘
  196. │ MCP Protocol (JSON-RPC over stdio)
  197. ▼
  198. ┌─────────────────────────────────────────────────────────────────┐
  199. │ FlaUI-MCP Server (.NET 8) │
  200. │ - Implements MCP tool handlers │
  201. │ - Builds agent-friendly accessibility snapshots │
  202. │ - Maps element refs ↔ AutomationElements │
  203. └─────────────────────────────────────────────────────────────────┘
  204. │
  205. ▼
  206. ┌─────────────────────────────────────────────────────────────────┐
  207. │ FlaUI Library (github.com/FlaUI/FlaUI) │
  208. │ - UIA3Automation for modern apps (WPF, UWP, Win32) │
  209. │ - Control patterns: Invoke, Value, Toggle, Selection │
  210. │ - Tree walking and element discovery │
  211. └─────────────────────────────────────────────────────────────────┘
  212. ```
  213. ## Supported Applications
  214. Works with any Windows application that supports UI Automation:
  215. - ✅ Win32 apps (Notepad, Explorer, etc.)
  216. - ✅ WPF applications
  217. - ✅ WinForms applications
  218. - ✅ UWP/Store apps (Calculator, Settings, etc.)
  219. - ⚠️ Electron apps (partial - depends on accessibility implementation)
  220. - ❌ Games (typically no UI Automation support)
  221. ## Contributing
  222. Contributions welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
  223. ## License
  224. MIT License - see [LICENSE](LICENSE) for details.
  225. ## Acknowledgments
  226. - [FlaUI](https://github.com/FlaUI/FlaUI) - The excellent .NET UI Automation library this project is built on
  227. - [Playwright](https://playwright.dev/) - Inspiration for the snapshot/ref interaction model
  228. - [Model Context Protocol](https://modelcontextprotocol.io/) - The protocol that makes this possible

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.