Skip to content
Back to search

computer-control-mcp

MCP server that provides computer control capabilities, like mouse, keyboard, OCR, etc. using PyAutoGUI, RapidOCR, ONNXRuntime. Similar to 'computer-use' by Anthropic. With Zero External Dependencies.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/ab498--computer-control-mcp/badge.svg)](/listings/ab498--computer-control-mcp)

HTML

<a href="/listings/ab498--computer-control-mcp"><img src="/api/listings/ab498--computer-control-mcp/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

***Note:** Running `uvx computer-control-mcp@latest` for the first time will download python dependencies (around 70MB) which may take some time. Recommended to run this in a terminal before using it as MCP. Subsequent runs will be instant.*

{
  "mcpServers": {
    "computer-control-mcp": {
      "command": "uvx",
      "args": ["computer-control-mcp@latest"]
    }
  }
}

OR install globally with `pip`:

pip install computer-control-mcp

Then run the server with:

computer-control-mcp # instead of uvx computer-control-mcp, so you can use the latest version, also you can `uv cache clean` to clear the cache and `uvx` again to use latest version.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 224 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # Computer Control MCP
  2. ### MCP server that provides computer control capabilities, like mouse, keyboard, OCR, etc. using PyAutoGUI, RapidOCR, ONNXRuntime. Similar to 'computer-use' by Anthropic. With Zero External Dependencies.
  3. <div align="center" style="text-align:center;font-family: monospace; display: flex; align-items: center; justify-content: center; width: 100%; gap: 10px">
  4. <a href="https://nextjs-boilerplate-ashy-nine-64.vercel.app/demo-computer-control"><img
  5. src="https://komarev.com/ghpvc/?username=AB498&label=DEMO&style=for-the-badge&color=CC0000" /></a>
  6. <a href="https://discord.gg/ZeeqSBpjU2"><img
  7. src="https://img.shields.io/discord/1095854826786668545?style=for-the-badge&color=0000CC" alt="Discord"></a>
  8. <a href="https://img.shields.io/badge/License-MIT-yellow.svg"><img
  9. src="https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge&color=00CC00" alt="License: MIT"></a>
  10. <a href="https://pypi.org/project/computer-control-mcp"><img
  11. src="https://img.shields.io/pypi/v/computer-control-mcp?style=for-the-badge" alt="PyPi"></a>
  12. </div>
  13. ---
  14. ![MCP Computer Control Demo](https://github.com/AB498/computer-control-mcp/blob/main/demonstration.gif?raw=true)
  15. ## Quick Usage (MCP Setup Using `uvx`)
  16. ***Note:** Running `uvx computer-control-mcp@latest` for the first time will download python dependencies (around 70MB) which may take some time. Recommended to run this in a terminal before using it as MCP. Subsequent runs will be instant.*
  17. ```json
  18. {
  19. "mcpServers": {
  20. "computer-control-mcp": {
  21. "command": "uvx",
  22. "args": ["computer-control-mcp@latest"]
  23. }
  24. }
  25. }
  26. ```
  27. OR install globally with `pip`:
  28. ```bash
  29. pip install computer-control-mcp
  30. ```
  31. Then run the server with:
  32. ```bash
  33. computer-control-mcp # instead of uvx computer-control-mcp, so you can use the latest version, also you can `uv cache clean` to clear the cache and `uvx` again to use latest version.
  34. ```
  35. ## Features
  36. - Control mouse movements and clicks
  37. - Type text at the current cursor position
  38. - Take screenshots of the entire screen or specific windows with optional saving to downloads directory
  39. - Extract text from screenshots using OCR (Optical Character Recognition)
  40. - List and activate windows
  41. - Press keyboard keys
  42. - Drag and drop operations
  43. - Enhanced screenshot capture for GPU-accelerated windows (Windows only)
  44. ## Note on GPU-accelerated Windows
  45. Traditional screenshot methods like GDI/PrintWindow fail to capture GPU-accelerated windows, resulting in black screens. This impacts games, media players, Electron apps, browsers with GPU acceleration, streaming software, and CAD tools. Use WGC through `take_screenshot` tool's flag or `ENV` variable
  46. ## Configuration
  47. ### Custom Screenshot Directory
  48. By default, screenshots are saved to the OS downloads directory. You can customize this by setting the `COMPUTER_CONTROL_MCP_SCREENSHOT_DIR` environment variable:
  49. ```json
  50. {
  51. "mcpServers": {
  52. "computer-control-mcp": {
  53. "command": "uvx",
  54. "args": ["computer-control-mcp@latest"],
  55. "env": {
  56. "COMPUTER_CONTROL_MCP_SCREENSHOT_DIR": "C:\\Users\\YourName\\Pictures\\Screenshots"
  57. }
  58. }
  59. }
  60. }
  61. ```
  62. Or set it system-wide:
  63. ```bash
  64. # Windows (PowerShell)
  65. $env:COMPUTER_CONTROL_MCP_SCREENSHOT_DIR = "C:\Users\YourName\Pictures\Screenshots"
  66. # macOS/Linux
  67. export COMPUTER_CONTROL_MCP_SCREENSHOT_DIR="/home/yourname/Pictures/Screenshots"
  68. ```
  69. If the specified directory doesn't exist, the server will fall back to the default downloads directory.
  70. ### Automatic WGC for Specific Windows
  71. You can configure the system to automatically use Windows Graphics Capture (WGC) for specific windows by setting the `COMPUTER_CONTROL_MCP_WGC_PATTERNS` environment variable. This variable should contain comma-separated patterns that match window titles:
  72. ```json
  73. {
  74. "mcpServers": {
  75. "computer-control-mcp": {
  76. "command": "uvx",
  77. "args": ["computer-control-mcp@latest"],
  78. "env": {
  79. "COMPUTER_CONTROL_MCP_WGC_PATTERNS": "obs, discord, game, steam"
  80. }
  81. }
  82. }
  83. }
  84. ```
  85. Or set it system-wide:
  86. ```bash
  87. # Windows (PowerShell)
  88. $env:COMPUTER_CONTROL_MCP_WGC_PATTERNS = "obs, discord, game, steam"
  89. # macOS/Linux
  90. export COMPUTER_CONTROL_MCP_WGC_PATTERNS="obs, discord, game, steam"
  91. ```
  92. When this variable is set, any window whose title contains any of the specified patterns will automatically use WGC for screenshot capture, eliminating black screens for GPU-accelerated applications.
  93. ## Available Tools
  94. ### Batch & Sequence Execution
  95. - `execute_actions(actions: List[Dict[str, Any]], fail_fast: bool = True, failsafe: Optional[bool] = None)`: Execute a batch/sequence of operations in a single tool call without LLM round-trip delays. Ideal for smooth mouse movement / curves, multi-step macros, and complex workflows.
  96. Supported actions:
  97. - `move` / `move_mouse`: `{"type": "move", "x": 500, "y": 300, "duration": 0.0}`
  98. - `move_rel`: `{"type": "move_rel", "x_offset": 50, "y_offset": -20}`
  99. - `curve` / `move_curve`: `{"type": "curve", "points": [[x1, y1], [x2, y2], ...], "interval_ms": 5}`
  100. - `click` / `click_screen`: `{"type": "click", "x": 500, "y": 300, "button": "left", "clicks": 1}`
  101. - `double_click`: `{"type": "double_click", "x": 500, "y": 300}`
  102. - `right_click`: `{"type": "right_click", "x": 500, "y": 300}`
  103. - `mouse_down` / `mouse_up`: `{"type": "mouse_down", "button": "left"}`
  104. - `drag` / `drag_mouse`: `{"type": "drag", "from_x": 100, "from_y": 100, "to_x": 400, "to_y": 400}`
  105. - `scroll`: `{"type": "scroll", "clicks": -5}`
  106. - `type` / `type_text`: `{"type": "type", "text": "Hello world"}`
  107. - `press` / `press_keys`: `{"type": "press", "keys": "enter"}` or `{"type": "press", "keys": ["ctrl", "c"]}`
  108. - `hotkey`: `{"type": "hotkey", "keys": ["ctrl", "shift", "p"]}`
  109. - `key_down` / `key_up`: `{"type": "key_down", "key": "shift"}`
  110. - `wait` / `sleep`: `{"type": "wait", "milliseconds": 250}` or `{"type": "wait", "seconds": 0.5}`
  111. - `activate_window`: `{"type": "activate_window", "title_pattern": "Chrome"}`
  112. ### Mouse Control
  113. - `click_screen(x: int, y: int)`: Click at specified screen coordinates
  114. - `move_mouse(x: int, y: int)`: Move mouse cursor to specified coordinates
  115. - `drag_mouse(from_x: int, from_y: int, to_x: int, to_y: int, duration: float = 0.5)`: Drag mouse from one position to another
  116. - `mouse_down(button: str = "left")`: Hold down a mouse button ('left', 'right', 'middle')
  117. - `mouse_up(button: str = "left")`: Release a mouse button ('left', 'right', 'middle')
  118. ### Keyboard Control
  119. - `type_text(text: str)`: Type the specified text at current cursor position
  120. - `key_down(key: str)`: Hold down a specific keyboard key until released
  121. - `key_up(key: str)`: Release a specific keyboard key
  122. - `press_keys(keys: Union[str, List[Union[str, List[str]]]])`: Press keyboard keys (supports single keys, sequences, and combinations)
  123. ### Screen and Window Management
  124. - `take_screenshot(title_pattern: str = None, use_regex: bool = False, threshold: int = 10, scale_percent_for_ocr: int = None, save_to_downloads: bool = False, use_wgc: bool = False)`: Capture screen or window
  125. - `take_screenshot_with_ocr(title_pattern: str = None, use_regex: bool = False, threshold: int = 10, scale_percent_for_ocr: int = None, save_to_downloads: bool = False)`: Extract adn return text with coordinates using OCR from screen or window
  126. - `get_screen_size()`: Get current screen resolution
  127. - `list_windows()`: List all open windows
  128. - `activate_window(title_pattern: str, use_regex: bool = False, threshold: int = 60)`: Bring specified window to foreground
  129. - `wait_milliseconds(milliseconds: int)`: Wait for a specified number of milliseconds
  130. ### Testing MCP Server with `mcp-caller`
  131. You can quickly test the MCP server tools without an MCP client using [mcp-caller](https://www.npmjs.com/package/mcp-caller):
  132. ```bash
  133. # List all available tools
  134. npx mcp-caller --server-json '{"type":"stdio","command":"uvx","args":["computer-control-mcp@latest","server"]}' --list-tools
  135. # Call a specific tool
  136. npx mcp-caller --server-json '{"type":"stdio","command":"uvx","args":["computer-control-mcp@latest","server"]}' --tool list_windows --params-json '{}'
  137. ```
  138. ## Development
  139. ### Setting up the Development Environment
  140. ```bash
  141. # Clone the repository
  142. git clone https://github.com/AB498/computer-control-mcp.git
  143. cd computer-control-mcp
  144. # Build/Run:
  145. # 1. Install in development mode | Meaning that your edits to source code will be reflected in the installed package.
  146. pip install -e .
  147. # Then Start server | This is equivalent to `uvx computer-control-mcp@latest` just the local code is used
  148. computer-control-mcp
  149. # -- OR --
  150. # 2. Build after `pip install hatch` | This needs version increment in orer to reflect code changes
  151. hatch build
  152. # Windows
  153. $latest = Get-ChildItem .\dist\*.whl | Sort-Object LastWriteTime -Descending | Select-Object -First 1
  154. pip install $latest.FullName --upgrade
  155. # Non-windows
  156. pip install dist/*.whl --upgrade
  157. # Run
  158. computer-control-mcp
  159. ```
  160. ### Running Tests
  161. ```bash
  162. python -m pytest
  163. ```
  164. ## API Reference
  165. See the [API Reference](docs/api.md) for detailed information about the available functions and classes.
  166. ## License
  167. MIT
  168. ## For more information or help
  169. - [Email (abcd49800@gmail.com)](mailto:abcd49800@gmail.com)
  170. - [Discord (CodePlayground)](https://discord.gg/ZeeqSBpjU2)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.