Skip to content
Back to search

OpenGUI

OpenGUI is an Android GUI agent framework for phone-use AI that can see, plan, and operate real mobile apps through the GUI.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/Core-Mate/OpenGUI with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/Core-Mate/OpenGUI'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/core-mate--opengui/badge.svg)](/listings/core-mate--opengui)

HTML

<a href="/listings/core-mate--opengui"><img src="/api/listings/core-mate--opengui/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Use the repository scripts directly:

cd server
./start.sh
cd client
./start.sh

Reference docs:

- [docs/get-started.md](./docs/get-started.md) - [docs/troubleshooting.md](./docs/troubleshooting.md) - [server/start.sh](./server/start.sh) - [client/start.sh](./client/start.sh) - [server/apps/backend/README.md](./server/apps/backend/README.md) - [docs/DISCORD.md](./docs/DISCORD.md) - [client/README.md](./client/README.md)

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 294 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <p align="center">
  2. <strong>Language:</strong> <a href="./README.md">English</a> | <a href="./README.zh-CN.md">简体中文</a> | <a href="./README.ja-JP.md">日本語</a>
  3. </p>
  4. <p align="center">
  5. <img src="./docs/assets/opengui-banner.svg" alt="OpenGUI banner" width="100%">
  6. </p>
  7. <p align="center">
  8. <a href="https://trendshift.io/repositories/183339"><img src="https://trendshift.io/api/badge/trendshift/repositories/183339/daily?language=Kotlin" alt="OpenGUI — Trendshift Kotlin Repository of the Day #25" width="250" height="55"></a>
  9. </p>
  10. <p align="center">
  11. <a href="#use-opengui-in-workbuddy"><img src="https://img.shields.io/badge/INSTALL-WORKBUDDY_PLUGIN-168a70?style=for-the-badge" alt="Install the WorkBuddy plugin"></a>
  12. <a href="#use-opengui-in-deepseek-harness"><img src="https://img.shields.io/badge/INSTALL-DEEPSEEK_HARNESS_PLUGIN-6f42c1?style=for-the-badge" alt="Install the DeepSeek Harness plugin"></a>
  13. <a href="./skills/open-gui-bootstrap/SKILL.md"><img src="https://img.shields.io/badge/BOOTSTRAP-WITH_AI_AGENTS-ffb000?style=for-the-badge" alt="Bootstrap with Claude Code, Codex, or OpenCode"></a>
  14. <img src="https://img.shields.io/badge/SYSTEM-MULTI_ROLE_OPERATOR-1f6feb?style=for-the-badge" alt="Multi-role operator system">
  15. <a href="./docs/get-started.md"><img src="https://img.shields.io/badge/MANUAL_SETUP-DOCS-4b4b4b?style=for-the-badge" alt="Manual setup docs"></a>
  16. </p>
  17. <p align="center">
  18. <strong>A mobile GUI agent framework for Android.</strong>
  19. </p>
  20. <p align="center">
  21. OpenGUI helps AI agents see, understand, and operate Android app interfaces on real devices.
  22. </p>
  23. <p align="center">
  24. <strong>Use OpenGUI in WorkBuddy or DeepSeek Harness.</strong><br>
  25. Install the plugin, connect your phone, and describe your task. No full backend deployment is required.
  26. </p>
  27. ## Features at a Glance
  28. OpenGUI uses real screenshots to understand apps and carry out phone operations and testing tasks from natural-language instructions.
  29. - **Phone operations**: open apps, tap, swipe, type, navigate back, and move between screens to complete multi-step workflows.
  30. - **App testing**: check features and workflows, record issues with reproduction steps and screenshots, and summarize results.
  31. - **Content and information**: read screen content, organize information, fill forms, and prepare social media drafts.
  32. - **Visible execution**: use the WorkBuddy workbench to view the phone screen, task steps, and reports, or take over and stop a task.
  33. - **Host integration**: use OpenGUI in WorkBuddy or DeepSeek Harness; the DSH plugin also supports managed browser operations.
  34. ## Meet CoreMate
  35. **The AI workforce for social media and growth teams.** Scale outreach and follow-ups across many accounts and cloud phones, without adding headcount.
  36. CoreMate puts AI agents to work in real mobile apps. They tap, type, read screens, send messages, and record results, running on **Android cloud phones that we host, update, and maintain for you**.
  37. Describe your goal in plain language. CoreMate turns it into a step-by-step plan your team can review, then executes it in parallel across as many cloud phones as you need.
  38. Try CoreMate through the [Core-Mate](https://www.core-mate.com/).
  39. ## Common Commands
  40. After installation and authorization, enter these instructions in the corresponding host's chat. Replace `[placeholders]` with your task details.
  41. | Purpose | Host | Example input |
  42. |---|---|---|
  43. | Open OpenGUI | WorkBuddy | `/opengui` |
  44. | Check connected devices | WorkBuddy | `/opengui List connected devices without operating a phone.` |
  45. | Operate a phone | WorkBuddy | `/opengui Open Settings, then return to the home screen. Stop after confirming it is visible.` |
  46. | Test an app | WorkBuddy | `/opengui Test [feature or workflow] in [app/page]. Record issues, steps, and screenshots. Stop at [end condition].` |
  47. | Prepare a Tiktok draft | WorkBuddy | `/opengui Prepare a Tiktok image-and-text draft about [topic]. Stop before publishing.` |
  48. | Run a task in DSH | DeepSeek Harness | `@OpenGUI Open Settings and report the Android version` |
  49. ## Use OpenGUI in WorkBuddy
  50. Supported on **macOS (Apple Silicon / Intel) with WorkBuddy 5.5.3 or later**. The current public release is [0.4.2, a public-testing prerelease](https://github.com/Core-Mate/OpenGUI/releases/tag/opengui-workbuddy-v0.4.2). China and international WorkBuddy use the same OpenGUI package and version. Custom MCP installation is available without a connector marketplace listing.
  51. OpenGUI and WorkBuddy accounts are separate. OpenGUI SMS sign-in currently requires a **mainland-China phone number**; signing into international WorkBuddy with Google/GitHub does not sign you into OpenGUI.
  52. > [!IMPORTANT]
  53. > **Restart WorkBuddy once after installation.** Finish other tasks, **fully quit with ⌘Q**, then reopen WorkBuddy before authorizing OpenGUI. Closing its window is not enough.
  54. **Step 1. Install with a Terminal command**
  55. Open macOS Terminal and run the command for your WorkBuddy edition. If you use both editions, install into each one separately. Complete the remaining steps in the edition selected here.
  56. **China edition — WorkBuddy.app**
  57. ```sh
  58. cd "$HOME" && curl -fsSL https://github.com/Core-Mate/OpenGUI/releases/download/opengui-workbuddy-v0.4.2/install.sh | bash -s -- --app "/Applications/WorkBuddy.app" --download-source cn
  59. ```
  60. **International edition — WorkBuddy AI.app**
  61. ```sh
  62. cd "$HOME" && curl -fsSL https://github.com/Core-Mate/OpenGUI/releases/download/opengui-workbuddy-v0.4.2/install.sh | bash -s -- --app "/Applications/WorkBuddy AI.app" --download-source official
  63. ```
  64. These commands explicitly select the target application, so having both editions installed will not cause OpenGUI to be installed into the wrong one. If your application is elsewhere or has been renamed, replace the path after `--app` with its actual path. The script downloads and verifies the installation files, prepares the runtime, and configures OpenGUI.
  65. The China command uses npmmirror for Node and npm dependencies with official fallback; the international command uses official sources. To use official sources with China WorkBuddy, change `--download-source cn` to `--download-source official`. This setting applies only to this installation. Video assets use GitHub separately; an npm mirror does not accelerate them. See [download source options](workbuddy-plugin/INSTALL.md#download-sources) for a custom video mirror.
  66. You can also [download install.sh separately](https://github.com/Core-Mate/OpenGUI/releases/download/opengui-workbuddy-v0.4.2/install.sh) and run `bash /actual/path/install.sh` with the same `--app` and `--download-source` arguments shown above.
  67. See [international installation and account limitations](workbuddy-plugin/INSTALL.md#international-workbuddy), or the [GitHub installation flow (Chinese)](workbuddy-plugin/docs/install-from-github.zh-CN.md) for a copyable WorkBuddy prompt, native authorization and connection verification.
  68. **Step 2. Restart WorkBuddy once**
  69. After the installer finishes, complete other WorkBuddy tasks, fully quit with **⌘Q**, and reopen WorkBuddy to load the newly installed MCP configuration.
  70. **Step 3. Authorize the Skill and connector**
  71. - **China edition**: authorize and enable OpenGUI under **专家·技能·连接器 → 技能**. Then open **连接器 → 自定义连接器**, find `opengui`, complete the trust prompt, and enable it.
  72. - **International edition**: authorize and enable OpenGUI under **Experts · Skills · Connectors → Skills**. Then open **Connectors → Custom connectors**, click **Trust** for `opengui`, and enable it.
  73. If an update triggers another Trust prompt, review and confirm it yourself.
  74. **Both the Skill and connector require authorization.**
  75. ![Authorize the OpenGUI Skill and MCP connector](./workbuddy-plugin/resources/Skill和MCP授权.gif)
  76. If OpenGUI is not listed under Connectors, check **Custom connectors** for its MCP service. If the authorization page did not open automatically, open the `AUTHORIZATION_GUIDE` path printed in Terminal; no reinstallation is needed.
  77. **Step 4. Verify the connection and start using OpenGUI**
  78. You can first ask WorkBuddy: “Call `opengui_list_devices` once to verify the connection, report the device count, and do not operate a phone.” A successful call returning zero devices also confirms the plugin connection.
  79. To start a phone task:
  80. 1. Enable USB debugging on your Android phone, connect it to your computer, and allow debugging on the phone. You can also start a local Android emulator from the workbench.
  81. 2. Enter `/opengui` in WorkBuddy to open the task home on the right, then sign into OpenGUI as prompted.
  82. 3. Describe your goal, confirm the device, and click **开始执行** (Start task). Execution follows the current WorkBuddy conversation model.
  83. Try it: Post on Tiktok
  84. ```text
  85. /opengui Prepare a Tiktok image-and-text draft about [topic] using [materials]. Stop before publishing.
  86. ```
  87. Try it: Vibe Testing
  88. ```text
  89. /opengui Help me test [feature or workflow] in [app/page], focusing on [issues to check]. If you find any problems, record the steps and screenshots. Stop when [end condition] is reached.
  90. ```
  91. **UI language (0.4.2)**: Chinese/English switching, following WorkBuddy’s display language, and emulator-picker improvements are included. The language option translates interface labels; task descriptions, TODO content, and review drafts retain their original text.
  92. ## Use OpenGUI in DeepSeek Harness
  93. The shortest path on macOS is to let Codex run the stable installer Skill from `main`. Each run resolves the latest stable OpenGUI plugin release, while an explicit version remains available for rollback. It requires Node.js 22.19+ or 24+ and installs the compatible DSH version automatically. Paste this as one prompt:
  94. ```text
  95. Install and run the OpenGUI installer Skill from https://github.com/Core-Mate/OpenGUI/tree/main/deepseek-harness-plugin/skills/opengui-coremate-install for my DSH web profile. Install the latest stable release. Proceed autonomously, and only pause when I need to authorize or select a phone, add or select a DSH workspace, or provide fallback visual-model credentials.
  96. ```
  97. The Skill downloads the public release package and checksum, verifies SHA-256, installs only the OpenGUI plugin, starts DSH when needed, and opens DSH. It preserves unrelated DSH plugins and settings. The installer reports whether it reloaded a managed DSH or whether you need to quit an existing process and rerun it. For Linux or Windows, use the [manual package guide](./deepseek-harness-plugin/README.md#1-download-the-release-package).
  98. Supported DSH versions are defined in the [installer compatibility list](./deepseek-harness-plugin/skills/opengui-coremate-install/dsh-compatibility.json). The default is `0.1.1-rc.2`; use `--dsh-version VERSION` to select a listed version. The installer preserves other plugins, workspaces, model settings, and credentials. See the [plugin installation guide](./deepseek-harness-plugin/README.md#requirements-and-support) for version changes and rollback constraints.
  99. After installation, add or select a DSH workspace, connect and select an authorized Android phone, then send:
  100. ```text
  101. @OpenGUI Open Settings and report the Android version
  102. ```
  103. The plugin adds phone and browser operation to DSH without requiring the full OpenGUI backend stack. The current source implementation admits one OpenGUI task per DSH session and separate tabs on non-conflicting phone sets; the managed browser remains globally serial. This source behavior is not a release claim. See more [use cases](./deepseek-harness-plugin/docs/use-cases.md) or download the [v0.1.13 release package](https://github.com/Core-Mate/OpenGUI/releases/tag/dsh-coremate-mobile-v0.1.13).
  104. GUI execution requires a model with image input and tool calling. DSH uses the current session model by default; configure a fallback visual model when needed, following the [plugin guide](./deepseek-harness-plugin/README.md). Evaluate execution quality, latency, and actual costs on your own tasks.
  105. ## Run the Full OpenGUI Stack
  106. Use the full stack to self-host the backend and Android client or dispatch tasks through Feishu, Telegram, Discord, or REST APIs. WorkBuddy and DSH plugin users do not need this deployment.
  107. Open Claude Code, Codex, or OpenCode in the repository root and send:
  108. ```text
  109. Read ./skills/open-gui-bootstrap/SKILL.md and help me run OpenGUI. Only ask me for phone-side actions.
  110. ```
  111. The full-stack Android client requires Android 11 (API 30) or later, USB debugging, AccessibilityService, overlay permission, and a battery optimization exemption. See the [deployment guide](./docs/get-started.md) and [Android permission guide](./docs/android-permissions.md) for model configuration, manual startup, and phone authorization.
  112. After deployment, run the CLI from the `server` directory:
  113. ```bash
  114. pnpm opengui -- devices --json
  115. pnpm opengui -- do "Observe the current Android screen, summarize what you see, and stop" --json
  116. pnpm opengui -- status <executionId> --json
  117. pnpm opengui -- cancel <executionId> --json
  118. ```
  119. `do` returns an `executionId` asynchronously. Use it with `status` to check progress or `cancel` to stop the task. See the [CLI and API guide](./docs/codex-remote-control.md) and [Discord setup](./docs/DISCORD.md) for remote integration.
  120. ## Recent Updates
  121. - `[2026.10.8]` Published the [WorkBuddy 0.4.0 public-testing prerelease](https://github.com/Core-Mate/OpenGUI/releases/tag/opengui-workbuddy-v0.4.0), with a Terminal installer and an authorization guide that opens after installation.
  122. - `[2026.9.1]` Published the [DSH 0.1.13 plugin release](https://github.com/Core-Mate/OpenGUI/releases/tag/dsh-coremate-mobile-v0.1.13).
  123. - `[2026.5.16]` Added [Codex / Claude Code remote control](./docs/codex-remote-control.md) with a local REST API, `pnpm opengui -- ...` CLI, and the [`open-gui-remote-control`](./skills/open-gui-remote-control/SKILL.md) Skill for dispatching Android app tasks from coding agents.
  124. - `[2026.5.12]` Added a troubleshooting guide for backend connection, Android permissions, model configuration, and local Redis/PostgreSQL conflicts.
  125. - `[2026.5.9]` Added a [Discord IM channel](./docs/DISCORD.md) for remote Android task dispatch, including prefix commands, slash commands, allowlists, and guild-scoped command registration.
  126. - `[2026.5.7]` Hardened local startup to avoid common PostgreSQL and Redis port conflicts during Docker-based backend setup.
  127. - `[2026.5.1]` Improved backend onboarding with `.env.example`, startup checks, and graph-agent VLM environment configuration.
  128. ## Requirements and Limitations
  129. | Setup | Computer and runtime | Android device preparation |
  130. |---|---|---|
  131. | WorkBuddy plugin | macOS (Apple Silicon / Intel), WorkBuddy 5.5.3 or later; the installer prepares a private runtime. | Enable and authorize USB debugging. Uses host-side ADB / scrcpy; no full backend or repository Android client is required. |
  132. | DSH plugin | macOS, Linux x64, or Windows x64; see the [plugin guide](./deepseek-harness-plugin/README.md#requirements-and-support) for Node.js and DSH requirements. | Enable and authorize USB debugging, then select the device in DSH. No full backend or repository Android client is required. |
  133. | Full stack | Local backend and build tools; see the [deployment guide](./docs/get-started.md). | Android 11 or later, with the client's accessibility, overlay, and battery permissions. |
  134. - Some phones need an additional vendor-specific USB input permission. Follow the connection diagnostics on the phone.
  135. - WorkBuddy uses its current conversation model; DSH session models and full-backend model settings are separate setup paths.
  136. - Results depend on the model, app UI, network, and task length. Long-running tasks and device compatibility still need more real-world verification.
  137. - WorkBuddy SMS sign-in uses the official CoreMate account service by default. No online model catalog is read. Task screenshots go to the current WorkBuddy model; local preview video is not uploaded frame by frame. See the [WorkBuddy guide](./workbuddy-plugin/README.md) and [DSH guide](./deepseek-harness-plugin/README.md) for their data flows.
  138. ## Roadmap
  139. - Add more real app examples and test reports.
  140. - Improve one-command local setup.
  141. - Add more ready-to-run phone-use task templates.
  142. - Improve execution recovery and failure reporting.
  143. - Add benchmark tasks for Android GUI agent reliability.
  144. - Expand docs for model configuration and cost-saving profiles.
  145. - Launch a hosted OpenGUI Agent service for teams that want GUI operation without running the full stack themselves.
  146. ## The System
  147. This diagram describes the full backend and Android client. WorkBuddy and DSH each use their own plugin runtime.
  148. ```mermaid
  149. flowchart LR
  150. U["User or IM command"] --> BS["Bootstrap Skill / API / IM entry"]
  151. BS --> SP["Plan Supervisor"]
  152. SP --> EX["Executor Graph"]
  153. EX --> AC["Android Client"]
  154. AC --> GX["AccessibilityService + screenshots + actions"]
  155. EX --> RV["Execution review and retry"]
  156. RV --> SP
  157. SP --> SM["Summarizer"]
  158. SM --> SR["Structured Results"]
  159. RD["Feishu / Telegram / Discord / REST API"] --> ST["Standby Gateway"]
  160. ST --> AC
  161. SP --> MR["Model Routing"]
  162. MR --> MA["Claude / GPT / Gemini / Kimi / MiniMax / compatible"]
  163. EX --> MR
  164. ```
  165. ### Core Runtime Pieces
  166. - **Backend graph**: `server/apps/backend/src/modules/graph-agent/graph/`
  167. - **Task APIs**: `server/apps/backend/src/modules/task/task.controller.ts`
  168. - **Standby dispatch**: `server/apps/backend/src/common/ws/standby.gateway.ts`
  169. - **IM channel dispatch**: `server/apps/backend/src/modules/im-channel/`
  170. - **Android standby connection**: `client/core_network/src/main/java/com/coremate/opengui/network/websocket/StandbySocketManager.kt`
  171. - **Android execution path**: `client/core_accessibility/src/main/java/com/coremate/opengui/accessibility/GestureService.kt`
  172. ## Documentation
  173. - [WorkBuddy installation verification and troubleshooting](./workbuddy-plugin/INSTALL.md)
  174. - [WorkBuddy plugin guide](./workbuddy-plugin/README.md)
  175. - [DeepSeek Harness plugin guide](./deepseek-harness-plugin/README.md)
  176. - [skills/open-gui-bootstrap/SKILL.md](./skills/open-gui-bootstrap/SKILL.md)
  177. - [docs/get-started.md](./docs/get-started.md)
  178. - [server/apps/backend/README.md](./server/apps/backend/README.md)
  179. - [docs/DISCORD.md](./docs/DISCORD.md)
  180. - [client/README.md](./client/README.md)
  181. - [CONTRIBUTING.md](./CONTRIBUTING.md)
  182. - [SECURITY.md](./SECURITY.md)
  183. - [CLAUDE.md](./CLAUDE.md)
  184. ## Community / Support
  185. Join the [OpenGUI Discord community](https://discord.gg/pqHHw7XgJ3) to discuss GUI agent development, share real use cases, and get release updates. A verified WeChat community entry will be published here when it is ready.
  186. Community members will also be able to apply for trial Agent credits when the hosted OpenGUI Agent service opens. Availability, eligibility, and validity will be announced with the service.
  187. The most useful project feedback is:
  188. - open issues for bugs and feature requests
  189. - share real use cases and deployment feedback
  190. - contribute docs, integrations, and fixes
  191. ## License
  192. OpenGUI is source-available under the Business Source License 1.1 (BUSL-1.1).
  193. You may copy, modify, distribute, and use the source for non-production purposes. Production use, commercial use, hosted services, and integration into commercial products require a separate commercial license from Core-Mate.
  194. For this version:
  195. - Change Date: 2030-04-29
  196. - Change License: Apache License, Version 2.0
  197. This is public source, but it is not OSI-approved open source until the Change Date.
  198. See [LICENSE](./LICENSE).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.