Skip to content
Back to search

midscene-skills

AI-powered, vision-driven UI automation for every platform.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/web-infra-dev-midscene-skills/badge.svg)](/listings/web-infra-dev-midscene-skills)

HTML

<a href="/listings/web-infra-dev-midscene-skills"><img src="/api/listings/web-infra-dev-midscene-skills/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Make sure you have [Node.js](https://nodejs.org) installed.

Then install the skills:

# General installation
npx skills add web-infra-dev/midscene-skills

# Claude Code 
npx skills add web-infra-dev/midscene-skills -a claude-code

# OpenClaw
npx skills add web-infra-dev/midscene-skills -a openclaw

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 111 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <p align="center">
  2. <img alt="Midscene.js" width="260" src="https://github.com/user-attachments/assets/2e0b2957-2ab7-46ec-a4a1-21459e0a7e48">
  3. </p>
  4. # Midscene Skills
  5. <strong> Vision-driven cross-platform automation </strong>
  6. - Natural-language driven UI control
  7. - Built on [Midscene.js](https://midscenejs.com)'s vision-based automation capabilities — Operates entirely from screenshots, making cross-platform support reliable
  8. - This repository contains Skills for the following platforms:
  9. - Browser (Puppeteer, headless Chrome; also supports CDP connection): [`skills/browser`](skills/browser)
  10. - Chrome Bridge (user's own Chrome browser): [`skills/chrome-bridge`](skills/chrome-bridge)
  11. - Desktop (macOS, Windows, Linux): [`skills/computer-automation`](skills/computer-automation)
  12. - Android (controlled via ADB): [`skills/android-automation`](skills/android-automation)
  13. - iOS (controlled via WebDriverAgent): [`skills/ios-automation`](skills/ios-automation)
  14. - HarmonyOS (controlled via HDC): [`skills/harmony-automation`](skills/harmony-automation)
  15. - Vitest + Midscene E2E (scaffold, convert, and manage AI-powered E2E tests for Web, Android, iOS): [`skills/vitest-midscene-e2e`](skills/vitest-midscene-e2e)
  16. ## Safety Warning
  17. ⚠️ AI-driven UI automation may produce unpredictable results since it can control EVERYTHING on the screen. Please evaluate the risks carefully before use.
  18. ## Installation
  19. Make sure you have [Node.js](https://nodejs.org) installed.
  20. Then install the skills:
  21. ```bash
  22. # General installation
  23. npx skills add web-infra-dev/midscene-skills
  24. # Claude Code
  25. npx skills add web-infra-dev/midscene-skills -a claude-code
  26. # OpenClaw
  27. npx skills add web-infra-dev/midscene-skills -a openclaw
  28. ```
  29. ## Model Setup
  30. Midscene requires models with strong **visual grounding** capabilities (accurate UI element localization from screenshots).
  31. Because of this, you need to prepare model access and configuration separately from skill installation.
  32. Make sure these environment variables are available in your system. You can also define them in a `.env` file in the current directory, and Midscene will load them automatically:
  33. ```bash
  34. MIDSCENE_MODEL_API_KEY="your-api-key"
  35. MIDSCENE_MODEL_NAME="model-name"
  36. MIDSCENE_MODEL_BASE_URL="https://..."
  37. MIDSCENE_MODEL_FAMILY="family-identifier"
  38. ```
  39. Example: Gemini (Gemini-3-Flash)
  40. ```bash
  41. MIDSCENE_MODEL_API_KEY="your-google-api-key"
  42. MIDSCENE_MODEL_NAME="gemini-3-flash"
  43. MIDSCENE_MODEL_BASE_URL="https://generativelanguage.googleapis.com/v1beta/openai/"
  44. MIDSCENE_MODEL_FAMILY="gemini"
  45. ```
  46. Example: Qwen3-VL
  47. ```bash
  48. MIDSCENE_MODEL_API_KEY="your-openrouter-api-key"
  49. MIDSCENE_MODEL_NAME="qwen/qwen3-vl-235b-a22b-instruct"
  50. MIDSCENE_MODEL_BASE_URL="https://openrouter.ai/api/v1"
  51. MIDSCENE_MODEL_FAMILY="qwen3-vl"
  52. ```
  53. Example: Doubao Seed 1.6
  54. ```bash
  55. MIDSCENE_MODEL_API_KEY="your-doubao-api-key"
  56. MIDSCENE_MODEL_NAME="doubao-seed-1-6-250615"
  57. MIDSCENE_MODEL_BASE_URL="https://ark.cn-beijing.volces.com/api/v3"
  58. MIDSCENE_MODEL_FAMILY="doubao-vision"
  59. ```
  60. Commonly used models: Doubao Seed 1.6, Qwen3-VL, Zhipu GLM-4.6V, Gemini-3-Pro, Gemini-3-Flash.
  61. Model setup docs:
  62. - Midscene model strategy: https://midscenejs.com/model-strategy
  63. - Qwen3-VL: https://midscenejs.com/model-common-config#qwen3-vl
  64. - Gemini-3-Flash: https://midscenejs.com/model-common-config#gemini-3-pro-and-gemini-3-flash
  65. - Doubao Seed 1.6: https://midscenejs.com/model-common-config
  66. ## Use
  67. In your chatbot or coding agent, you can say:
  68. ```
  69. Use Midscene computer skill to open the Keynote app and create a new presentation.
  70. ```
  71. ```
  72. Use Midscene browser skill to open the Google search page and search for "Midscene".
  73. ```
  74. ## Issues
  75. For bug reports, feature requests, and discussions, please visit the main Midscene repository: https://github.com/web-infra-dev/midscene/issues
  76. ## License
  77. MIT

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.