Skip to content
Back to search

human-mcp

Bringing Human Capabilities to AI Agents

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/mrgoonie--human-mcp/badge.svg)](/listings/mrgoonie--human-mcp)

HTML

<a href="/listings/mrgoonie--human-mcp"><img src="/api/listings/mrgoonie--human-mcp/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Getting Your Google Gemini API Key

Before installation, you'll need a Google Gemini API key to enable visual analysis capabilities.

Step 1: Access Google AI Studio

1. Visit [Google AI Studio](https://aistudio.google.com/) in your web browser 2. Sign in with your Google account (create one if needed) 3. Accept the terms of service when prompted

Step 2: Create an API Key

1. In the Google AI Studio interface, look for the "Get API Key" button or navigate to the API keys section 2. Click "Create API key" or "Generate API key" 3. Choose "Create API key in new project" (recommended) or select an existing Google Cloud project 4. Your API key will be generated and displayed 5. **Important**: Copy the API key immediately as it may not be shown again

Step 3: Secure Your API Key

⚠️ **Security Warning**: Treat your API key like a password. Never share it publicly or commit it to version control.

**Best Practices:** - Store the key in environment variables (not in code) - Don't include it in screenshots or documentation - Regenerate the key if accidentally exposed - Set usage quotas and monitoring in Google Cloud Console - Restrict API key usage to specific services if possible

Step 4: Set Up Environment Variable

Configure your API key using one of these methods:

**Method 1: Shell Environment (Recommended)**

# Add to your shell profile (.bashrc, .zshrc, .bash_profile)
export GOOGLE_GEMINI_API_KEY="your_api_key_here"

# Reload your shell configuration
source ~/.zshrc  # or ~/.bashrc

**Method 2: Project-specific .env File**

# Create a .env file in your project directory
echo "GOOGLE_GEMINI_API_KEY=your_api_key_here" > .env

# Add .env to your .gitignore file
echo ".env" >> .gitignore

**Method 3: MCP Client Configuration** You can also provide the API key directly in your MCP client configuration (shown in setup examples below).

Step 5: Verify API Access

Test your API key works correctly:

# Test with curl (optional verification)
curl -H "Content-Type: application/json" \
     -d '{"contents":[{"parts":[{"text":"Hello"}]}]}' \
     -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent?key=YOUR_API_KEY"

Choosing Your Gemini Provider

Human MCP supports two ways to access Google's Gemini models:

Option 1: Google AI Studio (Default - Recommended for Getting Started)

**Best for:** Quick start, development, prototyping

**Setup:** 1. Get your API key from [Google AI Studio](https://aistudio.google.com/) 2. Set environment variable: `export GOOGLE_GEMINI_API_KEY=your_api_key`

**Pros:** - Simple setup (just one API key) - No GCP account required - Free tier available - Perfect for development and testing

Option 2: Vertex AI (Recommended for Production)

**Best for:** Production deployments, enterprise use, GCP integration

**Setup:** 1. Create a GCP project and enable Vertex AI API 2. Set up authentication:

   # Option A: Application Default Credentials (for local dev)
   gcloud auth application-default login

   # Option B: Service Account (for production)
   export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json

3. Set environment variables:

   export USE_VERTEX=1
   export VERTEX_PROJECT_ID=your-gcp-project-id
   export VERTEX_LOCATION=us-central1  # optional, defaults to us-central1

**Pros:** - Enterprise-grade quotas and SLAs - Better integration with GCP services - Advanced IAM and security controls - Usage tracking via Cloud Console - Better for production workloads

**Configuration Example (Claude Desktop):**

{
  "mcpServers": {
    "human-mcp-vertex": {
      "command": "npx",
      "args": ["@goonnguyen/human-mcp"],
      "env": {
        "USE_VERTEX": "1",
        "VERTEX_PROJECT_ID": "your-gcp-project-id",
        "VERTEX_LOCATION": "us-central1"
      }
    }
  }
}

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 1,760 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Human MCP 👁️
  2. > Bringing Human Capabilities to AI Agents
  3. ![Human MCP](human-mcp.png)
  4. Human MCP v2.16.0 is a comprehensive Model Context Protocol server that provides AI coding agents with human-like capabilities including visual analysis, document processing, speech generation, content creation, image editing, browser automation, and advanced reasoning for debugging, understanding, and enhancing multimodal content.
  5. ## "Human MCP" is a part of [ClaudeKit](https://claudekit.cc)
  6. ![ClaudeKit.cc](claudekit.png)
  7. ## Features
  8. 🎯 **Visual Analysis (Eyes) - ✅ Complete (4 tools)**
  9. - **eyes_analyze**: Analyze images, videos, and GIFs for UI bugs, errors, and accessibility
  10. - **eyes_compare**: Compare two images to find visual differences
  11. - **eyes_read_document**: Extract text and data from PDF, DOCX, XLSX, PPTX, and more
  12. - **eyes_summarize_document**: Generate summaries and insights from documents
  13. ✋ **Content Generation & Image Editing (Hands) - ✅ Complete (18 tools)**
  14. - **Image Generation** (1 tool): gemini_gen_image - Generate images from text using Imagen API
  15. - **Video Generation** (2 tools): gemini_gen_video, gemini_image_to_video - Create videos with Veo 3.0
  16. - **Music Generation** (2 tools): minimax_gen_music, elevenlabs_gen_music - Generate music with vocals
  17. - **Sound Effects** (1 tool): elevenlabs_gen_sfx - Generate sound effects from text descriptions
  18. - **AI Image Editing** (5 tools): Gemini-powered editing with inpainting, outpainting, style transfer, object manipulation, composition
  19. - **Jimp Processing** (4 tools): Local image manipulation - crop, resize, rotate, mask
  20. - **Background Removal** (1 tool): rmbg_remove_background - AI-powered background removal
  21. - **Browser Automation** (3 tools): playwright_screenshot_fullpage, playwright_screenshot_viewport, playwright_screenshot_element - Automated web screenshots
  22. 🗣️ **Speech Generation (Mouth) - ✅ Complete (4 tools)**
  23. - **mouth_speak**: Convert text to speech with 30+ voices and 24 languages
  24. - **mouth_narrate**: Long-form content narration with chapter breaks
  25. - **mouth_explain**: Generate spoken code explanations with technical analysis
  26. - **mouth_customize**: Test and compare different voices and styles
  27. 🧠 **Advanced Reasoning (Brain) - ✅ Complete (3 tools)**
  28. - **mcp__reasoning__sequentialthinking**: Native sequential thinking with thought revision
  29. - **brain_analyze_simple**: Fast pattern-based analysis (problem solving, root cause, SWOT, etc.)
  30. - **brain_patterns_info**: List available reasoning patterns and frameworks
  31. - **brain_reflect_enhanced**: AI-powered meta-cognitive reflection for complex analysis
  32. ## Total: 29 MCP Tools Across 4 Human Capabilities
  33. **👁️ Eyes (4 tools)** - Visual analysis and document processing
  34. **✋ Hands (18 tools)** - Content generation, image editing, music/SFX, and browser automation
  35. **🗣️ Mouth (4 tools)** - Speech generation and narration
  36. **🧠 Brain (3 tools)** - Advanced reasoning and problem solving
  37. ### Technology Stack
  38. - **Google Gemini 2.5 Flash** - Vision, document, and reasoning AI
  39. - **Gemini Imagen API** - High-quality image generation
  40. - **Gemini Veo 3.0 API** - Professional video generation
  41. - **Gemini Speech API** - Natural voice synthesis (30+ voices, 24 languages)
  42. - **Minimax API** - Alternative speech (Speech 2.6), music (Music 2.5), video (Hailuo 2.3)
  43. - **ZhipuAI (Z.AI) API** - Alternative vision (GLM-4.6V), image (GLM-Image), video (CogVideoX-3)
  44. - **ElevenLabs API** - Text-to-speech (70+ languages), music generation, sound effects
  45. - **Playwright** - Browser automation for web screenshots
  46. - **Jimp** - Fast local image processing
  47. - **rmbg** - AI-powered background removal (U2Net+, ModNet, BRIAI models)
  48. ### Supported Providers & Models
  49. Human MCP supports multiple AI providers per capability. Set via per-request `provider` parameter or environment variable defaults.
  50. #### Vision (Eyes)
  51. | Provider | Models | Features | Env Var |
  52. |----------|--------|----------|---------|
  53. | **Google Gemini** (default) | `gemini-2.5-flash`, `gemini-2.5-pro` | Image, video, GIF analysis; document processing | `GOOGLE_GEMINI_API_KEY` |
  54. | **ZhipuAI** | `glm-4.6v`, `glm-4.6v-flash` (FREE) | Image analysis via GLM-4.6V vision | `ZHIPUAI_API_KEY` |
  55. #### Image Generation (Hands)
  56. | Provider | Models | Features | Env Var |
  57. |----------|--------|----------|---------|
  58. | **Google Gemini** (default) | `gemini-2.5-flash-image`, `gemini-3.1-flash-image-preview` | Text-to-image, 14 aspect ratios, 5 styles | `GOOGLE_GEMINI_API_KEY` |
  59. | **ZhipuAI** | `cogview-4-250304` | Text-to-image, CogView-4 | `ZHIPUAI_API_KEY` |
  60. #### Video Generation (Hands)
  61. | Provider | Models | Features | Env Var |
  62. |----------|--------|----------|---------|
  63. | **Google Gemini** (default) | `veo-3.0-generate-001` | Text-to-video, image-to-video, 4-12s, camera controls | `GOOGLE_GEMINI_API_KEY` |
  64. | **Minimax** | `MiniMax-Hailuo-2.3`, `MiniMax-Hailuo-2.3-Fast` | Text-to-video, image-to-video, 768P/1080P | `MINIMAX_API_KEY` |
  65. | **ZhipuAI** | `cogvideox-3` | Text-to-video, image-to-video, async polling | `ZHIPUAI_API_KEY` |
  66. #### Speech / TTS (Mouth)
  67. | Provider | Models | Features | Env Var |
  68. |----------|--------|----------|---------|
  69. | **Google Gemini** (default) | `gemini-2.5-flash-preview-tts`, `gemini-2.5-pro-preview-tts` | 31 voices, 24 languages, style prompts | `GOOGLE_GEMINI_API_KEY` |
  70. | **Minimax** | `speech-2.6-hd`, `speech-2.6-turbo` | Emotion control, speed adjustment | `MINIMAX_API_KEY` |
  71. | **ElevenLabs** | `eleven_v3`, `eleven_multilingual_v2`, `eleven_flash_v2_5`, `eleven_turbo_v2_5` | 70+ languages, voice cloning, stability/style controls | `ELEVENLABS_API_KEY` |
  72. #### Music Generation (Hands)
  73. | Provider | Models | Features | Env Var |
  74. |----------|--------|----------|---------|
  75. | **Minimax** | `music-2.5` | Vocals from lyrics, structure tags, MP3/WAV | `MINIMAX_API_KEY` |
  76. | **ElevenLabs** | Music API | Instrumental mode, 3s-10min, prompt-based | `ELEVENLABS_API_KEY` |
  77. #### Sound Effects (Hands)
  78. | Provider | Models | Features | Env Var |
  79. |----------|--------|----------|---------|
  80. | **ElevenLabs** | Sound Generation API | Text-to-SFX, 0.5-30s, looping, prompt influence | `ELEVENLABS_API_KEY` |
  81. #### Provider Configuration
  82. ```bash
  83. # Default provider per capability (optional, all default to "gemini")
  84. SPEECH_PROVIDER=gemini # Options: gemini, minimax, elevenlabs
  85. VIDEO_PROVIDER=gemini # Options: gemini, minimax, zhipuai
  86. VISION_PROVIDER=gemini # Options: gemini, zhipuai
  87. IMAGE_PROVIDER=gemini # Options: gemini, zhipuai
  88. ```
  89. Or override per request:
  90. ```json
  91. { "provider": "minimax", "prompt": "..." }
  92. ```
  93. ### Google Gemini Documentation
  94. - [Gemini API](https://ai.google.dev/gemini-api/docs?hl=en)
  95. - [Gemini Models](https://ai.google.dev/gemini-api/docs/models)
  96. - [Video Understanding](https://ai.google.dev/gemini-api/docs/video-understanding?hl=en)
  97. - [Image Understanding](https://ai.google.dev/gemini-api/docs/image-understanding)
  98. - [Document Understanding](https://ai.google.dev/gemini-api/docs/document-processing)
  99. - [Audio Understanding](https://ai.google.dev/gemini-api/docs/audio)
  100. - [Speech Generation](https://ai.google.dev/gemini-api/docs/speech-generation)
  101. - [Image Generation](https://ai.google.dev/gemini-api/docs/image-generation)
  102. - [Video Generation](https://ai.google.dev/gemini-api/docs/video)
  103. ## Quick Start
  104. ### Getting Your Google Gemini API Key
  105. Before installation, you'll need a Google Gemini API key to enable visual analysis capabilities.
  106. #### Step 1: Access Google AI Studio
  107. 1. Visit [Google AI Studio](https://aistudio.google.com/) in your web browser
  108. 2. Sign in with your Google account (create one if needed)
  109. 3. Accept the terms of service when prompted
  110. #### Step 2: Create an API Key
  111. 1. In the Google AI Studio interface, look for the "Get API Key" button or navigate to the API keys section
  112. 2. Click "Create API key" or "Generate API key"
  113. 3. Choose "Create API key in new project" (recommended) or select an existing Google Cloud project
  114. 4. Your API key will be generated and displayed
  115. 5. **Important**: Copy the API key immediately as it may not be shown again
  116. #### Step 3: Secure Your API Key
  117. ⚠️ **Security Warning**: Treat your API key like a password. Never share it publicly or commit it to version control.
  118. **Best Practices:**
  119. - Store the key in environment variables (not in code)
  120. - Don't include it in screenshots or documentation
  121. - Regenerate the key if accidentally exposed
  122. - Set usage quotas and monitoring in Google Cloud Console
  123. - Restrict API key usage to specific services if possible
  124. #### Step 4: Set Up Environment Variable
  125. Configure your API key using one of these methods:
  126. **Method 1: Shell Environment (Recommended)**
  127. ```bash
  128. # Add to your shell profile (.bashrc, .zshrc, .bash_profile)
  129. export GOOGLE_GEMINI_API_KEY="your_api_key_here"
  130. # Reload your shell configuration
  131. source ~/.zshrc # or ~/.bashrc
  132. ```
  133. **Method 2: Project-specific .env File**
  134. ```bash
  135. # Create a .env file in your project directory
  136. echo "GOOGLE_GEMINI_API_KEY=your_api_key_here" > .env
  137. # Add .env to your .gitignore file
  138. echo ".env" >> .gitignore
  139. ```
  140. **Method 3: MCP Client Configuration**
  141. You can also provide the API key directly in your MCP client configuration (shown in setup examples below).
  142. #### Step 5: Verify API Access
  143. Test your API key works correctly:
  144. ```bash
  145. # Test with curl (optional verification)
  146. curl -H "Content-Type: application/json" \
  147. -d '{"contents":[{"parts":[{"text":"Hello"}]}]}' \
  148. -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent?key=YOUR_API_KEY"
  149. ```
  150. ### Choosing Your Gemini Provider
  151. Human MCP supports two ways to access Google's Gemini models:
  152. #### Option 1: Google AI Studio (Default - Recommended for Getting Started)
  153. **Best for:** Quick start, development, prototyping
  154. **Setup:**
  155. 1. Get your API key from [Google AI Studio](https://aistudio.google.com/)
  156. 2. Set environment variable: `export GOOGLE_GEMINI_API_KEY=your_api_key`
  157. **Pros:**
  158. - Simple setup (just one API key)
  159. - No GCP account required
  160. - Free tier available
  161. - Perfect for development and testing
  162. #### Option 2: Vertex AI (Recommended for Production)
  163. **Best for:** Production deployments, enterprise use, GCP integration
  164. **Setup:**
  165. 1. Create a GCP project and enable Vertex AI API
  166. 2. Set up authentication:
  167. ```bash
  168. # Option A: Application Default Credentials (for local dev)
  169. gcloud auth application-default login
  170. # Option B: Service Account (for production)
  171. export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
  172. ```
  173. 3. Set environment variables:
  174. ```bash
  175. export USE_VERTEX=1
  176. export VERTEX_PROJECT_ID=your-gcp-project-id
  177. export VERTEX_LOCATION=us-central1 # optional, defaults to us-central1
  178. ```
  179. **Pros:**
  180. - Enterprise-grade quotas and SLAs
  181. - Better integration with GCP services
  182. - Advanced IAM and security controls
  183. - Usage tracking via Cloud Console
  184. - Better for production workloads
  185. **Configuration Example (Claude Desktop):**
  186. ```json
  187. {
  188. "mcpServers": {
  189. "human-mcp-vertex": {
  190. "command": "npx",
  191. "args": ["@goonnguyen/human-mcp"],
  192. "env": {
  193. "USE_VERTEX": "1",
  194. "VERTEX_PROJECT_ID": "your-gcp-project-id",
  195. "VERTEX_LOCATION": "us-central1"
  196. }
  197. }
  198. }
  199. }
  200. ```
  201. **Note:** With Vertex AI, you don't need `GOOGLE_GEMINI_API_KEY` - authentication is handled by GCP credentials.
  202. #### Vertex AI Authentication Methods
  203. **1. Application Default Credentials (ADC)** - Best for local development
  204. ```bash
  205. gcloud auth application-default login
  206. ```
  207. **2. Service Account** - Best for production
  208. ```bash
  209. # Download service account JSON from GCP Console
  210. export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
  211. ```
  212. **3. Workload Identity** - Best for GKE deployments
  213. - Automatically configured when running on GKE
  214. - No credentials file needed
  215. - Recommended for Kubernetes deployments
  216. **Troubleshooting Vertex AI:**
  217. If you encounter authentication errors:
  218. 1. Verify your GCP project ID: `gcloud config get-value project`
  219. 2. Check ADC status: `gcloud auth application-default print-access-token`
  220. 3. Ensure Vertex AI API is enabled: Visit [Vertex AI Console](https://console.cloud.google.com/vertex-ai)
  221. 4. Verify IAM permissions: Your account needs `Vertex AI User` role
  222. **Cost Considerations:**
  223. Both Google AI Studio and Vertex AI use the same Gemini models and pricing, but:
  224. - Google AI Studio: Generous free tier for testing
  225. - Vertex AI: Production-grade quotas, better for high-volume usage
  226. #### Alternative Methods for API Key
  227. **Using Google Cloud Console:**
  228. 1. Go to [Google Cloud Console](https://console.cloud.google.com/)
  229. 2. Create a new project or select existing one
  230. 3. Enable the "Generative AI API"
  231. 4. Go to "Credentials" > "Create Credentials" > "API Key"
  232. 5. Optionally restrict the key to specific APIs and IPs
  233. **API Key Restrictions (Recommended):**
  234. - Restrict to "Generative AI API" only
  235. - Set IP restrictions if using from specific locations
  236. - Configure usage quotas to prevent unexpected charges
  237. - Enable API key monitoring and alerts
  238. #### Troubleshooting API Key Issues
  239. **Common Problems:**
  240. - **Invalid API Key**: Ensure you copied the complete key without extra spaces
  241. - **API Not Enabled**: Make sure Generative AI API is enabled in your Google Cloud project
  242. - **Quota Exceeded**: Check your usage limits in Google Cloud Console
  243. - **Authentication Errors**: Verify the key hasn't expired or been revoked
  244. **Testing Your Setup:**
  245. ```bash
  246. # Verify environment variable is set
  247. echo $GOOGLE_GEMINI_API_KEY
  248. # Should output your API key (first few characters)
  249. ```
  250. ### Prerequisites
  251. - Node.js v22+ or [Bun](https://bun.sh) v1.2+
  252. - Google [Gemini API key](http://aistudio.google.com/) (configured as shown above)
  253. ### Development (For Contributors)
  254. If you want to contribute to Human MCP development:
  255. ```bash
  256. # Clone the repository
  257. git clone https://github.com/human-mcp/human-mcp.git
  258. cd human-mcp
  259. # Install dependencies
  260. bun install
  261. # Copy environment template
  262. cp .env.example .env
  263. # Add your Gemini API key to .env
  264. GOOGLE_GEMINI_API_KEY=your_api_key_here
  265. # Start development server
  266. bun run dev
  267. # Build for production
  268. bun run build
  269. # Run tests
  270. bun test
  271. # Type checking
  272. bun run typecheck
  273. ```
  274. ### Usage with MCP Clients
  275. Human MCP can be configured with various MCP clients for different development workflows. Follow the setup instructions for your preferred client below.
  276. #### Claude Desktop
  277. Claude Desktop is a desktop application that provides a user-friendly interface for interacting with MCP servers.
  278. **Configuration Location:**
  279. - **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`
  280. - **Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
  281. - **Linux**: `~/.config/Claude/claude_desktop_config.json`
  282. **Configuration Example:**
  283. ```json
  284. {
  285. "mcpServers": {
  286. "human-mcp": {
  287. "command": "npx",
  288. "args": ["@goonnguyen/human-mcp"],
  289. "env": {
  290. "GOOGLE_GEMINI_API_KEY": "your_gemini_api_key_here"
  291. }
  292. }
  293. }
  294. }
  295. ```
  296. #### Claude Code (CLI)
  297. Claude Code is the official CLI for Claude that supports MCP servers for enhanced coding workflows.
  298. **Prerequisites:**
  299. - Node.js v22+ or Bun v1.2+
  300. - Google Gemini API key
  301. - Claude Code CLI installed
  302. **Configuration Methods:**
  303. Claude Code offers multiple ways to configure MCP servers. Choose the method that best fits your workflow:
  304. **Method 1: Using Claude Code CLI (Recommended)**
  305. ```bash
  306. # Add Human MCP server with automatic configuration
  307. claude mcp add --scope user human-mcp npx @goonnguyen/human-mcp --env GOOGLE_GEMINI_API_KEY=your_api_key_here
  308. # Alternative: Add locally installed version
  309. claude mcp add --scope project human-mcp npx @goonnguyen/human-mcp --env GOOGLE_GEMINI_API_KEY=your_api_key_here
  310. # List configured MCP servers
  311. claude mcp list
  312. # Remove server if needed
  313. claude mcp remove human-mcp
  314. ```
  315. **Method 2: Manual JSON Configuration**
  316. **Configuration Location:**
  317. - **All platforms**: `~/.config/claude/config.json`
  318. **Configuration Example:**
  319. ```json
  320. {
  321. "mcpServers": {
  322. "human-mcp": {
  323. "command": "npx",
  324. "args": ["@goonnguyen/human-mcp"],
  325. "env": {
  326. "GOOGLE_GEMINI_API_KEY": "your_gemini_api_key_here",
  327. "LOG_LEVEL": "info",
  328. "MCP_TIMEOUT": "30000"
  329. }
  330. }
  331. }
  332. }
  333. ```
  334. **Configuration Scopes:**
  335. Claude Code supports different configuration scopes:
  336. - **User Scope** (`--scope user`): Available across all projects (default)
  337. - **Project Scope** (`--scope project`): Shared via `.mcp.json`, checked into version control
  338. - **Local Scope** (`--scope local`): Private to current project only
  339. ```bash
  340. # Project-wide configuration (team sharing)
  341. claude mcp add --scope project human-mcp npx @goonnguyen/human-mcp --env GOOGLE_GEMINI_API_KEY=your_api_key_here
  342. # Local project configuration (private)
  343. claude mcp add --scope local human-mcp npx @goonnguyen/human-mcp --env GOOGLE_GEMINI_API_KEY=your_api_key_here
  344. ```
  345. **Setup Steps:**
  346. 1. Install Claude Code CLI: `npm install -g @anthropic-ai/claude-code`
  347. 2. Configure your Google Gemini API key (see Environment Setup section)
  348. 3. Add Human MCP server using CLI or manual configuration
  349. 4. Verify configuration: `claude mcp list`
  350. **Verification:**
  351. ```bash
  352. # List all configured MCP servers
  353. claude mcp list
  354. # Test Human MCP connection
  355. claude mcp test human-mcp
  356. # Start Claude with MCP servers enabled
  357. claude --enable-mcp
  358. # Check server logs for debugging
  359. claude mcp logs human-mcp
  360. ```
  361. **Usage Examples:**
  362. ```bash
  363. # Start Claude Code with MCP servers enabled
  364. claude --enable-mcp
  365. # Analyze a screenshot in your current project
  366. claude "Analyze this screenshot for UI issues" --attach screenshot.png
  367. # Use Human MCP tools in conversation
  368. claude "Use eyes_analyze to check this UI screenshot for accessibility issues"
  369. # Pass additional arguments to the MCP server
  370. claude -- --server-arg value "Analyze this image"
  371. ```
  372. **Windows-Specific Configuration:**
  373. For Windows users, wrap `npx` commands with `cmd /c`:
  374. ```bash
  375. # Windows configuration
  376. claude mcp add --scope user human-mcp cmd /c npx @goonnguyen/human-mcp --env GOOGLE_GEMINI_API_KEY=your_api_key_here
  377. ```
  378. Or via JSON configuration:
  379. ```json
  380. {
  381. "mcpServers": {
  382. "human-mcp": {
  383. "command": "cmd",
  384. "args": ["/c", "npx", "@goonnguyen/human-mcp"],
  385. "env": {
  386. "GOOGLE_GEMINI_API_KEY": "your_gemini_api_key_here"
  387. }
  388. }
  389. }
  390. }
  391. ```
  392. #### OpenCode
  393. OpenCode is a powerful AI coding agent that supports MCP servers for enhanced capabilities. Use Human MCP to add visual analysis tools to your OpenCode workflow.
  394. **Configuration Location:**
  395. - **Global**: `~/.config/opencode/opencode.json`
  396. - **Project**: `./opencode.json` in your project root
  397. **Configuration Example (STDIO - Recommended):**
  398. ```json
  399. {
  400. "$schema": "https://opencode.ai/config.json",
  401. "mcp": {
  402. "human": {
  403. "type": "local",
  404. "command": ["npx", "@goonnguyen/human-mcp"],
  405. "enabled": true,
  406. "environment": {
  407. "GOOGLE_GEMINI_API_KEY": "your_gemini_api_key_here",
  408. "TRANSPORT_TYPE": "stdio",
  409. "LOG_LEVEL": "info"
  410. }
  411. }
  412. }
  413. }
  414. ```
  415. **Setup Steps:**
  416. 1. Configure your Google Gemini API key (see Environment Setup section)
  417. 2. Add Human MCP server using CLI or manual configuration
  418. 3. Verify configuration: `claude mcp list`
  419. **Important Notes:**
  420. - **STDIO Mode**: Human MCP uses stdio transport by default, which provides the best compatibility with OpenCode
  421. - **No R2 Uploads**: In stdio mode, all images and videos are processed locally and sent to Gemini using inline base64 - no Cloudflare R2 uploads occur
  422. - **Security**: Never commit API keys to version control. Use environment variables or secure credential storage
  423. **Verification:**
  424. - Check OpenCode logs for successful MCP connection
  425. - Try using `eyes_analyze` tool: "Analyze this screenshot for UI issues"
  426. - Verify no external network calls to Cloudflare R2 in stdio mode
  427. #### Gemini CLI
  428. While Gemini CLI doesn't directly support MCP, you can use Human MCP as a bridge to access visual analysis capabilities.
  429. **Integration Example:**
  430. ```bash
  431. # Create a wrapper script for Gemini CLI integration
  432. #!/bin/bash
  433. # gemini-visual-analysis.sh
  434. # Set environment variables
  435. export GOOGLE_GEMINI_API_KEY="your_api_key"
  436. # Run Human MCP analysis
  437. echo '{"source": "'$1'", "type": "image", "analysis_type": "ui_debug"}' | \
  438. npx @goonnguyen/human-mcp
  439. ```
  440. #### MCP Coding Clients (Cline, Cursor, Windsurf)
  441. These IDE extensions support MCP servers for enhanced AI-assisted coding with visual analysis capabilities.
  442. ##### Cline (VS Code Extension)
  443. **Configuration Location:**
  444. - VS Code Settings: `.vscode/settings.json` in your workspace
  445. - Or Global Settings: VS Code → Preferences → Settings → Extensions → Cline
  446. **Configuration Example:**
  447. ```json
  448. {
  449. "cline.mcpServers": {
  450. "human-mcp": {
  451. "command": "npx",
  452. "args": ["@goonnguyen/human-mcp"],
  453. "env": {
  454. "GOOGLE_GEMINI_API_KEY": "your_gemini_api_key_here"
  455. }
  456. }
  457. }
  458. }
  459. ```
  460. **Setup Steps:**
  461. 1. Configure your Google Gemini API key (see Environment Setup section)
  462. 2. Add Human MCP server using CLI or manual configuration
  463. 3. Verify configuration: `claude mcp list`
  464. **Verification:**
  465. - Check Cline logs for successful MCP connection
  466. - Try using `eyes_analyze` tool: "Analyze this screenshot for UI issues"
  467. ##### Cursor
  468. **Configuration Location:**
  469. - Cursor Settings: `.cursor/settings.json` in your workspace
  470. - Or via Cursor → Settings → Extensions → MCP
  471. **Configuration Example:**
  472. ```json
  473. {
  474. "mcp.servers": {
  475. "human-mcp": {
  476. "command": "npx",
  477. "args": ["@goonnguyen/human-mcp"],
  478. "env": {
  479. "GOOGLE_GEMINI_API_KEY": "your_gemini_api_key_here"
  480. }
  481. }
  482. }
  483. }
  484. ```
  485. **Setup Steps:**
  486. 1. Install latest version of Cursor
  487. 2. Configure your Google Gemini API key (see Environment Setup section)
  488. 3. Add Human MCP server using CLI or manual configuration
  489. 4. Configure MCP servers in settings (use `npx` version for reliability)
  490. 5. Enable MCP integration in Cursor preferences
  491. 6. Test visual analysis features
  492. ##### Windsurf
  493. **Configuration Location:**
  494. - Windsurf config: `~/.windsurf/mcp_servers.json`
  495. - Or project-specific: `.windsurf/mcp_servers.json`
  496. **Configuration Example:**
  497. ```json
  498. {
  499. "servers": {
  500. "human-mcp": {
  501. "command": "npx",
  502. "args": ["@goonnguyen/human-mcp"],
  503. "env": {
  504. "GOOGLE_GEMINI_API_KEY": "your_gemini_api_key_here"
  505. },
  506. "timeout": 30000
  507. }
  508. }
  509. }
  510. ```
  511. **Setup Steps:**
  512. 1. Install Windsurf IDE
  513. 2. Configure your Google Gemini API key (see Environment Setup section)
  514. 3. Add Human MCP server using CLI or manual configuration
  515. 4. Restart Windsurf
  516. 5. Verify connection in MCP panel
  517. ### Environment Variable Setup
  518. For all clients, ensure your Google Gemini API key is properly configured:
  519. **Option 1: System Environment Variables (Recommended)**
  520. ```bash
  521. # Add to your shell profile (.bashrc, .zshrc, etc.)
  522. export GOOGLE_GEMINI_API_KEY="your_api_key_here"
  523. # Reload your shell
  524. source ~/.zshrc # or ~/.bashrc
  525. ```
  526. **Option 2: Client Configuration**
  527. Include the API key directly in the MCP server configuration (as shown in examples above). This is the most reliable method for ensuring the key is available to the Human MCP server.
  528. **Option 3: Global .env File (Advanced)**
  529. ```bash
  530. # Create a global .env file (optional)
  531. echo "GOOGLE_GEMINI_API_KEY=your_api_key_here" >> ~/.env
  532. # Source it in your shell profile
  533. echo "source ~/.env" >> ~/.zshrc
  534. ```
  535. ### Connection Verification
  536. **Test Human MCP Server:**
  537. ```bash
  538. # For development/testing, use the MCP inspector from source
  539. # (only if you have cloned the repository for development)
  540. cd /path/to/human-mcp && bun run inspector
  541. ```
  542. **Test with MCP Clients:**
  543. 1. Check client logs for connection status
  544. 2. Try using `eyes_analyze` tool with a test image
  545. 3. Verify API responses are returned correctly
  546. 4. Look for the Human MCP server in the client's MCP server list
  547. ### Troubleshooting
  548. **Common Issues:**
  549. 1. **Connection Failed**
  550. - Verify Node.js/npm or Bun is installed and accessible
  551. - Ensure `@goonnguyen/human-mcp` package is installed in the client configuration
  552. - Check Google Gemini API key is valid and properly configured
  553. 2. **Tool Not Found**
  554. - Restart the MCP client after configuration changes
  555. - Check Human MCP server logs for errors
  556. - Verify the server starts: `npx @goonnguyen/human-mcp`
  557. 3. **API Errors**
  558. - Validate Google Gemini API key
  559. - Check API quota and usage limits
  560. - Review network connectivity and firewall settings
  561. 4. **Permission Errors**
  562. - Check npm global installation permissions
  563. - Use `npx` instead of global installation if needed
  564. - Verify API key has necessary permissions
  565. **Getting Help:**
  566. - Check [Human MCP Issues](https://github.com/human-mcp/human-mcp/issues)
  567. - Review client-specific MCP documentation
  568. - Test package installation: `npx @goonnguyen/human-mcp --help`
  569. ## HTTP Transport & Local Files
  570. ### Overview
  571. Human MCP supports HTTP transport mode for clients like Claude Desktop that require HTTP-based communication instead of stdio. When using HTTP transport with local files, the server automatically handles file uploading to ensure compatibility.
  572. ### Using Local Files with HTTP Transport
  573. When Claude Desktop or other HTTP transport clients access local files, they often use virtual paths like `/mnt/user-data/uploads/file.png`. The Human MCP server automatically detects these paths and uploads files to Cloudflare R2 for processing.
  574. #### Automatic Upload (Default Behavior)
  575. When you provide a local file path, the server automatically:
  576. 1. Detects the local file path or Claude Desktop virtual path
  577. 2. Uploads it to Cloudflare R2 (if configured)
  578. 3. Returns the CDN URL for processing
  579. 4. Uses the fast Cloudflare CDN for delivery
  580. #### Manual Upload Options
  581. ##### Option 1: Upload File Directly
  582. ```bash
  583. # Upload file to Cloudflare R2 and get CDN URL
  584. curl -X POST http://localhost:3000/mcp/upload \
  585. -F "file=@/path/to/image.png" \
  586. -H "Authorization: Bearer your_secret"
  587. # Response:
  588. {
  589. "result": {
  590. "success": true,
  591. "url": "https://cdn.example.com/human-mcp/abc123.png",
  592. "originalName": "image.png",
  593. "size": 102400,
  594. "mimeType": "image/png"
  595. }
  596. }
  597. ```
  598. ##### Option 2: Upload Base64 Data
  599. ```bash
  600. # Upload base64 data to Cloudflare R2
  601. curl -X POST http://localhost:3000/mcp/upload-base64 \
  602. -H "Content-Type: application/json" \
  603. -H "Authorization: Bearer your_secret" \
  604. -d '{
  605. "data": "iVBORw0KGgoAAAANSUhEUgA...",
  606. "mimeType": "image/png",
  607. "filename": "screenshot.png"
  608. }'
  609. ```
  610. ##### Option 3: Use Existing CDN URLs
  611. If your files are already hosted, use the public URL directly:
  612. - Cloudflare R2: `https://cdn.example.com/path/to/file.jpg`
  613. - Other CDNs: Any publicly accessible URL
  614. ### Cloudflare R2 Configuration
  615. #### Required Environment Variables
  616. Add these to your `.env` file:
  617. ```env
  618. # Cloudflare R2 Storage Configuration
  619. CLOUDFLARE_CDN_PROJECT_NAME=human-mcp
  620. CLOUDFLARE_CDN_BUCKET_NAME=your-bucket-name
  621. CLOUDFLARE_CDN_ACCESS_KEY=your_access_key
  622. CLOUDFLARE_CDN_SECRET_KEY=your_secret_key
  623. CLOUDFLARE_CDN_ENDPOINT_URL=https://your-account-id.r2.cloudflarestorage.com
  624. CLOUDFLARE_CDN_BASE_URL=https://cdn.example.com
  625. ```
  626. #### Setting up Cloudflare R2
  627. 1. **Create Cloudflare Account**: Sign up at [cloudflare.com](https://cloudflare.com)
  628. 2. **Enable R2 Storage**: Go to R2 Object Storage in your Cloudflare dashboard
  629. 3. **Create a Bucket**:
  630. - Name: `your-bucket-name`
  631. - Location: Choose based on your needs
  632. 4. **Generate API Credentials**:
  633. - Go to "Manage R2 API Tokens"
  634. - Create token with R2:Object:Write permissions
  635. - Copy the access key and secret key
  636. 5. **Set up Custom Domain** (Optional):
  637. - Add custom domain to your R2 bucket
  638. - Update `CLOUDFLARE_CDN_BASE_URL` with your domain
  639. #### Claude Desktop HTTP Configuration
  640. For Claude Desktop with HTTP transport and automatic file uploads:
  641. ```json
  642. {
  643. "mcpServers": {
  644. "human-mcp-http": {
  645. "command": "node",
  646. "args": ["path/to/http-wrapper.js"],
  647. "env": {
  648. "GOOGLE_GEMINI_API_KEY": "your_key",
  649. "TRANSPORT_TYPE": "http",
  650. "HTTP_PORT": "3000",
  651. "CLOUDFLARE_CDN_BUCKET_NAME": "your-bucket",
  652. "CLOUDFLARE_CDN_ACCESS_KEY": "your-access-key",
  653. "CLOUDFLARE_CDN_SECRET_KEY": "your-secret-key",
  654. "CLOUDFLARE_CDN_ENDPOINT_URL": "https://account.r2.cloudflarestorage.com",
  655. "CLOUDFLARE_CDN_BASE_URL": "https://cdn.example.com"
  656. }
  657. }
  658. }
  659. }
  660. ```
  661. ### Benefits of Cloudflare R2 Integration
  662. - **Fast Global Delivery**: Files served from Cloudflare's 300+ edge locations
  663. - **Automatic Handling**: No manual conversion needed for local files
  664. - **Large File Support**: Handle files up to 100MB
  665. - **Persistent URLs**: Files remain accessible for future reference
  666. - **Cost Effective**: Cloudflare R2 offers competitive pricing with no egress fees
  667. - **Enhanced Security**: Files isolated from server filesystem
  668. ### Alternative Solutions
  669. #### Using stdio Transport
  670. For users who need direct local file access without cloud uploads:
  671. ```json
  672. {
  673. "mcpServers": {
  674. "human-mcp": {
  675. "command": "npx",
  676. "args": ["@goonnguyen/human-mcp"],
  677. "env": {
  678. "GOOGLE_GEMINI_API_KEY": "key",
  679. "TRANSPORT_TYPE": "stdio"
  680. }
  681. }
  682. }
  683. }
  684. ```
  685. #### Pre-uploading Files
  686. Batch upload files using the upload endpoints:
  687. ```bash
  688. #!/bin/bash
  689. # Upload script
  690. for file in *.png; do
  691. curl -X POST http://localhost:3000/mcp/upload \
  692. -F "file=@$file" \
  693. -H "Authorization: Bearer $MCP_SECRET"
  694. done
  695. ```
  696. ## MCP Tools Reference
  697. ### 👁️ Eyes Tools (Visual Analysis & Document Processing)
  698. **eyes_analyze** - Analyze images, videos, and GIFs
  699. ```json
  700. {
  701. "source": "path/to/image.png or URL",
  702. "focus": "What to analyze (optional)",
  703. "detail": "quick or detailed (default: detailed)"
  704. }
  705. ```
  706. **eyes_compare** - Compare two images
  707. ```json
  708. {
  709. "image1": "path/to/first.png",
  710. "image2": "path/to/second.png",
  711. "focus": "differences, similarities, layout, or content"
  712. }
  713. ```
  714. **eyes_read_document** - Extract content from documents
  715. ```json
  716. {
  717. "document": "path/to/document.pdf",
  718. "pages": "1-5 or all (default: all)",
  719. "extract": "text, tables, or both (default: both)"
  720. }
  721. ```
  722. **eyes_summarize_document** - Summarize documents
  723. ```json
  724. {
  725. "document": "path/to/document.pdf",
  726. "length": "brief, medium, or detailed",
  727. "focus": "Specific topics (optional)"
  728. }
  729. ```
  730. ### 🗣️ Mouth Tools (Speech Generation)
  731. **mouth_speak** - Text to speech
  732. ```json
  733. {
  734. "text": "Your text here (max 32k tokens)",
  735. "voice": "Zephyr (or 30+ other voices)",
  736. "language": "en-US (or 24 languages)",
  737. "style_prompt": "Speaking style description (optional)"
  738. }
  739. ```
  740. **mouth_narrate** - Long-form narration
  741. ```json
  742. {
  743. "content": "Long content to narrate",
  744. "voice": "Sage",
  745. "narration_style": "professional, casual, educational, or storytelling",
  746. "chapter_breaks": true
  747. }
  748. ```
  749. **mouth_explain** - Code explanation
  750. ```json
  751. {
  752. "code": "function example() {}",
  753. "programming_language": "javascript",
  754. "voice": "Apollo",
  755. "explanation_level": "beginner, intermediate, or advanced"
  756. }
  757. ```
  758. **mouth_customize** - Voice testing
  759. ```json
  760. {
  761. "text": "Test sample",
  762. "voice": "Charon",
  763. "style_variations": ["professional", "casual"],
  764. "compare_voices": ["Puck", "Sage"]
  765. }
  766. ```
  767. ### ✋ Hands Tools (Content Generation & Image Editing)
  768. #### Image Generation (1 tool)
  769. **gemini_gen_image** - Generate images from text
  770. ```json
  771. {
  772. "prompt": "A modern minimalist login form",
  773. "style": "photorealistic, artistic, cartoon, sketch, or digital_art",
  774. "aspect_ratio": "1:1, 16:9, 9:16, 4:3, or 3:4",
  775. "negative_prompt": "What to avoid (optional)"
  776. }
  777. ```
  778. #### Video Generation (2 tools)
  779. **gemini_gen_video** - Generate videos from text
  780. ```json
  781. {
  782. "prompt": "Mountain landscape at sunrise",
  783. "duration": "4s, 8s, or 12s",
  784. "style": "realistic, cinematic, artistic, cartoon, or animation",
  785. "camera_movement": "static, pan_left, pan_right, zoom_in, zoom_out, dolly_forward, dolly_backward",
  786. "fps": 24
  787. }
  788. ```
  789. **gemini_image_to_video** - Animate images
  790. ```json
  791. {
  792. "prompt": "Animate with flowing water",
  793. "image_input": "base64 or URL",
  794. "duration": "8s",
  795. "camera_movement": "zoom_in"
  796. }
  797. ```
  798. #### Music Generation (2 tools)
  799. **minimax_gen_music** - Generate music with vocals (Minimax Music 2.5)
  800. ```json
  801. {
  802. "prompt": "Upbeat pop song about summer",
  803. "lyrics": "Optional lyrics text",
  804. "duration": 240
  805. }
  806. ```
  807. **elevenlabs_gen_music** - Generate music (ElevenLabs Music API)
  808. ```json
  809. {
  810. "text": "Calm ambient music for meditation",
  811. "duration_seconds": 120,
  812. "prompt_influence": 0.7
  813. }
  814. ```
  815. #### Sound Effects Generation (1 tool)
  816. **elevenlabs_gen_sfx** - Generate sound effects from text (ElevenLabs Sound Generation API)
  817. ```json
  818. {
  819. "text": "Door creaking open slowly",
  820. "duration_seconds": 5.0,
  821. "prompt_influence": 0.5
  822. }
  823. ```
  824. #### AI Image Editing (5 tools)
  825. **gemini_edit_image** - Comprehensive AI editing (5 operations: inpaint, outpaint, style_transfer, object_manipulation, multi_image_compose)
  826. **gemini_inpaint_image** - Add/modify areas with text (no mask required)
  827. ```json
  828. {
  829. "input_image": "base64 or path",
  830. "prompt": "What to add/change",
  831. "mask_prompt": "Where to edit (optional)"
  832. }
  833. ```
  834. **gemini_outpaint_image** - Expand image borders
  835. ```json
  836. {
  837. "input_image": "base64 or path",
  838. "prompt": "What to add in expanded area",
  839. "expand_direction": "all, left, right, top, bottom, horizontal, vertical",
  840. "expansion_ratio": 1.5
  841. }
  842. ```
  843. **gemini_style_transfer_image** - Apply artistic styles
  844. ```json
  845. {
  846. "input_image": "base64 or path",
  847. "prompt": "Desired style",
  848. "style_image": "Reference image (optional)",
  849. "style_strength": 0.7
  850. }
  851. ```
  852. **gemini_compose_images** - Combine multiple images
  853. ```json
  854. {
  855. "input_image": "Primary image",
  856. "secondary_images": ["image1", "image2"],
  857. "prompt": "How to compose",
  858. "composition_layout": "blend, collage, overlay, side_by_side"
  859. }
  860. ```
  861. #### Jimp Processing (4 tools - Local, Fast)
  862. **jimp_crop_image** - Crop images (6 modes)
  863. ```json
  864. {
  865. "input_image": "path or URL",
  866. "mode": "manual, center, top_left, aspect_ratio",
  867. "width": 800,
  868. "height": 600
  869. }
  870. ```
  871. **jimp_resize_image** - Resize images (5 algorithms)
  872. ```json
  873. {
  874. "input_image": "path or URL",
  875. "width": 1920,
  876. "algorithm": "bilinear, bicubic, nearestNeighbor",
  877. "maintain_aspect_ratio": true
  878. }
  879. ```
  880. **jimp_rotate_image** - Rotate images
  881. ```json
  882. {
  883. "input_image": "path or URL",
  884. "angle": 90,
  885. "background_color": "#ffffff"
  886. }
  887. ```
  888. **jimp_mask_image** - Apply grayscale masks
  889. ```json
  890. {
  891. "input_image": "path or URL",
  892. "mask_image": "path or URL (black=transparent, white=opaque)"
  893. }
  894. ```
  895. #### Background Removal (1 tool)
  896. **rmbg_remove_background** - AI background removal (3 quality levels: fast, balanced, high)
  897. ```json
  898. {
  899. "input_image": "path or URL",
  900. "quality": "fast, balanced, or high",
  901. "output_format": "png or jpeg"
  902. }
  903. ```
  904. #### Browser Automation (3 tools)
  905. **playwright_screenshot_fullpage** - Capture full page including scrollable content
  906. ```json
  907. {
  908. "url": "https://example.com",
  909. "format": "png or jpeg",
  910. "quality": 80,
  911. "timeout": 30000,
  912. "wait_until": "load, domcontentloaded, or networkidle",
  913. "viewport": { "width": 1920, "height": 1080 }
  914. }
  915. ```
  916. **playwright_screenshot_viewport** - Capture visible viewport area only
  917. ```json
  918. {
  919. "url": "https://example.com",
  920. "format": "png or jpeg",
  921. "quality": 80,
  922. "timeout": 30000,
  923. "wait_until": "networkidle",
  924. "viewport": { "width": 1920, "height": 1080 }
  925. }
  926. ```
  927. **playwright_screenshot_element** - Capture specific element on page
  928. ```json
  929. {
  930. "url": "https://example.com",
  931. "selector": ".main-content or 'Click me' or 'button'",
  932. "selector_type": "css, text, or role",
  933. "format": "png or jpeg",
  934. "timeout": 30000,
  935. "wait_for_selector": true
  936. }
  937. ```
  938. ### 🧠 Brain Tools (Advanced Reasoning)
  939. **mcp__reasoning__sequentialthinking** - Native sequential thinking with thought revision
  940. ```json
  941. {
  942. "problem": "Complex issue description",
  943. "thought": "Current thinking step",
  944. "thoughtNumber": 1,
  945. "totalThoughts": 5,
  946. "nextThoughtNeeded": true,
  947. "isRevision": false
  948. }
  949. ```
  950. **brain_analyze_simple** - Fast pattern-based analysis
  951. ```json
  952. {
  953. "problem": "Issue to analyze",
  954. "pattern": "problem_solving, root_cause, pros_cons, swot, or cause_effect",
  955. "context": "Additional background (optional)"
  956. }
  957. ```
  958. **brain_patterns_info** - List reasoning patterns
  959. ```json
  960. {
  961. "pattern": "Specific pattern name (optional)"
  962. }
  963. ```
  964. **brain_reflect_enhanced** - AI-powered meta-cognitive reflection
  965. ```json
  966. {
  967. "originalAnalysis": "Previous analysis to reflect on",
  968. "focusAreas": ["assumptions", "logic_gaps", "alternative_approaches"],
  969. "improvementGoal": "What to improve (optional)",
  970. "detailLevel": "concise or detailed"
  971. }
  972. ```
  973. ## Example Use Cases
  974. ### Debugging UI Issues
  975. ```bash
  976. # Analyze a screenshot for layout problems
  977. {
  978. "source": "broken-layout.png",
  979. "type": "image",
  980. "analysis_type": "ui_debug"
  981. }
  982. ```
  983. ### Error Investigation
  984. ```bash
  985. # Analyze screen recording of an error
  986. {
  987. "source": "error-recording.mp4",
  988. "type": "video",
  989. "analysis_type": "error_detection"
  990. }
  991. ```
  992. ### Accessibility Audit
  993. ```bash
  994. # Check accessibility compliance
  995. {
  996. "source": "page-screenshot.png",
  997. "type": "image",
  998. "analysis_type": "accessibility",
  999. "check_accessibility": true
  1000. }
  1001. ```
  1002. ### Image Generation for Design
  1003. ```bash
  1004. # Generate UI mockups and design elements
  1005. {
  1006. "prompt": "Professional dashboard interface with data visualization charts",
  1007. "style": "digital_art",
  1008. "aspect_ratio": "16:9"
  1009. }
  1010. ```
  1011. ### Prototype Creation
  1012. ```bash
  1013. # Create visual prototypes for development
  1014. {
  1015. "prompt": "Mobile app login screen with modern design, dark theme",
  1016. "style": "photorealistic",
  1017. "aspect_ratio": "9:16",
  1018. "negative_prompt": "old-fashioned, bright colors"
  1019. }
  1020. ```
  1021. ### Video Generation for Prototyping
  1022. ```bash
  1023. # Create animated prototypes and demonstrations
  1024. {
  1025. "prompt": "User interface animation showing a smooth login process with form transitions",
  1026. "duration": "8s",
  1027. "style": "digital_art",
  1028. "aspect_ratio": "16:9",
  1029. "camera_movement": "static",
  1030. "fps": 30
  1031. }
  1032. ```
  1033. ### Marketing Video Creation
  1034. ```bash
  1035. # Generate promotional videos for products
  1036. {
  1037. "prompt": "Elegant product showcase video with professional lighting and smooth camera movement",
  1038. "duration": "12s",
  1039. "style": "cinematic",
  1040. "aspect_ratio": "16:9",
  1041. "camera_movement": "dolly_forward"
  1042. }
  1043. ```
  1044. ### Code Explanation Audio
  1045. ```bash
  1046. # Generate spoken explanations for code reviews
  1047. {
  1048. "code": "const useAuth = () => { const [user, setUser] = useState(null); return { user, login: setUser }; }",
  1049. "programming_language": "javascript",
  1050. "voice": "Apollo",
  1051. "explanation_level": "advanced",
  1052. "include_examples": true
  1053. }
  1054. ```
  1055. ### Documentation Narration
  1056. ```bash
  1057. # Convert technical documentation to audio
  1058. {
  1059. "content": "This API endpoint handles user authentication and returns a JWT token...",
  1060. "voice": "Sage",
  1061. "narration_style": "professional",
  1062. "chapter_breaks": true
  1063. }
  1064. ```
  1065. ### User Interface Voice Feedback
  1066. ```bash
  1067. # Generate voice responses for applications
  1068. {
  1069. "text": "File uploaded successfully. Processing will complete in approximately 30 seconds.",
  1070. "voice": "Kore",
  1071. "language": "en-US",
  1072. "style_prompt": "Speak in a helpful, reassuring tone"
  1073. }
  1074. ```
  1075. ### Advanced Problem Solving
  1076. ```bash
  1077. # Analyze complex technical issues with multi-step reasoning
  1078. {
  1079. "problem": "Database performance degradation in production environment",
  1080. "initial_thoughts": 8,
  1081. "allow_revision": true,
  1082. "enable_branching": true,
  1083. "thinking_style": "systematic"
  1084. }
  1085. ```
  1086. ### Architecture Decision Analysis
  1087. ```bash
  1088. # Deep analysis of system design decisions
  1089. {
  1090. "subject": "Microservices vs monolithic architecture for e-commerce platform",
  1091. "analysis_depth": "detailed",
  1092. "consider_alternatives": true,
  1093. "track_assumptions": true
  1094. }
  1095. ```
  1096. ### Hypothesis-Driven Debugging
  1097. ```bash
  1098. # Systematic problem solving with hypothesis testing
  1099. {
  1100. "problem_statement": "API response time increased by 300% after deployment",
  1101. "solution_approach": "scientific",
  1102. "verify_hypotheses": true,
  1103. "max_iterations": 15
  1104. }
  1105. ```
  1106. ### Code Review Reasoning
  1107. ```bash
  1108. # Reflect on code analysis and optimization approaches
  1109. {
  1110. "previous_analysis": "Initial code review findings",
  1111. "reflection_focus": ["performance_assumptions", "security_gaps", "maintainability"],
  1112. "optimize_process": true
  1113. }
  1114. ```
  1115. ### Automated Web Screenshots
  1116. ```bash
  1117. # Capture full page screenshot for documentation
  1118. {
  1119. "url": "https://example.com/dashboard",
  1120. "format": "png",
  1121. "wait_until": "networkidle",
  1122. "viewport": { "width": 1920, "height": 1080 }
  1123. }
  1124. ```
  1125. ### Element-Specific Screenshots
  1126. ```bash
  1127. # Capture specific UI component for bug reporting
  1128. {
  1129. "url": "https://example.com/app",
  1130. "selector": ".error-message",
  1131. "selector_type": "css",
  1132. "wait_for_selector": true,
  1133. "format": "png"
  1134. }
  1135. ```
  1136. ### Responsive Testing Screenshots
  1137. ```bash
  1138. # Capture mobile viewport for responsive design testing
  1139. {
  1140. "url": "https://example.com",
  1141. "format": "png",
  1142. "viewport": { "width": 375, "height": 812 },
  1143. "wait_until": "networkidle"
  1144. }
  1145. ```
  1146. ## Prompts
  1147. Human MCP includes pre-built prompts for common debugging scenarios:
  1148. - `debug_ui_screenshot` - Analyze UI screenshots for issues
  1149. - `analyze_error_recording` - Debug errors in screen recordings
  1150. - `accessibility_audit` - Perform accessibility audits
  1151. - `performance_visual_audit` - Analyze performance indicators
  1152. - `layout_comparison` - Compare layouts for differences
  1153. ## Resources
  1154. Access built-in documentation:
  1155. - `humanmcp://docs/api` - Complete API reference
  1156. - `humanmcp://examples/debugging` - Real-world debugging examples
  1157. ## Configuration
  1158. ### Transport Configuration
  1159. Human MCP supports multiple transport modes for maximum compatibility with different MCP clients:
  1160. #### Standard Mode (Default)
  1161. Uses modern Streamable HTTP transport with SSE notifications.
  1162. ```bash
  1163. # Transport configuration
  1164. TRANSPORT_TYPE=stdio # Options: stdio, http, both
  1165. HTTP_PORT=3000 # HTTP server port
  1166. HTTP_HOST=0.0.0.0 # HTTP server host
  1167. HTTP_SESSION_MODE=stateful # Options: stateful, stateless
  1168. HTTP_ENABLE_SSE=true # Enable SSE notifications
  1169. HTTP_ENABLE_JSON_RESPONSE=true # Enable JSON responses
  1170. ```
  1171. #### Legacy Client Support
  1172. For older MCP clients that only support the deprecated HTTP+SSE transport:
  1173. ```bash
  1174. # SSE Fallback configuration (for legacy clients)
  1175. HTTP_ENABLE_SSE_FALLBACK=true # Enable legacy SSE transport
  1176. HTTP_SSE_STREAM_PATH=/sse # SSE stream endpoint path
  1177. HTTP_SSE_MESSAGE_PATH=/messages # SSE message endpoint path
  1178. ```
  1179. When enabled, Human MCP provides isolated SSE fallback endpoints:
  1180. - **GET /sse** - Establishes SSE connection for legacy clients
  1181. - **POST /messages** - Handles incoming messages from legacy clients
  1182. **Important Notes:**
  1183. - SSE fallback is disabled by default following YAGNI principles
  1184. - Sessions are segregated between transport types to prevent mixing
  1185. - Modern clients should use the standard `/mcp` endpoints
  1186. - Legacy clients use separate `/sse` and `/messages` endpoints
  1187. ### Environment Variables
  1188. ```bash
  1189. # Required
  1190. GOOGLE_GEMINI_API_KEY=your_api_key
  1191. # Optional Provider API Keys
  1192. MINIMAX_API_KEY=your_minimax_api_key
  1193. ZHIPUAI_API_KEY=your_zhipuai_api_key
  1194. ELEVENLABS_API_KEY=your_elevenlabs_api_key
  1195. # Optional Core Configuration
  1196. GOOGLE_GEMINI_MODEL=gemini-2.5-flash
  1197. GOOGLE_GEMINI_IMAGE_MODEL=gemini-2.5-flash-image
  1198. LOG_LEVEL=info
  1199. PORT=3000
  1200. MAX_REQUEST_SIZE=50MB
  1201. ENABLE_CACHING=true
  1202. CACHE_TTL=3600
  1203. # Security Configuration
  1204. HTTP_SECRET=your_http_secret_here
  1205. HTTP_CORS_ENABLED=true
  1206. HTTP_CORS_ORIGINS=*
  1207. HTTP_DNS_REBINDING_ENABLED=true
  1208. HTTP_ALLOWED_HOSTS=127.0.0.1,localhost
  1209. HTTP_ENABLE_RATE_LIMITING=false
  1210. ```
  1211. ## Architecture
  1212. ```
  1213. Human MCP Server v2.16.0
  1214. ├── 👁️ Eyes Tools (4) - Visual Analysis & Document Processing
  1215. │ ├── eyes_analyze - Images, videos, GIFs analysis
  1216. │ ├── eyes_compare - Image comparison
  1217. │ ├── eyes_read_document - Document content extraction
  1218. │ └── eyes_summarize_document - Document summarization
  1219. │
  1220. ├── ✋ Hands Tools (18) - Content Generation, Image Editing, Music/SFX & Browser Automation
  1221. │ ├── Image Generation (1)
  1222. │ │ └── gemini_gen_image
  1223. │ ├── Video Generation (2)
  1224. │ │ ├── gemini_gen_video
  1225. │ │ └── gemini_image_to_video
  1226. │ ├── Music Generation (2)
  1227. │ │ ├── minimax_gen_music
  1228. │ │ └── elevenlabs_gen_music
  1229. │ ├── Sound Effects (1)
  1230. │ │ └── elevenlabs_gen_sfx
  1231. │ ├── AI Image Editing (5)
  1232. │ │ ├── gemini_edit_image
  1233. │ │ ├── gemini_inpaint_image
  1234. │ │ ├── gemini_outpaint_image
  1235. │ │ ├── gemini_style_transfer_image
  1236. │ │ └── gemini_compose_images
  1237. │ ├── Jimp Processing (4)
  1238. │ │ ├── jimp_crop_image
  1239. │ │ ├── jimp_resize_image
  1240. │ │ ├── jimp_rotate_image
  1241. │ │ └── jimp_mask_image
  1242. │ ├── Background Removal (1)
  1243. │ │ └── rmbg_remove_background
  1244. │ └── Browser Automation (3)
  1245. │ ├── playwright_screenshot_fullpage
  1246. │ ├── playwright_screenshot_viewport
  1247. │ └── playwright_screenshot_element
  1248. │
  1249. ├── 🗣️ Mouth Tools (4) - Speech Generation
  1250. │ ├── mouth_speak - Text-to-speech (Gemini/Minimax/ElevenLabs)
  1251. │ ├── mouth_narrate - Long-form narration
  1252. │ ├── mouth_explain - Code explanation
  1253. │ └── mouth_customize - Voice testing
  1254. │
  1255. └── 🧠 Brain Tools (3) - Advanced Reasoning
  1256. ├── mcp__reasoning__sequentialthinking - Native sequential thinking
  1257. ├── brain_analyze_simple - Pattern-based analysis
  1258. ├── brain_patterns_info - Reasoning frameworks
  1259. └── brain_reflect_enhanced - AI-powered reflection
  1260. Total: 29 MCP Tools
  1261. ```
  1262. **Documentation:**
  1263. - **[Project Roadmap](docs/project-roadmap.md)** - Development roadmap and future vision
  1264. - **[Project Overview](docs/project-overview-pdr.md)** - Product requirements and specifications
  1265. - **[Architecture & Code Standards](docs/codebase-structure-architecture-code-standards.md)** - Technical architecture
  1266. - **[Codebase Summary](docs/codebase-summary.md)** - Comprehensive codebase overview
  1267. ## Development Roadmap & Vision
  1268. **Mission**: Transform AI coding agents with complete human-like sensory capabilities, bridging the gap between artificial and human intelligence through sophisticated multimodal analysis.
  1269. ### Current Status: v2.16.0 - 29 Production-Ready MCP Tools
  1270. **👁️ Eyes (4 tools)** - Visual Analysis & Document Processing
  1271. - ✅ Image, video, GIF analysis with UI debugging and accessibility auditing
  1272. - ✅ Image comparison with visual difference detection
  1273. - ✅ Document processing for 12+ formats (PDF, DOCX, XLSX, PPTX, etc.)
  1274. - ✅ Document summarization and content extraction
  1275. **✋ Hands (18 tools)** - Content Generation, Image Editing, Music/SFX & Browser Automation
  1276. - ✅ Image generation with Gemini Imagen API (5 styles, 5 aspect ratios)
  1277. - ✅ Video generation with Gemini Veo 3.0 API (duration, FPS, camera controls)
  1278. - ✅ Music generation with Minimax Music 2.5 and ElevenLabs Music API
  1279. - ✅ Sound effects generation with ElevenLabs Sound Generation API
  1280. - ✅ AI-powered image editing: inpainting, outpainting, style transfer, composition
  1281. - ✅ Fast local Jimp processing: crop, resize, rotate, mask
  1282. - ✅ AI background removal with 3 quality models
  1283. - ✅ Browser automation: full page, viewport, and element screenshots with Playwright
  1284. **🗣️ Mouth (4 tools)** - Speech Generation
  1285. - ✅ Text-to-speech with 30+ voices and 24 languages
  1286. - ✅ Long-form narration with chapter breaks
  1287. - ✅ Code explanation with technical analysis
  1288. - ✅ Voice testing and customization
  1289. **🧠 Brain (3 tools)** - Advanced Reasoning
  1290. - ✅ Native sequential thinking (fast, no API calls)
  1291. - ✅ Pattern-based analysis (problem solving, root cause, SWOT, etc.)
  1292. - ✅ AI-powered reflection for complex analysis
  1293. ### Future Development
  1294. #### Phase 3: Audio Processing - Ears (Planned Q1 2025)
  1295. Only remaining capability to complete the human sensory suite:
  1296. - Speech-to-text transcription with speaker identification
  1297. - Audio content analysis and classification
  1298. - Audio quality assessment and debugging
  1299. - Support for 20+ audio formats (WAV, MP3, AAC, OGG, FLAC)
  1300. **Note:** Phases 1, 2, 4, 5, and 6 are complete with 27 production-ready tools
  1301. ### System Architecture (v2.16.0)
  1302. Complete human-like capabilities through 29 MCP tools with multi-provider support:
  1303. ```
  1304. ┌─────────────────┐ ┌──────────────────────────┐ ┌─────────────────────────┐
  1305. │ AI Agent │◄──►│ Human MCP Server │◄──►│ Google Gemini │
  1306. │ (MCP Client) │ │ v2.16.0 │ │ • Gemini 2.5/3.1 Flash │
  1307. └─────────────────┘ │ │ │ • Imagen API │
  1308. │ 👁️ Eyes (4 tools) ✅ │ │ • Veo 3.0 API │
  1309. │ • Visual Analysis │ │ • Speech API │
  1310. │ • Document Processing │ └─────────────────────────┘
  1311. │ │
  1312. │ ✋ Hands (18 tools) ✅ │ ┌─────────────────────────┐
  1313. │ • Image Generation │ │ Minimax │
  1314. │ • Video Generation │ │ • Speech 2.6 (HD/Turbo) │
  1315. │ • Music & SFX │ │ • Music 2.5 │
  1316. │ • AI Image Editing │ │ • Hailuo 2.3 Video │
  1317. │ • Jimp Processing │ └─────────────────────────┘
  1318. │ • Background Removal │
  1319. │ • Browser Automation │ ┌─────────────────────────┐
  1320. │ │ │ ZhipuAI (Z.AI) │
  1321. │ 🗣️ Mouth (4 tools) ✅ │ │ • GLM-4.6V Vision │
  1322. │ • Text-to-Speech │ │ • CogView-4 Image │
  1323. │ • Narration │ │ • CogVideoX-3 Video │
  1324. │ • Code Explanation │ └─────────────────────────┘
  1325. │ │
  1326. │ 🧠 Brain (3 tools) ✅ │ ┌─────────────────────────┐
  1327. │ • Sequential Thinking │ │ ElevenLabs │
  1328. │ • Pattern Analysis │ │ • TTS (70+ languages) │
  1329. │ • AI Reflection │ │ • Music Generation │
  1330. │ │ │ • Sound Effects │
  1331. │ 👂 Ears (Planned) │ └─────────────────────────┘
  1332. └──────────────────────────┘
  1333. ```
  1334. ### Key Benefits
  1335. **For Developers:**
  1336. - Visual debugging with UI bug detection and accessibility auditing
  1337. - Automated web screenshots for testing and documentation
  1338. - Document processing for technical specifications and reports
  1339. - AI-powered image and video generation for prototyping
  1340. - Advanced image editing without complex tools
  1341. - Speech generation for documentation and code explanations
  1342. - Sophisticated problem-solving with sequential reasoning
  1343. **For AI Agents:**
  1344. - Human-like multimodal understanding (vision, speech, documents)
  1345. - Automated web interaction and screenshot capture
  1346. - Creative content generation (images, videos, speech)
  1347. - Advanced image editing capabilities (inpainting, style transfer, etc.)
  1348. - Fast local image processing (crop, resize, rotate, mask)
  1349. - Complex reasoning with thought revision and reflection
  1350. - Pattern-based analysis for common problems
  1351. ### Current Achievements (v2.16.0)
  1352. **Completed Phases:**
  1353. - ✅ Phase 1: Eyes - Visual Analysis (4 tools) — Gemini + ZhipuAI providers
  1354. - ✅ Phase 2: Document Understanding (integrated into Eyes)
  1355. - ✅ Phase 4: Mouth - Speech Generation (4 tools) — Gemini + Minimax + ElevenLabs providers
  1356. - ✅ Phase 5: Hands - Content Generation, Image Editing & Browser Automation (18 tools) — Gemini + Minimax + ZhipuAI + ElevenLabs providers
  1357. - ✅ Phase 6: Brain - Advanced Reasoning (3 tools)
  1358. **Remaining:**
  1359. - ⏳ Phase 3: Ears - Audio Processing (planned)
  1360. **Goals Achieved:**
  1361. - ✅ 29 production-ready MCP tools
  1362. - ✅ 4 AI providers: Google Gemini, Minimax, ZhipuAI, ElevenLabs
  1363. - ✅ Support for 30+ file formats (images, videos, documents, audio)
  1364. - ✅ Browser automation for automated web screenshots
  1365. - ✅ Sub-30 second response times for most operations
  1366. - ✅ Professional-grade content generation (images, videos, speech, music, sound effects)
  1367. - ✅ Advanced reasoning with native + AI-powered tools
  1368. - ✅ Comprehensive documentation and examples
  1369. ### Getting Involved
  1370. Human MCP is built for the developer community. Whether you're integrating with MCP clients, contributing to core development, or providing feedback, your involvement shapes the future of AI agent capabilities.
  1371. - **Beta Testing**: Early access to new phases and features
  1372. - **Integration Partners**: Work with us to optimize for your MCP client
  1373. - **Community Feedback**: Help prioritize features and improvements
  1374. ## Supported Formats
  1375. **Visual Analysis Formats**:
  1376. - **Images**: PNG, JPEG, WebP, GIF (static)
  1377. - **Videos**: MP4, WebM, MOV, AVI
  1378. - **GIFs**: Animated GIF with frame extraction
  1379. - **Sources**: File paths, URLs, base64 data URLs
  1380. **Document Processing Formats (v2.0.0)**:
  1381. - **Documents**: PDF, DOCX, XLSX, PPTX, TXT, MD, RTF, ODT
  1382. - **Data**: CSV, JSON, XML, HTML
  1383. - **Features**: Text extraction, table processing, structured data extraction
  1384. - **Auto-detection**: Automatic format detection from content and extensions
  1385. **Speech Generation Formats**:
  1386. - **Gemini Output**: WAV (Base64 encoded), 24kHz mono, 31 voices, 24 languages
  1387. - **Minimax Output**: MP3/WAV, emotion control, speed adjustment
  1388. - **ElevenLabs Output**: MP3, 70+ languages, voice cloning, stability/style controls
  1389. **Content Generation Formats**:
  1390. - **Images**: PNG, JPEG (Base64 output)
  1391. - **Videos**: MP4 (Base64 output)
  1392. - **Durations**: 4s, 8s, 12s video lengths
  1393. - **Quality**: Professional-grade output with customizable FPS (1-60)
  1394. **Reasoning Capabilities (Brain Tools)**:
  1395. - **Native Sequential Thinking**: Fast, API-free thought processes with revision support
  1396. - **Pattern Analysis**: Quick problem-solving using proven frameworks (root cause, SWOT, pros/cons, etc.)
  1397. - **AI Reflection**: Complex meta-cognitive analysis for improving reasoning quality
  1398. - **Output Formats**: Structured thought chains, pattern-based solutions, improvement recommendations
  1399. ---
  1400. ## Star History
  1401. [![Star History Chart](https://api.star-history.com/svg?repos=mrgoonie/human-mcp&type=date&legend=top-left)](https://www.star-history.com/#mrgoonie/human-mcp&type=date&legend=top-left)
  1402. ## Contributing
  1403. 1. Fork the repository
  1404. 2. Create a feature branch
  1405. 3. Make your changes
  1406. 4. Add tests
  1407. 5. Submit a pull request
  1408. ## License
  1409. MIT License - see [LICENSE](LICENSE) for details.
  1410. ## Documentation
  1411. Comprehensive documentation is available in the `/docs` directory:
  1412. - **[Project Roadmap](docs/project-roadmap.md)** - Development roadmap and future vision through 2025
  1413. - **[Project Overview & PDR](docs/project-overview-pdr.md)** - Project overview and product requirements
  1414. - **[Architecture & Code Standards](docs/codebase-structure-architecture-code-standards.md)** - Technical architecture and coding standards
  1415. - **[Codebase Summary](docs/codebase-summary.md)** - Comprehensive codebase overview
  1416. ## Support
  1417. - 📖 [Documentation](docs/) - Complete project documentation
  1418. - 💡 [Examples](humanmcp://examples/debugging) - Usage examples and debugging workflows
  1419. - 🐛 [Issues](https://github.com/human-mcp/human-mcp/issues) - Report bugs and request features
  1420. - 💬 [Discussions](https://github.com/human-mcp/human-mcp/discussions) - Community discussions
  1421. **Human MCP** - Making visual debugging as natural as asking a human to look at your screen.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.