Skip to content
Back to search

mcp-DEEPwebresearch

Enhanced MCP server for deep web research

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/qpd-v--mcp-deepwebresearch/badge.svg)](/listings/qpd-v--mcp-deepwebresearch)

HTML

<a href="/listings/qpd-v--mcp-deepwebresearch"><img src="/api/listings/qpd-v--mcp-deepwebresearch/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Global Installation (Recommended)

# Install globally using npm
npm install -g mcp-deepwebresearch

# Or using yarn
yarn global add mcp-deepwebresearch

# Or using pnpm
pnpm add -g mcp-deepwebresearch

Local Project Installation

# Using npm
npm install mcp-deepwebresearch

# Using yarn
yarn add mcp-deepwebresearch

# Using pnpm
pnpm add mcp-deepwebresearch

Claude Desktop Integration

After installing the package, add this entry to your `claude_desktop_config.json`:

Windows

{
  "mcpServers": {
    "deepwebresearch": {
      "command": "mcp-deepwebresearch",
      "args": []
    }
  }
}

Location: `%APPDATA%\Claude\claude_desktop_config.json`

macOS

{
  "mcpServers": {
    "deepwebresearch": {
      "command": "mcp-deepwebresearch",
      "args": []
    }
  }
}

Location: `~/Library/Application Support/Claude/claude_desktop_config.json`

This config allows Claude Desktop to automatically start the web research MCP server when needed.

First-time Setup

After installation, run this command to install required browser dependencies:

npx playwright install chromium

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 325 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # MCP Deep Web Research Server (v0.3.0)
  2. [![Node.js Version](https://img.shields.io/badge/node-%3E%3D18-brightgreen.svg)](https://nodejs.org/)
  3. [![TypeScript](https://img.shields.io/badge/TypeScript-5.0-blue.svg)](https://www.typescriptlang.org/)
  4. [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
  5. A Model Context Protocol (MCP) server for advanced web research.
  6. <a href="https://glama.ai/mcp/servers/5afpizjl6x"><img width="380" height="200" src="https://glama.ai/mcp/servers/5afpizjl6x/badge" alt="Web Research Server MCP server" /></a>
  7. ## Latest Changes
  8. - Added visit_page tool for direct webpage content extraction
  9. - Optimized performance to work within MCP timeout limits
  10. * Reduced default maxDepth and maxBranching parameters
  11. * Improved page loading efficiency
  12. * Added timeout checks throughout the process
  13. * Enhanced error handling for timeouts
  14. > This project is a fork of [mcp-webresearch](https://github.com/mzxrai/mcp-webresearch) by [mzxrai](https://github.com/mzxrai), enhanced with additional features for deep web research capabilities. We're grateful to the original creators for their foundational work.
  15. Bring real-time info into Claude with intelligent search queuing, enhanced content extraction, and deep research capabilities.
  16. ## Features
  17. - Intelligent Search Queue System
  18. - Batch search operations with rate limiting
  19. - Queue management with progress tracking
  20. - Error recovery and automatic retries
  21. - Search result deduplication
  22. - Enhanced Content Extraction
  23. - TF-IDF based relevance scoring
  24. - Keyword proximity analysis
  25. - Content section weighting
  26. - Readability scoring
  27. - Improved HTML structure parsing
  28. - Structured data extraction
  29. - Better content cleaning and formatting
  30. - Core Features
  31. - Google search integration
  32. - Webpage content extraction
  33. - Research session tracking
  34. - Markdown conversion with improved formatting
  35. ## Prerequisites
  36. - [Node.js](https://nodejs.org/) >= 18 (includes `npm` and `npx`)
  37. - [Claude Desktop app](https://claude.ai/download)
  38. ## Installation
  39. ### Global Installation (Recommended)
  40. ```bash
  41. # Install globally using npm
  42. npm install -g mcp-deepwebresearch
  43. # Or using yarn
  44. yarn global add mcp-deepwebresearch
  45. # Or using pnpm
  46. pnpm add -g mcp-deepwebresearch
  47. ```
  48. ### Local Project Installation
  49. ```bash
  50. # Using npm
  51. npm install mcp-deepwebresearch
  52. # Using yarn
  53. yarn add mcp-deepwebresearch
  54. # Using pnpm
  55. pnpm add mcp-deepwebresearch
  56. ```
  57. ### Claude Desktop Integration
  58. After installing the package, add this entry to your `claude_desktop_config.json`:
  59. #### Windows
  60. ```json
  61. {
  62. "mcpServers": {
  63. "deepwebresearch": {
  64. "command": "mcp-deepwebresearch",
  65. "args": []
  66. }
  67. }
  68. }
  69. ```
  70. Location: `%APPDATA%\Claude\claude_desktop_config.json`
  71. #### macOS
  72. ```json
  73. {
  74. "mcpServers": {
  75. "deepwebresearch": {
  76. "command": "mcp-deepwebresearch",
  77. "args": []
  78. }
  79. }
  80. }
  81. ```
  82. Location: `~/Library/Application Support/Claude/claude_desktop_config.json`
  83. This config allows Claude Desktop to automatically start the web research MCP server when needed.
  84. ### First-time Setup
  85. After installation, run this command to install required browser dependencies:
  86. ```bash
  87. npx playwright install chromium
  88. ```
  89. ## Usage
  90. Simply start a chat with Claude and send a prompt that would benefit from web research. If you'd like a prebuilt prompt customized for deeper web research, you can use the `agentic-research` prompt that we provide through this package. Access that prompt in Claude Desktop by clicking the Paperclip icon in the chat input and then selecting `Choose an integration` → `deepwebresearch` → `agentic-research`.
  91. ### Tools
  92. 1. `deep_research`
  93. - Performs comprehensive research with content analysis
  94. - Arguments:
  95. ```typescript
  96. {
  97. topic: string;
  98. maxDepth?: number; // default: 2
  99. maxBranching?: number; // default: 3
  100. timeout?: number; // default: 55000 (55 seconds)
  101. minRelevanceScore?: number; // default: 0.7
  102. }
  103. ```
  104. - Returns:
  105. ```typescript
  106. {
  107. findings: {
  108. mainTopics: Array<{name: string, importance: number}>;
  109. keyInsights: Array<{text: string, confidence: number}>;
  110. sources: Array<{url: string, credibilityScore: number}>;
  111. };
  112. progress: {
  113. completedSteps: number;
  114. totalSteps: number;
  115. processedUrls: number;
  116. };
  117. timing: {
  118. started: string;
  119. completed?: string;
  120. duration?: number;
  121. operations?: {
  122. parallelSearch?: number;
  123. deduplication?: number;
  124. topResultsProcessing?: number;
  125. remainingResultsProcessing?: number;
  126. total?: number;
  127. };
  128. };
  129. }
  130. ```
  131. 2. `parallel_search`
  132. - Performs multiple Google searches in parallel with intelligent queuing
  133. - Arguments: `{ queries: string[], maxParallel?: number }`
  134. - Note: maxParallel is limited to 5 to ensure reliable performance
  135. 3. `visit_page`
  136. - Visit a webpage and extract its content
  137. - Arguments: `{ url: string }`
  138. - Returns:
  139. ```typescript
  140. {
  141. url: string;
  142. title: string;
  143. content: string; // Markdown formatted content
  144. }
  145. ```
  146. ### Prompts
  147. #### `agentic-research`
  148. A guided research prompt that helps Claude conduct thorough web research. The prompt instructs Claude to:
  149. - Start with broad searches to understand the topic landscape
  150. - Prioritize high-quality, authoritative sources
  151. - Iteratively refine the research direction based on findings
  152. - Keep you informed and let you guide the research interactively
  153. - Always cite sources with URLs
  154. ## Configuration Options
  155. The server can be configured through environment variables:
  156. - `MAX_PARALLEL_SEARCHES`: Maximum number of concurrent searches (default: 5)
  157. - `SEARCH_DELAY_MS`: Delay between searches in milliseconds (default: 200)
  158. - `MAX_RETRIES`: Number of retry attempts for failed requests (default: 3)
  159. - `TIMEOUT_MS`: Request timeout in milliseconds (default: 55000)
  160. - `LOG_LEVEL`: Logging level (default: 'info')
  161. ## Error Handling
  162. ### Common Issues
  163. 1. Rate Limiting
  164. - Symptom: "Too many requests" error
  165. - Solution: Increase `SEARCH_DELAY_MS` or decrease `MAX_PARALLEL_SEARCHES`
  166. 2. Network Timeouts
  167. - Symptom: "Request timed out" error
  168. - Solution: Ensure requests complete within the 60-second MCP timeout
  169. 3. Browser Issues
  170. - Symptom: "Browser failed to launch" error
  171. - Solution: Ensure Playwright is properly installed (`npx playwright install`)
  172. ### Debugging
  173. This is beta software. If you run into issues:
  174. 1. Check Claude Desktop's MCP logs:
  175. ```bash
  176. # On macOS
  177. tail -n 20 -f ~/Library/Logs/Claude/mcp*.log
  178. # On Windows
  179. Get-Content -Path "$env:APPDATA\Claude\logs\mcp*.log" -Tail 20 -Wait
  180. ```
  181. 2. Enable debug logging:
  182. ```bash
  183. export LOG_LEVEL=debug
  184. ```
  185. ## Development
  186. ### Setup
  187. ```bash
  188. # Install dependencies
  189. pnpm install
  190. # Build the project
  191. pnpm build
  192. # Watch for changes
  193. pnpm watch
  194. # Run in development mode
  195. pnpm dev
  196. ```
  197. ### Testing
  198. ```bash
  199. # Run all tests
  200. pnpm test
  201. # Run tests in watch mode
  202. pnpm test:watch
  203. # Run tests with coverage
  204. pnpm test:coverage
  205. ```
  206. ### Code Quality
  207. ```bash
  208. # Run linter
  209. pnpm lint
  210. # Fix linting issues
  211. pnpm lint:fix
  212. # Type check
  213. pnpm type-check
  214. ```
  215. ## Contributing
  216. 1. Fork the repository
  217. 2. Create your feature branch (`git checkout -b feature/amazing-feature`)
  218. 3. Commit your changes (`git commit -m 'Add some amazing feature'`)
  219. 4. Push to the branch (`git push origin feature/amazing-feature`)
  220. 5. Open a Pull Request
  221. ### Coding Standards
  222. - Follow TypeScript best practices
  223. - Maintain test coverage above 80%
  224. - Document new features and APIs
  225. - Update CHANGELOG.md for significant changes
  226. - Follow semantic versioning
  227. ### Performance Considerations
  228. - Use batch operations where possible
  229. - Implement proper error handling and retries
  230. - Consider memory usage with large datasets
  231. - Cache results when appropriate
  232. - Use streaming for large content
  233. ## Requirements
  234. - Node.js >= 18
  235. - Playwright (automatically installed as a dependency)
  236. ## Verified Platforms
  237. - [x] macOS
  238. - [x] Windows
  239. - [ ] Linux
  240. ## License
  241. MIT
  242. ## Credits
  243. This project builds upon the excellent work of [mcp-webresearch](https://github.com/mzxrai/mcp-webresearch) by [mzxrai](https://github.com/mzxrai). The original codebase provided the foundation for our enhanced features and capabilities.
  244. ## Author
  245. [qpd-v](https://github.com/qpd-v)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.