mcp-DEEPwebresearch
Enhanced MCP server for deep web research
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Badge for your README
Markdown
[](/listings/qpd-v--mcp-deepwebresearch)HTML
<a href="/listings/qpd-v--mcp-deepwebresearch"><img src="/api/listings/qpd-v--mcp-deepwebresearch/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
Global Installation (Recommended)
# Install globally using npm npm install -g mcp-deepwebresearch # Or using yarn yarn global add mcp-deepwebresearch # Or using pnpm pnpm add -g mcp-deepwebresearch
Local Project Installation
# Using npm npm install mcp-deepwebresearch # Using yarn yarn add mcp-deepwebresearch # Using pnpm pnpm add mcp-deepwebresearch
Claude Desktop Integration
After installing the package, add this entry to your `claude_desktop_config.json`:
Windows
{
"mcpServers": {
"deepwebresearch": {
"command": "mcp-deepwebresearch",
"args": []
}
}
}Location: `%APPDATA%\Claude\claude_desktop_config.json`
macOS
{
"mcpServers": {
"deepwebresearch": {
"command": "mcp-deepwebresearch",
"args": []
}
}
}Location: `~/Library/Application Support/Claude/claude_desktop_config.json`
This config allows Claude Desktop to automatically start the web research MCP server when needed.
First-time Setup
After installation, run this command to install required browser dependencies:
npx playwright install chromium
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
README.md · README · 325 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
- # MCP Deep Web Research Server (v0.3.0)
- [](https://nodejs.org/)
- [](https://www.typescriptlang.org/)
- [](https://opensource.org/licenses/MIT)
- A Model Context Protocol (MCP) server for advanced web research.
- <a href="https://glama.ai/mcp/servers/5afpizjl6x"><img width="380" height="200" src="https://glama.ai/mcp/servers/5afpizjl6x/badge" alt="Web Research Server MCP server" /></a>
- ## Latest Changes
- - Added visit_page tool for direct webpage content extraction
- - Optimized performance to work within MCP timeout limits
- * Reduced default maxDepth and maxBranching parameters
- * Improved page loading efficiency
- * Added timeout checks throughout the process
- * Enhanced error handling for timeouts
- > This project is a fork of [mcp-webresearch](https://github.com/mzxrai/mcp-webresearch) by [mzxrai](https://github.com/mzxrai), enhanced with additional features for deep web research capabilities. We're grateful to the original creators for their foundational work.
- Bring real-time info into Claude with intelligent search queuing, enhanced content extraction, and deep research capabilities.
- ## Features
- - Intelligent Search Queue System
- - Batch search operations with rate limiting
- - Queue management with progress tracking
- - Error recovery and automatic retries
- - Search result deduplication
- - Enhanced Content Extraction
- - TF-IDF based relevance scoring
- - Keyword proximity analysis
- - Content section weighting
- - Readability scoring
- - Improved HTML structure parsing
- - Structured data extraction
- - Better content cleaning and formatting
- - Core Features
- - Google search integration
- - Webpage content extraction
- - Research session tracking
- - Markdown conversion with improved formatting
- ## Prerequisites
- - [Node.js](https://nodejs.org/) >= 18 (includes `npm` and `npx`)
- - [Claude Desktop app](https://claude.ai/download)
- ## Installation
- ### Global Installation (Recommended)
- ```bash
- # Install globally using npm
- npm install -g mcp-deepwebresearch
- # Or using yarn
- yarn global add mcp-deepwebresearch
- # Or using pnpm
- pnpm add -g mcp-deepwebresearch
- ```
- ### Local Project Installation
- ```bash
- # Using npm
- npm install mcp-deepwebresearch
- # Using yarn
- yarn add mcp-deepwebresearch
- # Using pnpm
- pnpm add mcp-deepwebresearch
- ```
- ### Claude Desktop Integration
- After installing the package, add this entry to your `claude_desktop_config.json`:
- #### Windows
- ```json
- {
- "mcpServers": {
- "deepwebresearch": {
- "command": "mcp-deepwebresearch",
- "args": []
- }
- }
- }
- ```
- Location: `%APPDATA%\Claude\claude_desktop_config.json`
- #### macOS
- ```json
- {
- "mcpServers": {
- "deepwebresearch": {
- "command": "mcp-deepwebresearch",
- "args": []
- }
- }
- }
- ```
- Location: `~/Library/Application Support/Claude/claude_desktop_config.json`
- This config allows Claude Desktop to automatically start the web research MCP server when needed.
- ### First-time Setup
- After installation, run this command to install required browser dependencies:
- ```bash
- npx playwright install chromium
- ```
- ## Usage
- Simply start a chat with Claude and send a prompt that would benefit from web research. If you'd like a prebuilt prompt customized for deeper web research, you can use the `agentic-research` prompt that we provide through this package. Access that prompt in Claude Desktop by clicking the Paperclip icon in the chat input and then selecting `Choose an integration` → `deepwebresearch` → `agentic-research`.
- ### Tools
- 1. `deep_research`
- - Performs comprehensive research with content analysis
- - Arguments:
- ```typescript
- {
- topic: string;
- maxDepth?: number; // default: 2
- maxBranching?: number; // default: 3
- timeout?: number; // default: 55000 (55 seconds)
- minRelevanceScore?: number; // default: 0.7
- }
- ```
- - Returns:
- ```typescript
- {
- findings: {
- mainTopics: Array<{name: string, importance: number}>;
- keyInsights: Array<{text: string, confidence: number}>;
- sources: Array<{url: string, credibilityScore: number}>;
- };
- progress: {
- completedSteps: number;
- totalSteps: number;
- processedUrls: number;
- };
- timing: {
- started: string;
- completed?: string;
- duration?: number;
- operations?: {
- parallelSearch?: number;
- deduplication?: number;
- topResultsProcessing?: number;
- remainingResultsProcessing?: number;
- total?: number;
- };
- };
- }
- ```
- 2. `parallel_search`
- - Performs multiple Google searches in parallel with intelligent queuing
- - Arguments: `{ queries: string[], maxParallel?: number }`
- - Note: maxParallel is limited to 5 to ensure reliable performance
- 3. `visit_page`
- - Visit a webpage and extract its content
- - Arguments: `{ url: string }`
- - Returns:
- ```typescript
- {
- url: string;
- title: string;
- content: string; // Markdown formatted content
- }
- ```
- ### Prompts
- #### `agentic-research`
- A guided research prompt that helps Claude conduct thorough web research. The prompt instructs Claude to:
- - Start with broad searches to understand the topic landscape
- - Prioritize high-quality, authoritative sources
- - Iteratively refine the research direction based on findings
- - Keep you informed and let you guide the research interactively
- - Always cite sources with URLs
- ## Configuration Options
- The server can be configured through environment variables:
- - `MAX_PARALLEL_SEARCHES`: Maximum number of concurrent searches (default: 5)
- - `SEARCH_DELAY_MS`: Delay between searches in milliseconds (default: 200)
- - `MAX_RETRIES`: Number of retry attempts for failed requests (default: 3)
- - `TIMEOUT_MS`: Request timeout in milliseconds (default: 55000)
- - `LOG_LEVEL`: Logging level (default: 'info')
- ## Error Handling
- ### Common Issues
- 1. Rate Limiting
- - Symptom: "Too many requests" error
- - Solution: Increase `SEARCH_DELAY_MS` or decrease `MAX_PARALLEL_SEARCHES`
- 2. Network Timeouts
- - Symptom: "Request timed out" error
- - Solution: Ensure requests complete within the 60-second MCP timeout
- 3. Browser Issues
- - Symptom: "Browser failed to launch" error
- - Solution: Ensure Playwright is properly installed (`npx playwright install`)
- ### Debugging
- This is beta software. If you run into issues:
- 1. Check Claude Desktop's MCP logs:
- ```bash
- # On macOS
- tail -n 20 -f ~/Library/Logs/Claude/mcp*.log
- # On Windows
- Get-Content -Path "$env:APPDATA\Claude\logs\mcp*.log" -Tail 20 -Wait
- ```
- 2. Enable debug logging:
- ```bash
- export LOG_LEVEL=debug
- ```
- ## Development
- ### Setup
- ```bash
- # Install dependencies
- pnpm install
- # Build the project
- pnpm build
- # Watch for changes
- pnpm watch
- # Run in development mode
- pnpm dev
- ```
- ### Testing
- ```bash
- # Run all tests
- pnpm test
- # Run tests in watch mode
- pnpm test:watch
- # Run tests with coverage
- pnpm test:coverage
- ```
- ### Code Quality
- ```bash
- # Run linter
- pnpm lint
- # Fix linting issues
- pnpm lint:fix
- # Type check
- pnpm type-check
- ```
- ## Contributing
- 1. Fork the repository
- 2. Create your feature branch (`git checkout -b feature/amazing-feature`)
- 3. Commit your changes (`git commit -m 'Add some amazing feature'`)
- 4. Push to the branch (`git push origin feature/amazing-feature`)
- 5. Open a Pull Request
- ### Coding Standards
- - Follow TypeScript best practices
- - Maintain test coverage above 80%
- - Document new features and APIs
- - Update CHANGELOG.md for significant changes
- - Follow semantic versioning
- ### Performance Considerations
- - Use batch operations where possible
- - Implement proper error handling and retries
- - Consider memory usage with large datasets
- - Cache results when appropriate
- - Use streaming for large content
- ## Requirements
- - Node.js >= 18
- - Playwright (automatically installed as a dependency)
- ## Verified Platforms
- - [x] macOS
- - [x] Windows
- - [ ] Linux
- ## License
- MIT
- ## Credits
- This project builds upon the excellent work of [mcp-webresearch](https://github.com/mzxrai/mcp-webresearch) by [mzxrai](https://github.com/mzxrai). The original codebase provided the foundation for our enhanced features and capabilities.
- ## Author
- [qpd-v](https://github.com/qpd-v)
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If it’s helped, pass it on.