Skip to content
Back to search

open-websearch

Multi-engine MCP server, CLI, and local daemon for agent web search and content retrieval — skill-guided workflows, no API keys.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/aas-ee/open-websearch with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/aas-ee/open-websearch'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/aas-ee-open-websearch/badge.svg)](/listings/aas-ee-open-websearch)

HTML

<a href="/listings/aas-ee-open-websearch"><img src="/api/listings/aas-ee-open-websearch/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

If you are using `open-websearch` as an MCP server, continue with the MCP-oriented setup below.

NPX Quick Start (Recommended)

The fastest way to get started:

# Basic usage
npx open-websearch@latest

# With environment variables (Linux/macOS)
DEFAULT_SEARCH_ENGINE=duckduckgo ENABLE_CORS=true npx open-websearch@latest

# Windows PowerShell
$env:DEFAULT_SEARCH_ENGINE="duckduckgo"; $env:ENABLE_CORS="true"; npx open-websearch@latest

# Windows CMD
set MODE=stdio && set DEFAULT_SEARCH_ENGINE=duckduckgo && npx open-websearch@latest

# Cross-platform (requires cross-env, Used for local development)
npm install -g open-websearch
npx cross-env DEFAULT_SEARCH_ENGINE=duckduckgo ENABLE_CORS=true open-websearch

**Environment Variables:**

| Variable | Default | Options | Description | |----------|-------------------------|---------|-------------| | `ENABLE_CORS` | `false` | `true`, `false` | Enable CORS | | `CORS_ORIGIN` | `*` | Any valid origin | CORS origin configuration | | `DEFAULT_SEARCH_ENGINE` | `bing` | `bing`, `duckduckgo`, `exa`, `brave`, `baidu`, `csdn`, `linuxdo`, `juejin`, `startpage`, `sogou`, `hackernews` | Default search engine | | `USE_PROXY` | `false` | `true`, `false` | Enable HTTP proxy | | `PROXY_URL` | `http://127.0.0.1:7890` | Any valid URL | Proxy server URL | | `FAKE_IP_CIDRS` | empty | Comma-separated CIDR list | Treat DNS answers in these CIDRs as synthetic fake-IP results and do not block them as private-network DNS answers. Literal private/local targets and other private-network DNS answers remain blocked | | `FETCH_WEB_INSECURE_TLS` | `false` | `true`, `false` | Disable TLS verification only for the request leg of `fetchWebContent`; it does not affect Playwright browser navigation. Use only for broken certificate chains | | `MODE` | `both` | `both`, `http`, `stdio` | Server mode: both HTTP+STDIO, HTTP only, or STDIO only | | `PORT` | `3000` | 1-65535 | Server port | | `ALLOWED_SEARCH_ENGINES` | empty (all available) | Comma-separated engine names | Limit which search engines can be used; if the default engine is not in this list, the first allowed engine becomes the default | | `SEARCH_MODE` | `auto` | `request`, `auto`, `playwright` | Search strategy. Currently only affects Bing: force HTTP request mode (`request`), force Playwright mode (`playwright`), or let the agent choose (`auto`, default). Forced modes never expose a `searchMode` override to the agent. In `auto` mode the server checks whether Playwright is really usable (the client module can actually be loaded, and for local launches a real browser binary exists: explicit `PLAYWRIGHT_EXECUTABLE_PATH`, bundled browser, or system Chrome/Edge); if available, the search tool exposes a `searchMode` parameter and directs the agent to stay on the default `auto` and only retry with `playwright` when request results fail, return empty, or look blocked; otherwise it behaves as forced request mode. If `playwright` is forced but not usable, searches fail with a `browser_unavailable` error | | `PLAYWRIGHT_PACKAGE` | `auto` | `auto`, `playwright`, `playwright-core` | Which Playwright client package to resolve when browser mode is enabled | | `PLAYWRIGHT_MODULE_PATH` | empty | Absolute path or project-relative path | Reuse an existing Playwright client package outside this project | | `PLAYWRIGHT_EXECUTABLE_PATH` | empty | Any valid browser binary path | Launch an existing Chromium/Chrome executable without installing bundled browsers | | `PLAYWRIGHT_WS_ENDPOINT` | empty | Valid Playwright `ws://` / `wss://` endpoint | Connect to an existing remote Playwright browser server | | `PLAYWRIGHT_CDP_ENDPOINT` | empty | Valid Chromium CDP endpoint | Connect to an existing Chromium instance over CDP | | `PLAYWRIGHT_HEADLESS` | `true` | `true`, `false` | Whether Playwright Chromium runs in headless mode |

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 787 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <div align="center">
  2. # Open-WebSearch
  3. [![ModelScope](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/Aas-ee/3af09e0f4c7821fb2e9acb96483a5ff0/raw/badge.json&color=%23de5a16)](https://www.modelscope.cn/mcp/servers/Aasee1/open-webSearch)
  4. [![smithery badge](https://smithery.ai/badge/@Aas-ee/open-websearch)](https://smithery.ai/server/@Aas-ee/open-websearch)
  5. ![Version](https://img.shields.io/github/v/release/Aas-ee/open-websearch)
  6. ![License](https://img.shields.io/github/license/Aas-ee/open-websearch)
  7. ![Issues](https://img.shields.io/github/issues/Aas-ee/open-websearch)
  8. **[🇨🇳 中文](./README-zh.md) | 🇺🇸 English**
  9. </div>
  10. `open-websearch` provides an MCP server, CLI, and local daemon, and can also be paired with skill-guided agent workflows for live web search and content retrieval without API keys.
  11. ## Sponsor
  12. <div align="center">
  13. <a href="https://www.swiftproxy.net/?ref=openwebSearch" rel="sponsored">
  14. <img src="./docs/assets/sponsors/a69c410018edd7c45bffd4864629e25b.png" alt="Swiftproxy" width="600">
  15. </a>
  16. </div>
  17. > [**Swiftproxy**](https://www.swiftproxy.net/?ref=openwebSearch) provides high-quality static residential proxies with stable IPs for multi-account management, automation, web scraping, and secure online operations. Protect your accounts with clean IPs and reliable proxy infrastructure. Static proxy traffic is valid for 30 days with unlimited usage. Get **10% off** with code **`PROXY90`**.
  18. ## Features
  19. - Web search using multi-engine results
  20. - bing
  21. - baidu
  22. - ~~linux.do~~ temporarily unsupported
  23. - csdn
  24. - duckduckgo
  25. - exa
  26. - brave
  27. - juejin
  28. - startpage
  29. - sogou
  30. - hackernews
  31. - keenable
  32. - HTTP proxy configuration support for accessing restricted resources
  33. - No API keys or authentication required
  34. - Returns structured results with titles, URLs, and descriptions
  35. - Configurable number of results per search
  36. - Customizable default search engine
  37. - Support for fetching individual article content
  38. - csdn
  39. - github (README files)
  40. - generic HTTP(S) page / Markdown content
  41. ## Choose the Right Path
  42. - `MCP`
  43. - Best when you want to connect `open-websearch` to Claude Desktop, Cherry Studio, Cursor, or another MCP client.
  44. - `CLI`
  45. - Best for one-shot local commands, shell scripts, and direct terminal usage.
  46. - `Local daemon`
  47. - Best when you want a reusable long-lived local HTTP service exposing `status`, `GET /health`, and `POST /search` / `POST /fetch-*`. Start it explicitly with `open-websearch serve` and check it with `open-websearch status`.
  48. - `Skill`
  49. - Best as an agent-facing guidance layer for setup and usage. A skill does not replace MCP, CLI, or the local daemon; it typically works together with the CLI and/or local daemon to help an agent discover, activate, and use the smallest working path.
  50. ## Use with a Skill
  51. Install the `open-websearch` skill for your agent first:
  52. ```bash
  53. npx skills add https://github.com/Aas-ee/open-webSearch --skill open-websearch
  54. ```
  55. On first use, the skill typically follows this path: detect whether a usable `open-websearch` path already exists, guide setup/enablement if it does not, validate that the capability is active, and only then continue with search or fetch through the smallest working path.
  56. If the current environment cannot complete setup or activation automatically, you can explicitly have the agent start the local daemon first:
  57. ```bash
  58. open-websearch serve
  59. open-websearch status
  60. ```
  61. Keep installation proxy settings separate from runtime proxy settings:
  62. - Installation proxy / mirror
  63. - Use this when the skill or agent is installing `open-websearch`, `playwright`, or other npm packages.
  64. - In restricted networks, npm-specific flags or npm config often work better than generic shell proxy variables, for example:
  65. ```bash
  66. npm --proxy http://127.0.0.1:7890 --https-proxy http://127.0.0.1:7890 install -g open-websearch
  67. ```
  68. - Runtime proxy
  69. - Use this when the daemon is already installed and is about to perform live `search` / `fetch` work.
  70. - This affects the `open-websearch` network traffic after `serve` starts, for example:
  71. ```bash
  72. USE_PROXY=true PROXY_URL=http://127.0.0.1:7890 open-websearch serve
  73. ```
  74. If the agent can only get through the package-install step with npm proxy settings, but live search/fetch also needs a proxy after startup, those are two separate configuration steps and should be handled separately.
  75. ## CLI and Local Daemon
  76. CLI is for one-shot execution. The local daemon is a long-lived local HTTP service for repeated calls with lower startup friction. Use `open-websearch serve` as the explicit daemon start command and `open-websearch status` as the explicit daemon status command.
  77. Action commands such as `search` and `fetch-web` try the default local daemon first when it is available. If you pass `--daemon-url`, that daemon path becomes explicit and silent fallback to direct execution is disabled.
  78. Build first:
  79. ```bash
  80. npm run build
  81. ```
  82. Start the local daemon:
  83. ```bash
  84. npm run serve
  85. # globally installed: open-websearch serve
  86. ```
  87. Check status:
  88. ```bash
  89. npm run status -- --json
  90. # globally installed: open-websearch status --json
  91. ```
  92. Run a one-shot local CLI search:
  93. ```bash
  94. npm run search:cli -- "open web search" --json
  95. ```
  96. Notes:
  97. - Bare `open-websearch` is the MCP server compatibility entrypoint, not the recommended daemon start command for agent automation.
  98. - For content extraction, prefer searching first and then fetching a more specific result page. Some homepages and JS-heavy landing pages may not expose readable article text through `fetch-web`.
  99. For the local daemon HTTP API (`serve`, `status`, `GET /health`, `POST /search`, `POST /fetch-*`), see [docs/http-api.md](docs/http-api.md).
  100. ## TODO
  101. - Support for ~~Bing~~ (already supported), ~~DuckDuckGo~~ (already supported), ~~Exa~~ (already supported), ~~Brave~~ (already supported), ~~Sogou~~ (already supported), ~~Hacker News~~ (already supported), Google and other search engines
  102. - Support for more blogs, forums, and social platforms
  103. - Optimize article content extraction, add support for more sites
  104. - ~~Support for GitHub README fetching~~ (already supported)
  105. ## Installation Guide
  106. If you are using `open-websearch` as an MCP server, continue with the MCP-oriented setup below.
  107. ### NPX Quick Start (Recommended)
  108. The fastest way to get started:
  109. ```bash
  110. # Basic usage
  111. npx open-websearch@latest
  112. # With environment variables (Linux/macOS)
  113. DEFAULT_SEARCH_ENGINE=duckduckgo ENABLE_CORS=true npx open-websearch@latest
  114. # Windows PowerShell
  115. $env:DEFAULT_SEARCH_ENGINE="duckduckgo"; $env:ENABLE_CORS="true"; npx open-websearch@latest
  116. # Windows CMD
  117. set MODE=stdio && set DEFAULT_SEARCH_ENGINE=duckduckgo && npx open-websearch@latest
  118. # Cross-platform (requires cross-env, Used for local development)
  119. npm install -g open-websearch
  120. npx cross-env DEFAULT_SEARCH_ENGINE=duckduckgo ENABLE_CORS=true open-websearch
  121. ```
  122. **Environment Variables:**
  123. | Variable | Default | Options | Description |
  124. |----------|-------------------------|---------|-------------|
  125. | `ENABLE_CORS` | `false` | `true`, `false` | Enable CORS |
  126. | `CORS_ORIGIN` | `*` | Any valid origin | CORS origin configuration |
  127. | `DEFAULT_SEARCH_ENGINE` | `bing` | `bing`, `duckduckgo`, `exa`, `brave`, `baidu`, `csdn`, `linuxdo`, `juejin`, `startpage`, `sogou`, `hackernews`, `keenable` | Default search engine |
  128. | `USE_PROXY` | `false` | `true`, `false` | Enable HTTP proxy |
  129. | `PROXY_URL` | `http://127.0.0.1:7890` | Any valid URL | Proxy server URL |
  130. | `FAKE_IP_CIDRS` | empty | Comma-separated CIDR list | Treat DNS answers in these CIDRs as synthetic fake-IP results and do not block them as private-network DNS answers. Literal private/local targets and other private-network DNS answers remain blocked |
  131. | `FETCH_WEB_INSECURE_TLS` | `false` | `true`, `false` | Disable TLS verification only for the request leg of `fetchWebContent`; it does not affect Playwright browser navigation. Use only for broken certificate chains |
  132. | `MODE` | `both` | `both`, `http`, `stdio` | Server mode: both HTTP+STDIO, HTTP only, or STDIO only |
  133. | `PORT` | `3000` | 1-65535 | Server port |
  134. | `ALLOWED_SEARCH_ENGINES` | empty (all available) | Comma-separated engine names | Limit which search engines can be used; if the default engine is not in this list, the first allowed engine becomes the default |
  135. | `SEARCH_MODE` | `auto` | `request`, `auto`, `playwright` | Search strategy. Currently only affects Bing: force HTTP request mode (`request`), force Playwright mode (`playwright`), or let the agent choose (`auto`, default). Forced modes never expose a `searchMode` override to the agent. In `auto` mode the server checks whether Playwright is really usable (the client module can actually be loaded, and for local launches a real browser binary exists: explicit `PLAYWRIGHT_EXECUTABLE_PATH`, bundled browser, or system Chrome/Edge); if available, the search tool exposes a `searchMode` parameter and directs the agent to stay on the default `auto` and only retry with `playwright` when request results fail, return empty, or look blocked; otherwise it behaves as forced request mode. If `playwright` is forced but not usable, searches fail with a `browser_unavailable` error |
  136. | `PLAYWRIGHT_PACKAGE` | `auto` | `auto`, `playwright`, `playwright-core` | Which Playwright client package to resolve when browser mode is enabled |
  137. | `PLAYWRIGHT_MODULE_PATH` | empty | Absolute path or project-relative path | Reuse an existing Playwright client package outside this project |
  138. | `PLAYWRIGHT_EXECUTABLE_PATH` | empty | Any valid browser binary path | Launch an existing Chromium/Chrome executable without installing bundled browsers |
  139. | `PLAYWRIGHT_WS_ENDPOINT` | empty | Valid Playwright `ws://` / `wss://` endpoint | Connect to an existing remote Playwright browser server |
  140. | `PLAYWRIGHT_CDP_ENDPOINT` | empty | Valid Chromium CDP endpoint | Connect to an existing Chromium instance over CDP |
  141. | `PLAYWRIGHT_HEADLESS` | `true` | `true`, `false` | Whether Playwright Chromium runs in headless mode |
  142. | `PLAYWRIGHT_NAVIGATION_TIMEOUT_MS` | `20000` | Positive integer | Timeout for Playwright navigation and Bing result waits |
  143. | `OPEN_WEBSEARCH_PROFILE_DIR` | `<tmpdir>/open-websearch-browser-profiles` | Any writable directory | Base directory for persistent local browser profiles (see browser state note below) |
  144. | `MCP_TOOL_SEARCH_NAME` | `search` | Valid MCP tool name | Custom name for the search tool; set to `<disabled>` (quote as `'<disabled>'` in bash/zsh, `"<disabled>"` in Windows cmd) to disable the tool. Invalid names fallback to default with a warning
  145. | `MCP_TOOL_FETCH_LINUXDO_NAME` | `fetchLinuxDoArticle` | Valid MCP tool name | Custom name for the Linux.do article fetch tool; set to `<disabled>` (quote as `'<disabled>'` in bash/zsh, `"<disabled>"` in Windows cmd) to disable the tool. Invalid names fallback to default with a warning
  146. | `MCP_TOOL_FETCH_CSDN_NAME` | `fetchCsdnArticle` | Valid MCP tool name | Custom name for the CSDN article fetch tool; set to `<disabled>` (quote as `'<disabled>'` in bash/zsh, `"<disabled>"` in Windows cmd) to disable the tool. Invalid names fallback to default with a warning
  147. | `MCP_TOOL_FETCH_GITHUB_NAME` | `fetchGithubReadme` | Valid MCP tool name | Custom name for the GitHub README fetch tool; set to `<disabled>` (quote as `'<disabled>'` in bash/zsh, `"<disabled>"` in Windows cmd) to disable the tool. Invalid names fallback to default with a warning
  148. | `MCP_TOOL_FETCH_JUEJIN_NAME` | `fetchJuejinArticle` | Valid MCP tool name | Custom name for the Juejin article fetch tool; set to `<disabled>` (quote as `'<disabled>'` in bash/zsh, `"<disabled>"` in Windows cmd) to disable the tool. Invalid names fallback to default with a warning
  149. | `MCP_TOOL_FETCH_WEB_NAME` | `fetchWebContent` | Valid MCP tool name | Custom name for generic web/Markdown fetch tool; set to `<disabled>` (quote as `'<disabled>'` in bash/zsh, `"<disabled>"` in Windows cmd) to disable the tool. Invalid names fallback to default with a warning
  150. **Common configurations:**
  151. ```bash
  152. # Enable proxy for restricted regions
  153. USE_PROXY=true PROXY_URL=http://127.0.0.1:7890 npx open-websearch@latest
  154. # Only if a target website has a broken certificate chain
  155. FETCH_WEB_INSECURE_TLS=true npx open-websearch@latest
  156. # Request first, then fallback to Playwright if available
  157. SEARCH_MODE=auto npx open-websearch@latest
  158. # Force request-only Bing search
  159. SEARCH_MODE=request npx open-websearch@latest
  160. # Rename search tool to webSearch
  161. MCP_TOOL_SEARCH_NAME=webSearch npx open-websearch@latest
  162. # Disable search tool
  163. MCP_TOOL_SEARCH_NAME='<disabled>' npx open-websearch@latest
  164. # Full configuration
  165. DEFAULT_SEARCH_ENGINE=duckduckgo ENABLE_CORS=true USE_PROXY=true PROXY_URL=http://127.0.0.1:7890 PORT=8080 npx open-websearch@latest
  166. ```
  167. > **Note:** The `<disabled>` sentinel contains shell-special characters. In bash/zsh, quote it as `'<disabled>'`; in Windows cmd, use double quotes `"<disabled>"`.
  168. Browser-enhanced Bing fallback is opt-in. The published package does not bundle Playwright anymore. Enable it manually with one of these setups:
  169. 1. Full local Playwright install:
  170. ```bash
  171. npm install playwright
  172. npx playwright install chromium
  173. SEARCH_MODE=auto npx open-websearch@latest
  174. ```
  175. 2. Reuse an existing browser binary with a slim client:
  176. ```bash
  177. npm install playwright-core
  178. PLAYWRIGHT_PACKAGE=playwright-core PLAYWRIGHT_EXECUTABLE_PATH=/path/to/chromium SEARCH_MODE=auto npx open-websearch@latest
  179. ```
  180. 3. Reuse a Playwright package that already exists elsewhere on the machine:
  181. ```bash
  182. PLAYWRIGHT_MODULE_PATH=/absolute/path/to/node_modules/playwright SEARCH_MODE=playwright npx open-websearch@latest
  183. ```
  184. 4. Connect to an existing remote browser:
  185. ```bash
  186. npm install playwright-core
  187. PLAYWRIGHT_PACKAGE=playwright-core PLAYWRIGHT_WS_ENDPOINT=ws://127.0.0.1:3000/ SEARCH_MODE=auto npx open-websearch@latest
  188. ```
  189. 5. Reuse a local Chrome/Chromium session over CDP:
  190. ```bash
  191. npm install playwright-core
  192. # Start Chrome/Chromium with a debugging port first
  193. chrome --remote-debugging-port=9222 --user-data-dir=/tmp/open-websearch-chrome
  194. # Then connect through CDP
  195. PLAYWRIGHT_PACKAGE=playwright-core PLAYWRIGHT_CDP_ENDPOINT=http://127.0.0.1:9222 SEARCH_MODE=auto npx open-websearch@latest
  196. ```
  197. This is the most practical setup when you want to reuse your own logged-in or previously verified browser session.
  198. Windows PowerShell example:
  199. ```powershell
  200. npm install playwright-core
  201. & "$env:LOCALAPPDATA\Google\Chrome\Application\chrome.exe" `
  202. --remote-debugging-port=9222 `
  203. --user-data-dir="$env:TEMP\open-websearch-chrome"
  204. $env:PLAYWRIGHT_PACKAGE="playwright-core"
  205. $env:PLAYWRIGHT_CDP_ENDPOINT="http://127.0.0.1:9222"
  206. $env:SEARCH_MODE="auto"
  207. npx open-websearch@latest
  208. ```
  209. Mode behavior:
  210. - `request`: only uses request-based Bing scraping; the search tool exposes no `searchMode` parameter and no mode guidance
  211. - `playwright`: forces Playwright; the search tool exposes no `searchMode` parameter and no mode guidance. Playwright availability is checked at startup and an invalid configuration logs a warning; searches then fail with a clear `browser_unavailable` error
  212. - `auto`: checks whether Playwright is really usable (the client module actually loads; for local launches a real browser binary must exist). If available, the search tool exposes a `searchMode` parameter (request / auto / playwright) and directs the agent to stay on the default `auto`, retrying with `playwright` only when request results fail, return empty, or look blocked; otherwise the server behaves as request mode
  213. Notes:
  214. - `PLAYWRIGHT_MODULE_PATH` takes precedence over `PLAYWRIGHT_PACKAGE`
  215. - `PLAYWRIGHT_WS_ENDPOINT` takes precedence over `PLAYWRIGHT_CDP_ENDPOINT`
  216. - Remote endpoints ignore `PLAYWRIGHT_EXECUTABLE_PATH` and local proxy launch flags
  217. - When Playwright is available, blocked CSDN/Zhihu article fetches and generic web fetches can also retry with browser-acquired cookies
  218. - Without Playwright, `fetchWebContent` stays on the request-only path. Public pages can still work, but pages that require browser cookies or browser-rendered HTML may fail.
  219. Browser state note (local shared profiles):
  220. - Local browser mode reuses persistent contexts/pages across fetches and across process restarts. Cookies, storage, cache, and Service Worker state for the same origin therefore persist between browser-mode fetches.
  221. - This is intentional for single-user, stateful scraping (it keeps anti-bot state warm and avoids repeated browser launches). If you share a daemon between mutually distrusting callers, connect those callers to an isolated browser via `PLAYWRIGHT_WS_ENDPOINT`/`PLAYWRIGHT_CDP_ENDPOINT` instead of relying on local shared profiles, or point each caller at its own `OPEN_WEBSEARCH_PROFILE_DIR`.
  222. - This shared state is service-level and intended only for anonymous access to public web pages: the service itself never logs into sites, so it should never carry personal credentials. Do not rely on it to store logged-in or personalized sessions. Remote endpoints (`PLAYWRIGHT_WS_ENDPOINT`/`PLAYWRIGHT_CDP_ENDPOINT`) inherit the connected browser's state, so pointing them at a personal logged-in browser also shares that browser's login and personalization state with this service.
  223. - To clear profile state, delete the browser profile directories under `OPEN_WEBSEARCH_PROFILE_DIR` (default: `<tmpdir>/open-websearch-browser-profiles`) while no local browser is running.
  224. ### Local Installation
  225. 1. Clone or download this repository
  226. 2. Install dependencies:
  227. ```bash
  228. npm install
  229. ```
  230. This installs the core MCP server only. Browser fallback remains optional until you install or connect a Playwright client yourself.
  231. 3. Build the server:
  232. ```bash
  233. npm run build
  234. ```
  235. 4. Add the server to your MCP configuration:
  236. **Cherry Studio:**
  237. ```json
  238. {
  239. "mcpServers": {
  240. "web-search": {
  241. "name": "Web Search MCP",
  242. "type": "streamableHttp",
  243. "description": "Multi-engine web search with article fetching",
  244. "isActive": true,
  245. "baseUrl": "http://localhost:3000/mcp"
  246. }
  247. }
  248. }
  249. ```
  250. **VSCode (Claude Dev Extension):**
  251. ```json
  252. {
  253. "mcpServers": {
  254. "web-search": {
  255. "transport": {
  256. "type": "streamableHttp",
  257. "url": "http://localhost:3000/mcp"
  258. }
  259. },
  260. "web-search-sse": {
  261. "transport": {
  262. "type": "sse",
  263. "url": "http://localhost:3000/sse"
  264. }
  265. }
  266. }
  267. }
  268. ```
  269. **Claude Desktop:**
  270. ```json
  271. {
  272. "mcpServers": {
  273. "web-search": {
  274. "type": "http",
  275. "url": "http://localhost:3000/mcp"
  276. },
  277. "web-search-sse": {
  278. "type": "sse",
  279. "url": "http://localhost:3000/sse"
  280. }
  281. }
  282. }
  283. ```
  284. **NPX Command Line Configuration:**
  285. ```json
  286. {
  287. "mcpServers": {
  288. "web-search": {
  289. "args": [
  290. "open-websearch@latest"
  291. ],
  292. "command": "npx",
  293. "env": {
  294. "MODE": "stdio",
  295. "DEFAULT_SEARCH_ENGINE": "duckduckgo",
  296. "ALLOWED_SEARCH_ENGINES": "duckduckgo,bing,exa"
  297. }
  298. }
  299. }
  300. }
  301. ```
  302. Windows NPX configuration:
  303. ```json
  304. {
  305. "mcpServers": {
  306. "web-search": {
  307. "command": "cmd",
  308. "args": [
  309. "/c",
  310. "npx",
  311. "-y",
  312. "open-websearch@latest"
  313. ],
  314. "env": {
  315. "MODE": "stdio",
  316. "DEFAULT_SEARCH_ENGINE": "duckduckgo",
  317. "SYSTEMROOT": "C:/Windows"
  318. }
  319. }
  320. }
  321. }
  322. ```
  323. Proxy and TLS notes:
  324. - open-websearch now disables Axios environment-proxy auto-detection internally and only uses the explicit `USE_PROXY` + `PROXY_URL` path.
  325. - When `USE_PROXY=true`, all Axios-based network requests follow the configured `PROXY_URL` path instead of mixing direct requests with environment-proxy behavior.
  326. - If `PROXY_URL` points to a local rule-based proxy client, that client can still decide which destinations go `DIRECT` and which ones are proxied.
  327. - If `PROXY_URL` points to a fixed upstream proxy or overseas egress, region-sensitive sites such as Baidu, CSDN, Juejin, Linux.do, or GitHub may behave differently than before.
  328. - If your host machine already sets `HTTP_PROXY` or `HTTPS_PROXY`, they will no longer override the server's internal request behavior.
  329. - Prefer configuring `NODE_EXTRA_CA_CERTS` on Windows when a site has a missing intermediate CA.
  330. - Use `FETCH_WEB_INSECURE_TLS=true` only as a last resort for the request leg of `fetchWebContent`; it weakens TLS verification and does not affect Playwright browser navigation.
  331. **Local STDIO Configuration for Cherry Studio (Windows):**
  332. ```json
  333. {
  334. "mcpServers": {
  335. "open-websearch-local": {
  336. "command": "node",
  337. "args": ["C:/path/to/your/project/build/index.js"],
  338. "env": {
  339. "MODE": "stdio",
  340. "DEFAULT_SEARCH_ENGINE": "duckduckgo",
  341. "ALLOWED_SEARCH_ENGINES": "duckduckgo,bing,exa"
  342. }
  343. }
  344. }
  345. }
  346. ```
  347. ### Docker Deployment
  348. Quick deployment using Docker Compose:
  349. ```bash
  350. docker-compose up -d
  351. ```
  352. Or use Docker directly:
  353. ```bash
  354. docker run -d --name web-search -p 3000:3000 -e ENABLE_CORS=true -e CORS_ORIGIN=* ghcr.io/aas-ee/open-web-search:latest
  355. ```
  356. Environment variable configuration:
  357. | Variable | Default | Options | Description |
  358. |----------|-------------------------|---------|-------------|
  359. | `ENABLE_CORS` | `false` | `true`, `false` | Enable CORS |
  360. | `CORS_ORIGIN` | `*` | Any valid origin | CORS origin configuration |
  361. | `DEFAULT_SEARCH_ENGINE` | `bing` | `bing`, `duckduckgo`, `exa`, `brave`, `baidu`, `csdn`, `linuxdo`, `juejin`, `startpage`, `sogou`, `hackernews`, `keenable` | Default search engine |
  362. | `USE_PROXY` | `false` | `true`, `false` | Enable HTTP proxy |
  363. | `PROXY_URL` | `http://127.0.0.1:7890` | Any valid URL | Proxy server URL |
  364. | `FAKE_IP_CIDRS` | empty | Comma-separated CIDR list | Treat DNS answers in these CIDRs as synthetic fake-IP results and do not block them as private-network DNS answers. Literal private/local targets and other private-network DNS answers remain blocked |
  365. | `PORT` | `3000` | 1-65535 | Server port |
  366. Then configure in your MCP client:
  367. ```json
  368. {
  369. "mcpServers": {
  370. "web-search": {
  371. "name": "Web Search MCP",
  372. "type": "streamableHttp",
  373. "description": "Multi-engine web search with article fetching",
  374. "isActive": true,
  375. "baseUrl": "http://localhost:3000/mcp"
  376. },
  377. "web-search-sse": {
  378. "transport": {
  379. "name": "Web Search MCP",
  380. "type": "sse",
  381. "description": "Multi-engine web search with article fetching",
  382. "isActive": true,
  383. "url": "http://localhost:3000/sse"
  384. }
  385. }
  386. }
  387. }
  388. ```
  389. ## Usage Guide
  390. The server provides six tools: `search`, `fetchLinuxDoArticle`, `fetchCsdnArticle`, `fetchGithubReadme`, `fetchJuejinArticle`, and `fetchWebContent`.
  391. For the local daemon HTTP API (`serve`, `status`, `GET /health`, `POST /search`, `POST /fetch-*`), see [docs/http-api.md](docs/http-api.md).
  392. ### search Tool Usage
  393. ```typescript
  394. {
  395. "query": string, // Search query
  396. "limit": number, // Optional: Number of results to return (default: 10)
  397. "engines": string[], // Optional: Engines to use (bing,baidu,linuxdo,csdn,duckduckgo,exa,brave,juejin,startpage,sogou,hackernews,keenable) default runtime-configured engine
  398. "searchMode": string // Optional: request, auto, or playwright (currently only affects Bing)
  399. }
  400. ```
  401. Usage example:
  402. ```typescript
  403. use_mcp_tool({
  404. server_name: "web-search",
  405. tool_name: "search",
  406. arguments: {
  407. query: "search content",
  408. limit: 3, // Optional parameter
  409. engines: ["bing", "csdn", "duckduckgo", "exa", "brave", "juejin", "sogou"] // Optional parameter, supports multi-engine combined search
  410. }
  411. })
  412. ```
  413. Response example:
  414. ```json
  415. [
  416. {
  417. "title": "Example Search Result",
  418. "url": "https://example.com",
  419. "description": "Description text of the search result...",
  420. "source": "Source",
  421. "engine": "Engine used"
  422. }
  423. ]
  424. ```
  425. ### fetchCsdnArticle Tool Usage
  426. Used to fetch complete content of CSDN blog articles.
  427. ```typescript
  428. {
  429. "url": string // URL from CSDN search results using the search tool
  430. }
  431. ```
  432. Usage example:
  433. ```typescript
  434. use_mcp_tool({
  435. server_name: "web-search",
  436. tool_name: "fetchCsdnArticle",
  437. arguments: {
  438. url: "https://blog.csdn.net/xxx/article/details/xxx"
  439. }
  440. })
  441. ```
  442. Response example:
  443. ```json
  444. [
  445. {
  446. "content": "Example search result"
  447. }
  448. ]
  449. ```
  450. ### fetchLinuxDoArticle Tool Usage
  451. Used to fetch complete content of Linux.do forum articles.
  452. ```typescript
  453. {
  454. "url": string // URL from linuxdo search results using the search tool
  455. }
  456. ```
  457. Usage example:
  458. ```typescript
  459. use_mcp_tool({
  460. server_name: "web-search",
  461. tool_name: "fetchLinuxDoArticle",
  462. arguments: {
  463. url: "https://xxxx.json"
  464. }
  465. })
  466. ```
  467. Response example:
  468. ```json
  469. [
  470. {
  471. "content": "Example search result"
  472. }
  473. ]
  474. ```
  475. ### fetchGithubReadme Tool Usage
  476. Used to fetch README content from GitHub repositories.
  477. ```typescript
  478. {
  479. "url": string // GitHub repository URL (supports HTTPS, SSH formats)
  480. }
  481. ```
  482. Usage example:
  483. ```typescript
  484. use_mcp_tool({
  485. server_name: "web-search",
  486. tool_name: "fetchGithubReadme",
  487. arguments: {
  488. url: "https://github.com/Aas-ee/open-webSearch"
  489. }
  490. })
  491. ```
  492. Supported URL formats:
  493. - HTTPS: `https://github.com/owner/repo`
  494. - HTTPS with .git: `https://github.com/owner/repo.git`
  495. - SSH: `git@github.com:owner/repo.git`
  496. - URLs with parameters: `https://github.com/owner/repo?tab=readme`
  497. Response example:
  498. ```json
  499. [
  500. {
  501. "content": "<div align=\"center\">\n\n# Open-WebSearch MCP Server..."
  502. }
  503. ]
  504. ```
  505. ### fetchWebContent Tool Usage
  506. Fetch content directly from public HTTP(S) links, including Markdown files (`.md`), ordinary pages, and JavaScript-rendered pages when Playwright is configured.
  507. ```typescript
  508. {
  509. "url": string, // Public HTTP(S) URL
  510. "maxChars": number, // Optional: max returned content length (1000-200000, default 30000)
  511. "renderMode": string, // Optional: request, auto (default), or browser
  512. "readability": boolean, // Optional: use Mozilla Readability for HTML
  513. "includeLinks": boolean // Optional: preserve links from Readability output
  514. }
  515. ```
  516. `request` never starts a browser or uses browser cookies. `auto` preserves the existing request-first behavior and uses browser assistance only when needed. `browser` renders the page directly and returns a clear error if Playwright or its browser target is unavailable. Initial and final URLs remain subject to public-network safety checks.
  517. Browser requests are revalidated before continuation, but this process does not pin DNS answers to Chromium's eventual socket. Remote Playwright/CDP endpoints must therefore enforce their own trusted DNS and egress policy.
  518. Usage example:
  519. ```typescript
  520. use_mcp_tool({
  521. server_name: "web-search",
  522. tool_name: "fetchWebContent",
  523. arguments: {
  524. url: "https://raw.githubusercontent.com/Aas-ee/open-webSearch/main/README.md",
  525. maxChars: 12000,
  526. renderMode: "auto"
  527. }
  528. })
  529. ```
  530. Response example:
  531. ```json
  532. {
  533. "url": "https://raw.githubusercontent.com/Aas-ee/open-webSearch/main/README.md",
  534. "finalUrl": "https://raw.githubusercontent.com/Aas-ee/open-webSearch/main/README.md",
  535. "contentType": "text/plain; charset=utf-8",
  536. "title": "",
  537. "retrievalMethod": "request",
  538. "truncated": false,
  539. "content": "# Open-WebSearch MCP Server ..."
  540. }
  541. ```
  542. ### fetchJuejinArticle Tool Usage
  543. Used to fetch complete content of Juejin articles.
  544. ```typescript
  545. {
  546. "url": string // Juejin article URL from search results
  547. }
  548. ```
  549. Usage example:
  550. ```typescript
  551. use_mcp_tool({
  552. server_name: "web-search",
  553. tool_name: "fetchJuejinArticle",
  554. arguments: {
  555. url: "https://juejin.cn/post/7520959840199360563"
  556. }
  557. })
  558. ```
  559. Supported URL format:
  560. - `https://juejin.cn/post/{article_id}`
  561. Response example:
  562. ```json
  563. [
  564. {
  565. "content": "🚀 开源 AI 联网搜索工具:Open-WebSearch MCP 全新升级,支持多引擎 + 流式响应..."
  566. }
  567. ]
  568. ```
  569. ## Usage Limitations
  570. Since this tool works by scraping multi-engine search results, please note the following important limitations:
  571. 1. **Rate Limiting**:
  572. - Too many searches in a short time may cause the used engines to temporarily block requests
  573. - Recommendations:
  574. - Maintain reasonable search frequency
  575. - Use the limit parameter judiciously
  576. - Add delays between searches when necessary
  577. 2. **Result Accuracy**:
  578. - Depends on the HTML structure of corresponding engines, may fail when engines update
  579. - Some results may lack metadata like descriptions
  580. - Complex search operators may not work as expected
  581. 3. **Legal Terms**:
  582. - This tool is for personal use only
  583. - Please comply with the terms of service of corresponding engines
  584. - Implement appropriate rate limiting based on your actual use case
  585. 4. **Search Engine Configuration**:
  586. - Default search engine can be set via the `DEFAULT_SEARCH_ENGINE` environment variable
  587. - Supported engines: bing, duckduckgo, exa, brave, baidu, csdn, linuxdo, juejin, startpage, sogou, hackernews, keenable
  588. - The default engine is used when searching specific websites
  589. 5. **Proxy Configuration**:
  590. - HTTP proxy can be configured when certain search engines are unavailable in specific regions
  591. - Enable proxy with environment variable `USE_PROXY=true`
  592. - Configure proxy server address with `PROXY_URL`
  593. - For Clash fake-ip / TUN setups, configure synthetic DNS ranges with `FAKE_IP_CIDRS` (for example `198.18.0.0/15`)
  594. ## Contributing
  595. Welcome to submit issue reports and feature improvement suggestions!
  596. ### Contributor Guide
  597. If you want to fork this repository and publish your own Docker image, you need to make the following configurations:
  598. #### GitHub Secrets Configuration
  599. To enable automatic Docker image building and publishing, please add the following secrets in your GitHub repository settings (Settings → Secrets and variables → Actions):
  600. **Required Secrets:**
  601. - `GITHUB_TOKEN`: Automatically provided by GitHub (no setup needed)
  602. **Optional Secrets (for Alibaba Cloud ACR):**
  603. - `ACR_REGISTRY`: Your Alibaba Cloud Container Registry URL (e.g., `registry.cn-hangzhou.aliyuncs.com`)
  604. - `ACR_USERNAME`: Your Alibaba Cloud ACR username
  605. - `ACR_PASSWORD`: Your Alibaba Cloud ACR password
  606. - `ACR_IMAGE_NAME`: Your image name in ACR (e.g., `your-namespace/open-web-search`)
  607. #### CI/CD Workflow
  608. The repository includes a GitHub Actions workflow (`.github/workflows/docker.yml`) that automatically:
  609. 1. **Trigger Conditions**:
  610. - Push to `main` branch
  611. - Push version tags (`v*`)
  612. - Manual workflow trigger
  613. 2. **Build and Push to**:
  614. - GitHub Container Registry (ghcr.io) - always enabled
  615. - Alibaba Cloud Container Registry - only enabled when ACR secrets are configured
  616. 3. **Image Tags**:
  617. - `ghcr.io/your-username/open-web-search:latest`
  618. - `your-acr-address/your-image-name:latest` (if ACR is configured)
  619. #### Fork and Publish Steps:
  620. 1. **Fork the repository** to your GitHub account
  621. 2. **Configure secrets** (if you need ACR publishing):
  622. - Go to Settings → Secrets and variables → Actions in your forked repository
  623. - Add the ACR-related secrets listed above
  624. 3. **Push changes** to the `main` branch or create version tags
  625. 4. **GitHub Actions will automatically build and push** your Docker image
  626. 5. **Use your image**, update the Docker command:
  627. ```bash
  628. docker run -d --name web-search -p 3000:3000 -e ENABLE_CORS=true -e CORS_ORIGIN=* ghcr.io/your-username/open-web-search:latest
  629. ```
  630. #### Notes:
  631. - If you don't configure ACR secrets, the workflow will only publish to GitHub Container Registry
  632. - Make sure your GitHub repository has Actions enabled
  633. - The workflow will use your GitHub username (converted to lowercase) as the GHCR image name
  634. <div align="center">
  635. ## Star History
  636. If you find this project helpful, please consider giving it a ⭐ Star!
  637. [![Star History Chart](https://star-history.dera.page/svg?repos=Aas-ee/open-webSearch&type=Date)](https://star-history.dera.page/#Aas-ee/open-webSearch&Date)
  638. </div>

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.