Skip to content
Back to search

google-search

A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/web-agent-master--google-search/badge.svg)](/listings/web-agent-master--google-search)

HTML

<a href="/listings/web-agent-master--google-search"><img src="/api/listings/web-agent-master--google-search/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

# Install from source
git clone https://github.com/web-agent-master/google-search.git
cd google-search
# Install dependencies
npm install
# Or using yarn
yarn
# Or using pnpm
pnpm install

# Compile TypeScript code
npm run build
# Or using yarn
yarn build
# Or using pnpm
pnpm build

# Link package globally (required for MCP functionality)
npm link
# Or using yarn
yarn link
# Or using pnpm
pnpm link

Windows Environment Notes

This tool has been specially adapted for Windows environments:

1. `.cmd` files are provided to ensure command-line tools work properly in Windows Command Prompt and PowerShell 2. Log files are stored in the system temporary directory instead of the Unix/Linux `/tmp` directory 3. Windows-specific process signal handling has been added to ensure proper server shutdown 4. Cross-platform file path handling is used to support Windows path separators

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 353 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Google Search Tool
  2. A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches and extract results. It can be used directly as a command-line tool or as a Model Context Protocol (MCP) server to provide real-time search capabilities to AI assistants like Claude.
  3. [![Star History Chart](https://api.star-history.com/svg?repos=web-agent-master/google-search&type=Date)](https://star-history.com/#web-agent-master/google-search&Date)
  4. [中文文档](README.zh-CN.md)
  5. ## Key Features
  6. - **Local SERP API Alternative**: No need to rely on paid search engine results API services, all searches are executed locally
  7. - **Advanced Anti-Bot Detection Bypass Techniques**:
  8. - Intelligent browser fingerprint management that simulates real user behavior
  9. - Automatic saving and restoration of browser state to reduce verification frequency
  10. - Smart headless/headed mode switching, automatically switching to headed mode when verification is needed
  11. - Randomization of device and locale settings to reduce detection risk
  12. - **Raw HTML Retrieval**: Ability to fetch the raw HTML of search result pages (with CSS and JavaScript removed) for analysis and debugging when Google's page structure changes
  13. - **Page Screenshot**: Automatically captures and saves a full-page screenshot when saving HTML content
  14. - **MCP Server Integration**: Provides real-time search capabilities to AI assistants like Claude without requiring additional API keys
  15. - **Completely Open Source and Free**: All code is open source with no usage restrictions, freely customizable and extensible
  16. ## Technical Features
  17. - Developed with TypeScript, providing type safety and better development experience
  18. - Browser automation based on Playwright, supporting multiple browser engines
  19. - Command-line parameter support for search keywords
  20. - MCP server support for AI assistant integration
  21. - Returns search results with title, link, and snippet
  22. - Option to retrieve raw HTML of search result pages for analysis
  23. - JSON format output
  24. - Support for both headless and headed modes (for debugging)
  25. - Detailed logging output
  26. - Robust error handling
  27. - Browser state saving and restoration to effectively avoid anti-bot detection
  28. ## Installation
  29. ```bash
  30. # Install from source
  31. git clone https://github.com/web-agent-master/google-search.git
  32. cd google-search
  33. # Install dependencies
  34. npm install
  35. # Or using yarn
  36. yarn
  37. # Or using pnpm
  38. pnpm install
  39. # Compile TypeScript code
  40. npm run build
  41. # Or using yarn
  42. yarn build
  43. # Or using pnpm
  44. pnpm build
  45. # Link package globally (required for MCP functionality)
  46. npm link
  47. # Or using yarn
  48. yarn link
  49. # Or using pnpm
  50. pnpm link
  51. ```
  52. ### Windows Environment Notes
  53. This tool has been specially adapted for Windows environments:
  54. 1. `.cmd` files are provided to ensure command-line tools work properly in Windows Command Prompt and PowerShell
  55. 2. Log files are stored in the system temporary directory instead of the Unix/Linux `/tmp` directory
  56. 3. Windows-specific process signal handling has been added to ensure proper server shutdown
  57. 4. Cross-platform file path handling is used to support Windows path separators
  58. ## Usage
  59. ### Command Line Tool
  60. ```bash
  61. # Direct command line usage
  62. google-search "search keywords"
  63. # Using command line options
  64. google-search --limit 5 --timeout 60000 --no-headless "search keywords"
  65. # Or using npx
  66. npx google-search-cli "search keywords"
  67. # Run in development mode
  68. pnpm dev "search keywords"
  69. # Run in debug mode (showing browser interface)
  70. pnpm debug "search keywords"
  71. # Get raw HTML of search result page
  72. google-search "search keywords" --get-html
  73. # Get HTML and save to file
  74. google-search "search keywords" --get-html --save-html
  75. # Get HTML and save to specific file
  76. google-search "search keywords" --get-html --save-html --html-output "./output.html"
  77. ```
  78. #### Command Line Options
  79. - `-l, --limit <number>`: Result count limit (default: 10)
  80. - `-t, --timeout <number>`: Timeout in milliseconds (default: 60000)
  81. - `--no-headless`: Show browser interface (for debugging)
  82. - `--remote-debugging-port <number>`: Enable remote debugging port (default: 9222)
  83. - `--state-file <path>`: Browser state file path (default: ./browser-state.json)
  84. - `--no-save-state`: Don't save browser state
  85. - `--get-html`: Retrieve raw HTML of search result page instead of parsing results
  86. - `--save-html`: Save HTML to file (used with --get-html)
  87. - `--html-output <path>`: Specify HTML output file path (used with --get-html and --save-html)
  88. - `-V, --version`: Display version number
  89. - `-h, --help`: Display help information
  90. #### Output Example
  91. ```json
  92. {
  93. "query": "deepseek",
  94. "results": [
  95. {
  96. "title": "DeepSeek",
  97. "link": "https://www.deepseek.com/",
  98. "snippet": "DeepSeek-R1 is now live and open source, rivaling OpenAI's Model o1. Available on web, app, and API. Click for details. Into ..."
  99. },
  100. {
  101. "title": "DeepSeek",
  102. "link": "https://www.deepseek.com/",
  103. "snippet": "DeepSeek-R1 is now live and open source, rivaling OpenAI's Model o1. Available on web, app, and API. Click for details. Into ..."
  104. },
  105. {
  106. "title": "deepseek-ai/DeepSeek-V3",
  107. "link": "https://github.com/deepseek-ai/DeepSeek-V3",
  108. "snippet": "We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token."
  109. }
  110. // More results...
  111. ]
  112. }
  113. ```
  114. #### HTML Output Example
  115. When using the `--get-html` option, the output will include information about the HTML content:
  116. ```json
  117. {
  118. "query": "playwright automation",
  119. "url": "https://www.google.com/",
  120. "originalHtmlLength": 1291733,
  121. "cleanedHtmlLength": 456789,
  122. "htmlPreview": "<!DOCTYPE html><html itemscope=\"\" itemtype=\"http://schema.org/SearchResultsPage\" lang=\"zh-CN\"><head><meta charset=\"UTF-8\"><meta content=\"dark light\" name=\"color-scheme\"><meta content=\"origin\" name=\"referrer\">..."
  123. }
  124. ```
  125. If you also use the `--save-html` option, the output will include the path where the HTML was saved:
  126. ```json
  127. {
  128. "query": "playwright automation",
  129. "url": "https://www.google.com/",
  130. "originalHtmlLength": 1292241,
  131. "cleanedHtmlLength": 458976,
  132. "savedPath": "./google-search-html/playwright_automation-2025-04-06T03-30-06-852Z.html",
  133. "screenshotPath": "./google-search-html/playwright_automation-2025-04-06T03-30-06-852Z.png",
  134. "htmlPreview": "<!DOCTYPE html><html itemscope=\"\" itemtype=\"http://schema.org/SearchResultsPage\" lang=\"zh-CN\">..."
  135. }
  136. ```
  137. ### MCP Server
  138. This project provides Model Context Protocol (MCP) server functionality, allowing AI assistants like Claude to directly use Google search capabilities. MCP is an open protocol that enables AI assistants to safely access external tools and data.
  139. ```bash
  140. # Build the project
  141. pnpm build
  142. ```
  143. #### Integration with Claude Desktop
  144. 1. Edit the Claude Desktop configuration file:
  145. - Mac: `~/Library/Application Support/Claude/claude_desktop_config.json`
  146. - Windows: `%APPDATA%\Claude\claude_desktop_config.json`
  147. - Usually located at `C:\Users\username\AppData\Roaming\Claude\claude_desktop_config.json`
  148. - You can access it directly by entering `%APPDATA%\Claude` in Windows Explorer address bar
  149. 2. Add server configuration and restart Claude
  150. ```json
  151. {
  152. "mcpServers": {
  153. "google-search": {
  154. "command": "npx",
  155. "args": ["google-search-mcp"]
  156. }
  157. }
  158. }
  159. ```
  160. For Windows environments, you can also use the following configurations:
  161. 1. Using cmd.exe with npx:
  162. ```json
  163. {
  164. "mcpServers": {
  165. "google-search": {
  166. "command": "cmd.exe",
  167. "args": ["/c", "npx", "google-search-mcp"]
  168. }
  169. }
  170. }
  171. ```
  172. 2. Using node with full path (recommended if you encounter issues with the above method):
  173. ```json
  174. {
  175. "mcpServers": {
  176. "google-search": {
  177. "command": "node",
  178. "args": ["C:/path/to/your/google-search/dist/src/mcp-server.js"]
  179. }
  180. }
  181. }
  182. ```
  183. Note: For the second method, you must replace `C:/path/to/your/google-search` with the actual full path to where you installed the google-search package.
  184. After integration, you can directly use search functionality in Claude, such as "search for the latest AI research".
  185. ## Project Structure
  186. ```
  187. google-search/
  188. ├── package.json # Project configuration and dependencies
  189. ├── tsconfig.json # TypeScript configuration
  190. ├── src/
  191. │ ├── index.ts # Entry file (command line parsing and main logic)
  192. │ ├── search.ts # Search functionality implementation (Playwright browser automation)
  193. │ ├── mcp-server.ts # MCP server implementation
  194. │ └── types.ts # Type definitions (interfaces and type declarations)
  195. ├── dist/ # Compiled JavaScript files
  196. ├── bin/ # Executable files
  197. │ └── google-search # Command line entry script
  198. ├── README.md # Project documentation
  199. └── .gitignore # Git ignore file
  200. ```
  201. ## Technology Stack
  202. - **TypeScript**: Development language, providing type safety and better development experience
  203. - **Node.js**: Runtime environment for executing JavaScript/TypeScript code
  204. - **Playwright**: For browser automation, supporting multiple browsers
  205. - **Commander**: For parsing command line arguments and generating help information
  206. - **Model Context Protocol (MCP)**: Open protocol for AI assistant integration
  207. - **MCP SDK**: Development toolkit for implementing MCP servers
  208. - **Zod**: Schema definition library for validation and type safety
  209. - **pnpm**: Efficient package management tool, saving disk space and installation time
  210. ## Development Guide
  211. All commands can be run in the project root directory:
  212. ```bash
  213. # Install dependencies
  214. pnpm install
  215. # Install Playwright browsers
  216. pnpm run postinstall
  217. # Compile TypeScript code
  218. pnpm build
  219. # Clean compiled output
  220. pnpm clean
  221. ```
  222. ### CLI Development
  223. ```bash
  224. # Run in development mode
  225. pnpm dev "search keywords"
  226. # Run in debug mode (showing browser interface)
  227. pnpm debug "search keywords"
  228. # Run compiled code
  229. pnpm start "search keywords"
  230. # Test search functionality
  231. pnpm test
  232. ```
  233. ### MCP Server Development
  234. ```bash
  235. # Run MCP server in development mode
  236. pnpm mcp
  237. # Run compiled MCP server
  238. pnpm mcp:build
  239. ```
  240. ## Error Handling
  241. The tool has built-in robust error handling mechanisms:
  242. - Friendly error messages when browser startup fails
  243. - Automatic error status return for network connection issues
  244. - Detailed logs for search result parsing failures
  245. - Graceful exit and useful information return in timeout situations
  246. ## Notes
  247. ### General Notes
  248. - This tool is for learning and research purposes only
  249. - Please comply with Google's terms of service and policies
  250. - Do not send requests too frequently to avoid being blocked by Google
  251. - Some regions may require a proxy to access Google
  252. - Playwright needs to install browsers, which will be automatically downloaded on first use
  253. ### State Files
  254. - State files contain browser cookies and storage data, please keep them secure
  255. - Using state files can effectively avoid Google's anti-bot detection and improve search success rate
  256. ### MCP Server
  257. - MCP server requires Node.js v16 or higher
  258. - When using the MCP server, please ensure Claude Desktop is updated to the latest version
  259. - When configuring Claude Desktop, use absolute paths to the MCP server file
  260. ### Windows-Specific Notes
  261. - In Windows environments, you may need administrator privileges to install Playwright browsers for the first time
  262. - If you encounter permission issues, try running Command Prompt or PowerShell as administrator
  263. - Windows Firewall may block Playwright browser network connections; allow access when prompted
  264. - Browser state files are saved by default in the user's home directory as `.google-search-browser-state.json`
  265. - Log files are stored in the system temporary directory under the `google-search-logs` folder
  266. ## Comparison with Commercial SERP APIs
  267. Compared to paid search engine results API services (such as SerpAPI), this project offers the following advantages:
  268. - **Completely Free**: No API call fees
  269. - **Local Execution**: All searches are executed locally, no dependency on third-party services
  270. - **Privacy Protection**: Search queries are not recorded by third parties
  271. - **Customizability**: Fully open source, can be modified and extended as needed
  272. - **No Usage Limits**: Not subject to API call count or frequency limitations
  273. - **MCP Integration**: Native support for integration with AI assistants like Claude

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.