Skip to content
Back to search

mcp-read-website-fast

Quickly reads webpages and converts to markdown for fast, token efficient web scraping

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/just-every--mcp-read-website-fast/badge.svg)](/listings/just-every--mcp-read-website-fast)

HTML

<a href="/listings/just-every--mcp-read-website-fast"><img src="/api/listings/just-every--mcp-read-website-fast/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Claude Code

claude mcp add read-website-fast -s user -- npx -y @just-every/mcp-read-website-fast

VS Code

code --add-mcp '{"name":"read-website-fast","command":"npx","args":["-y","@just-every/mcp-read-website-fast"]}'

Cursor

cursor://anysphere.cursor-deeplink/mcp/install?name=read-website-fast&config=eyJyZWFkLXdlYnNpdGUtZmFzdCI6eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqdXN0LWV2ZXJ5L21jcC1yZWFkLXdlYnNpdGUtZmFzdCJdfX0=

JetBrains IDEs

Settings → Tools → AI Assistant → Model Context Protocol (MCP) → Add

Choose “As JSON” and paste:

{"command":"npx","args":["-y","@just-every/mcp-read-website-fast"]}

Or, in the chat window, type /add and fill in the same JSON—both paths land the server in a single step. 

Raw JSON (works in any MCP client)

{
  "mcpServers": {
    "read-website-fast": {
      "command": "npx",
      "args": ["-y", "@just-every/mcp-read-website-fast"]
    }
  }
}

Drop this into your client’s mcp.json (e.g. .vscode/mcp.json, ~/.cursor/mcp.json, or .mcp.json for Claude).

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 229 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # @just-every/mcp-read-website-fast
  2. Fast, token-efficient web content extraction for AI agents - converts websites to clean Markdown.
  3. [![npm version](https://badge.fury.io/js/@just-every%2Fmcp-read-website-fast.svg)](https://www.npmjs.com/package/@just-every/mcp-read-website-fast)
  4. [![GitHub Actions](https://github.com/just-every/mcp-read-website-fast/workflows/Release/badge.svg)](https://github.com/just-every/mcp-read-website-fast/actions)
  5. <a href="https://glama.ai/mcp/servers/@just-every/mcp-read-website-fast">
  6. <img width="380" height="200" src="https://glama.ai/mcp/servers/@just-every/mcp-read-website-fast/badge" alt="read-website-fast MCP server" />
  7. </a>
  8. ## Overview
  9. Existing MCP web crawlers are slow and consume large quantities of tokens. This pauses the development process and provides incomplete results as LLMs need to parse whole web pages.
  10. This MCP package fetches web pages locally, strips noise, and converts content to clean Markdown while preserving links. Designed for Claude Code, IDEs and LLM pipelines with minimal token footprint. Crawl sites locally with minimal dependencies.
  11. **Note:** This package now uses [@just-every/crawl](https://www.npmjs.com/package/@just-every/crawl) for its core crawling and markdown conversion functionality.
  12. ## Features
  13. - **Fast startup** using official MCP SDK with lazy loading for optimal performance
  14. - **Content extraction** using Mozilla Readability (same as Firefox Reader View)
  15. - **HTML to Markdown** conversion with Turndown + GFM support
  16. - **Smart caching** with SHA-256 hashed URLs
  17. - **Polite crawling** with robots.txt support and rate limiting
  18. - **Concurrent fetching** with configurable depth crawling
  19. - **Stream-first design** for low memory usage
  20. - **Link preservation** for knowledge graphs
  21. - **Optional chunking** for downstream processing
  22. ## Installation
  23. ### Claude Code
  24. ```bash
  25. claude mcp add read-website-fast -s user -- npx -y @just-every/mcp-read-website-fast
  26. ```
  27. ### VS Code
  28. ```bash
  29. code --add-mcp '{"name":"read-website-fast","command":"npx","args":["-y","@just-every/mcp-read-website-fast"]}'
  30. ```
  31. ### Cursor
  32. ```bash
  33. cursor://anysphere.cursor-deeplink/mcp/install?name=read-website-fast&config=eyJyZWFkLXdlYnNpdGUtZmFzdCI6eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqdXN0LWV2ZXJ5L21jcC1yZWFkLXdlYnNpdGUtZmFzdCJdfX0=
  34. ```
  35. ### JetBrains IDEs
  36. Settings → Tools → AI Assistant → Model Context Protocol (MCP) → Add
  37. Choose “As JSON” and paste:
  38. ```json
  39. {"command":"npx","args":["-y","@just-every/mcp-read-website-fast"]}
  40. ```
  41. Or, in the chat window, type /add and fill in the same JSON—both paths land the server in a single step. 
  42. ### Raw JSON (works in any MCP client)
  43. ```json
  44. {
  45. "mcpServers": {
  46. "read-website-fast": {
  47. "command": "npx",
  48. "args": ["-y", "@just-every/mcp-read-website-fast"]
  49. }
  50. }
  51. }
  52. ```
  53. Drop this into your client’s mcp.json (e.g. .vscode/mcp.json, ~/.cursor/mcp.json, or .mcp.json for Claude).
  54. ## Features
  55. - **Fast startup** using official MCP SDK with lazy loading for optimal performance
  56. - **Content extraction** using Mozilla Readability (same as Firefox Reader View)
  57. - **HTML to Markdown** conversion with Turndown + GFM support
  58. - **Smart caching** with SHA-256 hashed URLs
  59. - **Polite crawling** with robots.txt support and rate limiting
  60. - **Concurrent fetching** with configurable depth crawling
  61. - **Stream-first design** for low memory usage
  62. - **Link preservation** for knowledge graphs
  63. - **Optional chunking** for downstream processing
  64. ### Available Tools
  65. - `read_website` - Fetches a webpage and converts it to clean markdown
  66. - Parameters:
  67. - `url` (required): The HTTP/HTTPS URL to fetch
  68. - `pages` (optional): Maximum number of pages to crawl (default: 1, max: 100)
  69. ### Available Resources
  70. - `read-website-fast://status` - Get cache statistics
  71. - `read-website-fast://clear-cache` - Clear the cache directory
  72. ## Development Usage
  73. ### Install
  74. ```bash
  75. npm install
  76. npm run build
  77. ```
  78. ### Single page fetch
  79. ```bash
  80. npm run dev fetch https://example.com/article
  81. ```
  82. ### Crawl with depth
  83. ```bash
  84. npm run dev fetch https://example.com --depth 2 --concurrency 5
  85. ```
  86. ### Output formats
  87. ```bash
  88. # Markdown only (default)
  89. npm run dev fetch https://example.com
  90. # JSON output with metadata
  91. npm run dev fetch https://example.com --output json
  92. # Both URL and markdown
  93. npm run dev fetch https://example.com --output both
  94. ```
  95. ### CLI Options
  96. - `-p, --pages <number>` - Maximum number of pages to crawl (default: 1)
  97. - `-c, --concurrency <number>` - Max concurrent requests (default: 3)
  98. - `--no-robots` - Ignore robots.txt
  99. - `--all-origins` - Allow cross-origin crawling
  100. - `-u, --user-agent <string>` - Custom user agent
  101. - `--cache-dir <path>` - Cache directory (default: .cache)
  102. - `-t, --timeout <ms>` - Request timeout in milliseconds (default: 30000)
  103. - `-o, --output <format>` - Output format: json, markdown, or both (default: markdown)
  104. ### Clear cache
  105. ```bash
  106. npm run dev clear-cache
  107. ```
  108. ## Auto-Restart Feature
  109. The MCP server includes automatic restart capability by default for improved reliability:
  110. - Automatically restarts the server if it crashes
  111. - Handles unhandled exceptions and promise rejections
  112. - Implements exponential backoff (max 10 attempts in 1 minute)
  113. - Logs all restart attempts for monitoring
  114. - Gracefully handles shutdown signals (SIGINT, SIGTERM)
  115. For development/debugging without auto-restart:
  116. ```bash
  117. # Run directly without restart wrapper
  118. npm run serve:dev
  119. ```
  120. ## Architecture
  121. ```
  122. mcp/
  123. ├── src/
  124. │ ├── crawler/ # URL fetching, queue management, robots.txt
  125. │ ├── parser/ # DOM parsing, Readability, Turndown conversion
  126. │ ├── cache/ # Disk-based caching with SHA-256 keys
  127. │ ├── utils/ # Logger, chunker utilities
  128. │ ├── index.ts # CLI entry point
  129. │ ├── serve.ts # MCP server entry point
  130. │ └── serve-restart.ts # Auto-restart wrapper
  131. ```
  132. ## Development
  133. ```bash
  134. # Run in development mode
  135. npm run dev fetch https://example.com
  136. # Build for production
  137. npm run build
  138. # Run tests
  139. npm test
  140. # Type checking
  141. npm run typecheck
  142. # Linting
  143. npm run lint
  144. ```
  145. ## Contributing
  146. Contributions are welcome! Please:
  147. 1. Fork the repository
  148. 2. Create a feature branch
  149. 3. Add tests for new functionality
  150. 4. Submit a pull request
  151. ## Troubleshooting
  152. ### Cache Issues
  153. ```bash
  154. npm run dev clear-cache
  155. ```
  156. ### Timeout Errors
  157. - Increase timeout with `-t` flag
  158. - Check network connectivity
  159. - Verify URL is accessible
  160. ### Content Not Extracted
  161. - Some sites block automated access
  162. - Try custom user agent with `-u` flag
  163. - Check if site requires JavaScript (not supported)
  164. ## License
  165. MIT

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.