Skip to content
Back to search

MinerU-Document-Explorer

Agent-native knowledge engine with MCP tools for document indexing, wiki organization, fast retrieval and deep reading across PDF/DOCX/PPTX/Markdown

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/opendatalab--mineru-document-explorer/badge.svg)](/listings/opendatalab--mineru-document-explorer)

HTML

<a href="/listings/opendatalab--mineru-document-explorer"><img src="/api/listings/opendatalab--mineru-document-explorer/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

> Agent-Assisted Setup: If you're using an AI agent (Claude Code, Cursor, etc.), simply ask it to help you deploy MinerU Document Explorer and install skills — the agent can handle the entire setup process for you, including MCP configuration.

Follow the https://github.com/opendatalab/MinerU-Document-Explorer/blob/main/docs/quickstart.md to install MinerU Document Explorer and walk the user through configuration.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 29 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "qmd",
  3. "owner": {
  4. "name": "opendatalab",
  5. "email": ""
  6. },
  7. "plugins": [
  8. {
  9. "name": "qmd",
  10. "source": "./",
  11. "description": "Search and retrieve documents from local markdown files.",
  12. "version": "0.1.0",
  13. "author": {
  14. "name": "opendatalab",
  15. "email": ""
  16. },
  17. "repository": "https://github.com/opendatalab/MinerU-Document-Explorer",
  18. "license": "MIT",
  19. "keywords": ["markdown", "search", "qmd"],
  20. "skills": ["./skills/"],
  21. "mcpServers": {
  22. "qmd": {
  23. "command": "qmd",
  24. "args": ["mcp"]
  25. }
  26. }
  27. }
  28. ]
  29. }
README.md · README · 288 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. <div align="center">
  2. <h1 align="center">
  3. <img src="assets/logo.png" alt="logo" height="28" style="vertical-align: middle; margin-right: 8px;">
  4. MinerU Document Explorer
  5. </h1>
  6. <h4>Agent-native knowledge engine — search, deep-read, and build knowledge bases<br>from Markdown, PDF, DOCX, and PPTX.</h4>
  7. <p>
  8. <a href="https://www.npmjs.com/package/mineru-document-explorer"><img src="https://img.shields.io/npm/v/mineru-document-explorer?style=flat-square&color=cb3837" alt="npm"/></a>
  9. <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue?style=flat-square" alt="license"/></a>
  10. <a href="https://github.com/opendatalab/MinerU-Document-Explorer/actions"><img src="https://img.shields.io/github/actions/workflow/status/opendatalab/MinerU-Document-Explorer/ci.yml?style=flat-square&label=CI" alt="CI"/></a>
  11. <a href="https://github.com/opendatalab/MinerU-Document-Explorer"><img src="https://img.shields.io/github/stars/opendatalab/MinerU-Document-Explorer?style=flat-square" alt="stars"/></a>
  12. </p>
  13. <p>
  14. <a href="README-zh.md">中文文档</a> ·
  15. <a href="docs/mcp.md">MCP Setup</a> ·
  16. <a href="docs/cli.md">CLI Reference</a> ·
  17. <a href="demo/">Demo</a> ·
  18. <a href="CONTRIBUTING.md">Contributing</a>
  19. </p>
  20. </div>
  21. ---
  22. ## 🤔 Why MinerU Document Explorer?
  23. MinerU Document Explorer equips your agent with three tool suites — **Retrieve, Deep Read, and Ingest** — closing the full knowledge loop:
  24. ![Overview of MinerU Document Explorer](assets/overview_en.png)
  25. - **🔍 Retrieve** — Cross-collection search: BM25, vector, and hybrid with LLM reranking and query expansion
  26. - **📖 Deep Read** — Navigate inside a single document without loading the whole file: table of contents, section reading, inline search, and element extraction
  27. - **📝 Ingest** — Build and maintain a LLM wiki from raw documents, following the [Karpathy LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern
  28. Developed by the [MinerU](https://github.com/opendatalab/MinerU) team, building on [QMD](https://github.com/tobi/qmd) and [Karpathy's LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f).
  29. ### 💡 What can you do with it?
  30. - **Research assistant** — Point it at a folder of PDFs and ask your AI agent to survey the literature, compare methods, or find specific results across papers
  31. - **Project knowledge base** — Index your codebase, docs, and design specs; let the agent answer questions about architecture, find relevant code, or trace requirements
  32. - **Study companion** — Upload textbooks or lecture notes; use `doc_toc` + `doc_read` to navigate, `doc_grep` to search, and build a wiki of key concepts
  33. - **Enterprise document search** — Index contracts, reports, or manuals and search them with natural language queries enhanced by LLM reranking
  34. > **See it in action**: The [demo/](demo/) folder contains a complete end-to-end example — an AI agent automatically reads ~10 arXiv papers on RAG, builds an interlinked wiki knowledge base, and writes a research survey. See the [Demo Guide](demo/README.md) for step-by-step instructions.
  35. ## 🚀 Quick Start
  36. > Agent-Assisted Setup: If you're using an AI agent (Claude Code, Cursor, etc.), simply ask it to help you deploy MinerU Document Explorer and install skills — the agent can handle the entire setup process for you, including MCP configuration.
  37. ```
  38. Follow the https://github.com/opendatalab/MinerU-Document-Explorer/blob/main/docs/quickstart.md to install MinerU Document Explorer and walk the user through configuration.
  39. ```
  40. ## 📖 Document Deep Reading
  41. Navigate and search within a single document without reading the whole file:
  42. ```sh
  43. # View document structure
  44. qmd doc-toc papers/attention-is-all-you-need.pdf
  45. # Read specific sections by address
  46. qmd doc-read papers/attention-is-all-you-need.pdf "line:45-120"
  47. # Search within one document
  48. qmd doc-grep papers/attention-is-all-you-need.pdf "self-attention"
  49. ```
  50. ## 🔌 MCP Server — 15 Tools for AI Agents
  51. Integrate with AI agents via [Model Context Protocol](https://modelcontextprotocol.io).
  52. > **MCP Server vs CLI**: The MCP server runs as a **persistent process** — LLM models (embeddings, reranker, query expansion) are loaded once and stay in memory across requests. CLI commands like `qmd query` must reload all models on every invocation, adding ~5–15 s of startup overhead each time. **For agent workflows, always prefer the MCP server.**
  53. Two transport modes:
  54. | Mode | Command | Best for |
  55. |:-----|:--------|:---------|
  56. | **stdio** | `qmd mcp` | Claude Desktop, Claude Code — client spawns and manages the process |
  57. | **HTTP daemon** | `qmd mcp --http --daemon` | Cursor, Windsurf, VS Code, multi-client setups — one shared persistent server |
  58. ```sh
  59. # Start the HTTP daemon (recommended — models stay loaded across all requests)
  60. qmd mcp --http --daemon # default port 8181
  61. qmd mcp --http --daemon --port 8080 # custom port
  62. # Verify server is running
  63. curl http://localhost:8181/health
  64. # Stop the daemon
  65. qmd mcp stop
  66. ```
  67. ### Client Configuration
  68. <details>
  69. <summary><b>Cursor</b> — add to <code>.cursor/mcp.json</code> (project) or <code>~/.cursor/mcp.json</code> (global)</summary>
  70. **Option A — stdio** (Cursor manages the process lifecycle):
  71. ```json
  72. {
  73. "mcpServers": {
  74. "qmd": {
  75. "command": "qmd",
  76. "args": ["mcp"]
  77. }
  78. }
  79. }
  80. ```
  81. **Option B — HTTP** (run `qmd mcp --http --daemon` first; models stay loaded, faster responses):
  82. ```json
  83. {
  84. "mcpServers": {
  85. "qmd": {
  86. "url": "http://localhost:8181/mcp"
  87. }
  88. }
  89. }
  90. ```
  91. </details>
  92. <details>
  93. <summary><b>Claude Desktop</b> — add to <code>~/Library/Application Support/Claude/claude_desktop_config.json</code></summary>
  94. ```json
  95. {
  96. "mcpServers": {
  97. "qmd": {
  98. "command": "qmd",
  99. "args": ["mcp"]
  100. }
  101. }
  102. }
  103. ```
  104. </details>
  105. <details>
  106. <summary><b>Claude Code</b> — add to <code>~/.claude/settings.json</code> or run <code>claude mcp add qmd -- qmd mcp</code></summary>
  107. ```json
  108. {
  109. "mcpServers": {
  110. "qmd": {
  111. "command": "qmd",
  112. "args": ["mcp"]
  113. }
  114. }
  115. }
  116. ```
  117. </details>
  118. <details>
  119. <summary><b>Windsurf / VS Code / Other MCP Clients</b></summary>
  120. For **stdio** transport, use `"command": "qmd"`, `"args": ["mcp"]` in your client's MCP configuration.
  121. For **HTTP** transport, start `qmd mcp --http --daemon` and point your client to `http://localhost:8181/mcp`.
  122. </details>
  123. See [MCP setup guide](docs/mcp.md) for all 15 tools and HTTP transport details.
  124. ### Agent Skills
  125. MinerU Document Explorer ships with a built-in [Agent Skill](skills/mineru-document-explorer/SKILL.md) that teaches AI agents how to use the full tool suite effectively — decision trees, usage patterns, and best practices for all 15 MCP tools.
  126. ```sh
  127. # Install the skill (works with both npm and source installs)
  128. qmd skill install # local project (.agents/skills/)
  129. qmd skill install --global # global (~/.agents/skills/)
  130. # Or from source repo
  131. claude skill add ./skills/mineru-document-explorer/SKILL.md
  132. ```
  133. ## 📊 How It Compares
  134. | | MinerU Doc Explorer | LlamaIndex | Obsidian | NotebookLM |
  135. |:---|:---:|:---:|:---:|:---:|
  136. | **Runs 100% locally** | ✅ | ⚠️ LLM APIs | ✅ | ❌ Cloud |
  137. | **Agent integration (MCP)** | **15 tools** | Plugin | ❌ | ❌ |
  138. | **Deep reading within docs** | ✅ | ❌ | ❌ | ✅ |
  139. | **Wiki knowledge compilation** | ✅ | ❌ | Manual | ❌ |
  140. | **Formats** | MD, PDF, DOCX, PPTX | Many | MD | PDF, URL |
  141. | **Search pipeline** | BM25 + vec + rerank | Configurable | Basic | Proprietary |
  142. | **Zero-config search** | ✅ `qmd search` | ❌ | Plugin | N/A |
  143. | **Open source** | MIT | MIT | Partial | ❌ |
  144. ## ⚙️ Requirements
  145. | Requirement | Notes |
  146. |-------------|-------|
  147. | **Node.js** >= 22 or **Bun** | Runtime |
  148. | **Python** >= 3.10 | Document processing (`pymupdf`, `python-docx`, `python-pptx`) |
  149. | **macOS** | `brew install sqlite` for extension support |
  150. ### 📄 Document Processing Setup
  151. Python 3.10+ is required for document processing (PDF, DOCX, PPTX):
  152. ```sh
  153. # Check Python version
  154. python3 --version # needs >= 3.10
  155. # Install required Python packages
  156. pip install pymupdf python-docx python-pptx
  157. # Verify
  158. python3 -c "import pymupdf; import docx; import pptx; print('OK')"
  159. ```
  160. <details>
  161. <summary><b>MinerU Cloud</b> — high-quality PDF extraction for scanned documents and complex layouts (optional)</summary>
  162. ```sh
  163. pip install mineru-open-sdk
  164. export MINERU_API_KEY="your-key" # get from https://mineru.net
  165. ```
  166. When `MINERU_API_KEY` is set, MinerU Cloud is automatically used as the primary PDF provider with PyMuPDF as fallback.
  167. For advanced configuration (custom providers, local VLM models, GPT PageIndex), create `~/.config/qmd/doc-reading.json`:
  168. ```json
  169. {
  170. "docReading": {
  171. "providers": {
  172. "fullText": { "pdf": ["mineru_cloud", "pymupdf"] }
  173. },
  174. "credentials": {
  175. "mineru": { "api_key": "your-api-key" }
  176. }
  177. }
  178. }
  179. ```
  180. </details>
  181. ### 🤖 LLM Models (auto-downloaded on first use)
  182. | Model | Purpose | Size |
  183. |:------|:--------|:-----|
  184. | embeddinggemma-300M | Vector embeddings | ~300 MB |
  185. | qwen3-reranker-0.6b | Re-ranking | ~640 MB |
  186. | qmd-query-expansion-1.7B | Query expansion | ~1.1 GB |
  187. > Models are only needed for `qmd embed`, `qmd vsearch`, and `qmd query`. `qmd search` runs BM25 retrieval.
  188. ## 📚 Documentation
  189. | | |
  190. |:---|:---|
  191. | 🎯 [Demo Guide](demo/README.md) | End-to-end example: agent-driven RAG research survey |
  192. | 📖 [CLI Reference](docs/cli.md) | All commands, options, output formats |
  193. | 🔌 [MCP Server](docs/mcp.md) | Setup, 15 tools, HTTP transport |
  194. | 📦 [SDK / Library](docs/sdk.md) | TypeScript API, types, examples |
  195. | 🏗️ [Architecture](docs/architecture.md) | Search pipeline, scoring, data schema, chunking |
  196. | 🤝 [Contributing](CONTRIBUTING.md) | Development setup, code style, how to contribute |
  197. ## ❤️ Acknowledgments
  198. MinerU Document Explorer builds upon these foundational projects:
  199. - **[QMD](https://github.com/tobi/qmd)** by [Tobi Lutke](https://github.com/tobi) — An on-device search engine and CLI toolkit for markdown documents
  200. - **[LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)** by [Andrej Karpathy](https://github.com/karpathy) — the conceptual pattern for LLM-maintained knowledge bases
  201. - **[MinerU](https://github.com/opendatalab/MinerU)** by [OpenDataLab](https://github.com/opendatalab) — high-quality document parsing and extraction
  202. ---
  203. ## 📝 Changelog
  204. ### v1 — 2026-04-07 (Current)
  205. Rebuilt from an OpenClaw agent skill into a full agent-native knowledge engine: npm package (`npm install -g mineru-document-explorer`), `qmd` CLI, MCP server with 15 tools across three groups (Retrieval / Deep Reading / Knowledge Ingestion), multi-format support (MD, PDF, DOCX, PPTX), hybrid search (BM25 + vector + LLM reranking), and LLM Wiki knowledge base pattern.
  206. ### v0 — 2026-03-30 (Previous)
  207. OpenClaw-native agent skill (`doc-search` CLI). Four capabilities: Logic Retrieval, Semantic Retrieval, Keyword Retrieval, Evidence Extraction. See the [v0 repository](https://github.com/opendatalab/MinerU-Document-Explorer/tree/v0).

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.