Skip to content
Back to search

pdfmux

PDF extraction that audits its own output — and certifies any other extractor's, catching pages they silently dropped. Verify signed manifests offline: free, MIT, no account. 0.903 on opendataloader-bench, #2 of 8 engines. 7-tool MCP server.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/nameetp--pdfmux/badge.svg)](/listings/nameetp--pdfmux)

HTML

<a href="/listings/nameetp--pdfmux"><img src="/api/listings/nameetp--pdfmux/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

pip install pdfmux

That handles digital PDFs. **For any real-world batch, install `pdfmux[ocr]` too** — almost every directory of PDFs has at least one scan, and without OCR those pages return empty text:

pip install "pdfmux[ocr]"             # ⭐ recommended — RapidOCR for scanned pages (~200MB, CPU)

Other backends, by document type:

pip install "pdfmux[tables]"          # Docling — table-heavy docs (~500MB)
pip install "pdfmux[opendataloader]"  # OpenDataLoader — complex layouts (Java 11+)
pip install "pdfmux[marker]"          # Marker — neural extraction for academic papers
pip install "pdfmux[llm]"             # Gemini fallback (default LLM)
pip install "pdfmux[llm-claude]"      # Claude (Sonnet / Opus)
pip install "pdfmux[llm-openai]"      # GPT-4o family
pip install "pdfmux[llm-ollama]"      # Ollama (any local model)
pip install "pdfmux[llm-mistral]"     # Mistral OCR API ($0.002/page)
pip install "pdfmux[llm-all]"         # all LLM providers (incl. Gemma via Gemini key)
pip install "pdfmux[watch]"           # `pdfmux watch <dir>` auto-convert on change
pip install "pdfmux[all]"             # everything

Requires Python 3.11+.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 738 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # pdfmux
  2. [![CI](https://github.com/NameetP/pdfmux/actions/workflows/ci.yml/badge.svg)](https://github.com/NameetP/pdfmux/actions/workflows/ci.yml)
  3. [![PyPI](https://img.shields.io/pypi/v/pdfmux)](https://pypi.org/project/pdfmux/)
  4. [![Python 3.11+](https://img.shields.io/pypi/pyversions/pdfmux)](https://pypi.org/project/pdfmux/)
  5. [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT)
  6. [![Downloads](https://img.shields.io/pypi/dm/pdfmux)](https://pypi.org/project/pdfmux/)
  7. **Self-healing PDF extraction that flags the pages it can't read instead of dropping them — and now certifies any extractor's output for silent drops.** Open-source LlamaParse alternative for RAG pipelines, MCP server for Claude Desktop, LangChain + LlamaIndex loaders.
  8. > pdfmux extracts PDFs and checks its own work — and now certifies any extractor's, telling you which pages it silently dropped. Free, MIT. Patent-pending method. `pip install pdfmux`.
  9. **Two jobs, one tool:**
  10. - **Self-healing extraction.** The only PDF extractor that audits its own output. Catches blank pages, scrambled columns, broken tables — re-extracts them with a stronger backend, and flags what it still can't read instead of silently dropping it. So your LLM gets clean data, not silent garbage. Routes each page to the best of 7 built-in extraction backends + BYOK LLM fallback (Gemini / Claude / GPT-4o / Ollama). One CLI. One API. Zero config.
  11. - **[Certify Anything](#certify-anything) — new in v1.8.1.** `pdfmux verify` audits *any* extraction engine's output against the source PDF — Reducto, Mistral OCR, LlamaParse, Docling, your in-house parser — and tells you which pages it silently dropped. Free, MIT, patent-clean.
  12. <p align="center">
  13. <img src="demo.svg" alt="pdfmux terminal demo" width="700" />
  14. </p>
  15. ```
  16. PDF ──> pdfmux router ──> best extractor per page ──> audit ──> re-extract failures ──> Markdown / JSON / chunks
  17. |
  18. ├─ PyMuPDF (digital text, 0.01s/page)
  19. ├─ OpenDataLoader (complex layouts, 0.05s/page)
  20. ├─ RapidOCR (scanned pages, CPU-only)
  21. ├─ Docling (tables, 97.9% TEDS)
  22. ├─ Surya (heavy OCR fallback)
  23. ├─ Marker (academic papers, neural)
  24. ├─ Mistral OCR ($0.002/page, 96.6% tables)
  25. └─ YOUR LLM (Gemini / Gemma 3 / Claude / GPT-4o / Ollama / Mistral — BYOK via YAML)
  26. ```
  27. ## Install
  28. ```bash
  29. pip install pdfmux
  30. ```
  31. That handles digital PDFs. **For any real-world batch, install `pdfmux[ocr]` too** — almost every directory of PDFs has at least one scan, and without OCR those pages return empty text:
  32. ```bash
  33. pip install "pdfmux[ocr]" # ⭐ recommended — RapidOCR for scanned pages (~200MB, CPU)
  34. ```
  35. Other backends, by document type:
  36. ```bash
  37. pip install "pdfmux[tables]" # Docling — table-heavy docs (~500MB)
  38. pip install "pdfmux[opendataloader]" # OpenDataLoader — complex layouts (Java 11+)
  39. pip install "pdfmux[marker]" # Marker — neural extraction for academic papers
  40. pip install "pdfmux[llm]" # Gemini fallback (default LLM)
  41. pip install "pdfmux[llm-claude]" # Claude (Sonnet / Opus)
  42. pip install "pdfmux[llm-openai]" # GPT-4o family
  43. pip install "pdfmux[llm-ollama]" # Ollama (any local model)
  44. pip install "pdfmux[llm-mistral]" # Mistral OCR API ($0.002/page)
  45. pip install "pdfmux[llm-all]" # all LLM providers (incl. Gemma via Gemini key)
  46. pip install "pdfmux[watch]" # `pdfmux watch <dir>` auto-convert on change
  47. pip install "pdfmux[all]" # everything
  48. ```
  49. Requires Python 3.11+.
  50. ## Quick Start
  51. ### CLI
  52. ```bash
  53. # zero config — just works
  54. pdfmux convert invoice.pdf
  55. # invoice.pdf -> invoice.md (2 pages, 95% confidence, via pymupdf4llm)
  56. # RAG-ready chunks with token limits
  57. pdfmux convert report.pdf --chunk --max-tokens 500
  58. # cost-aware extraction with budget cap
  59. pdfmux convert report.pdf --mode economy --budget 0.50
  60. # schema-guided structured extraction (5 built-in presets)
  61. pdfmux convert invoice.pdf --schema invoice
  62. # BYOK any LLM for hardest pages
  63. pdfmux convert scan.pdf --llm-provider claude
  64. # use a built-in or saved profile (invoices, receipts, papers, contracts, bulk-rag)
  65. pdfmux convert invoice.pdf --profile invoices
  66. # predict cost before running anything
  67. pdfmux estimate big-report.pdf --llm-provider gemini
  68. # stream pages as NDJSON as they finish (great for long documents)
  69. pdfmux stream report.pdf --quality high
  70. # auto-convert any new PDFs that land in a folder
  71. pdfmux watch ./inbox/ -o ./output/
  72. # diff two extractions side-by-side
  73. pdfmux diff old.pdf new.pdf
  74. # batch a directory — writes manifest.json with per-doc confidence
  75. pdfmux convert ./docs/ -o ./output/
  76. # CI mode: fail the run if any document is below 0.20 confidence
  77. pdfmux convert ./docs/ -o ./output/ --strict --min-confidence 0.20
  78. # pre-flight a directory: which extras do you actually need for THIS batch?
  79. pdfmux doctor --check ./docs/
  80. # results are cached by file hash — re-runs are instant; bypass with --no-cache
  81. pdfmux convert report.pdf --no-cache
  82. pdfmux convert report.pdf --clear-cache
  83. ```
  84. ### Python
  85. For batch processing, use `batch_extract()` — not a `subprocess.run(['pdfmux', ...])` loop. Same pipeline, no per-file process spawn, handles non-ASCII filenames:
  86. ```python
  87. import pdfmux
  88. from pathlib import Path
  89. # Batch extract — yields (path, result) tuples as each PDF completes.
  90. pdfs = list(Path("./inbox").glob("*.pdf"))
  91. for path, result in pdfmux.batch_extract(pdfs, quality="standard"):
  92. if isinstance(result, Exception):
  93. print(f"FAILED {path.name}: {result}")
  94. continue
  95. if result.confidence < 0.50:
  96. print(f"REVIEW {path.name} ({result.confidence:.2f})")
  97. else:
  98. print(f"OK {path.name} ({result.confidence:.2f})")
  99. # Single-file helpers.
  100. text = pdfmux.extract_text("report.pdf") # markdown string
  101. data = pdfmux.extract_json("report.pdf") # locked schema dict
  102. chunks = pdfmux.chunk("report.pdf", max_tokens=500) # RAG-ready chunks
  103. ```
  104. > **Don't wrap pdfmux with your own pypdf/pdfplumber fallback.** pdfmux already routes per page through PyMuPDF → RapidOCR → vision LLM. PyMuPDF tolerates malformed PDFs that pypdf rejects ("Stream has ended unexpectedly"), so a downstream pypdf fallback turns recoverable PDFs into failures. Trust the router; check the confidence score on the result.
  105. ## Certify Anything
  106. `pdfmux verify` audits **any extraction engine's output** against the source PDF and tells you which pages it silently dropped — not just pdfmux's own extraction. Point it at the output of Reducto, Mistral OCR, LlamaParse, Docling, or your in-house parser and it re-derives the source text with pdfmux's own audit pass, aligns the extraction to it, and scores every page.
  107. **The failure it catches:** a page where the source has real text but the engine returned nothing — while reporting success. That "silent drop" is the exact failure that poisons a RAG index without a single error in the logs.
  108. ```bash
  109. # Certify pdfmux's own extraction of a document
  110. pdfmux verify --source report.pdf --engine pdfmux
  111. # Certify ANOTHER engine's output (JSON / Markdown / text)
  112. pdfmux verify --source report.pdf --extracted reducto.json --engine-name reducto
  113. # Batch a whole directory — the "M pages silently dropped across N docs" report
  114. pdfmux verify --source ./pdfs/ --extracted ./engine-outputs/ -o certification.json
  115. # CI gate: exit non-zero unless the overall verdict is PASS
  116. pdfmux verify --source report.pdf --extracted out.json --strict
  117. ```
  118. Every run prints a `PASS` / `REVIEW` / `FAIL` verdict, overall confidence and coverage, and — when it finds them — the silently dropped pages by number:
  119. ```
  120. pdfmux verify — report.pdf · engine: reducto
  121. FAIL confidence 71% · coverage 68%
  122. reducto: FAIL; 3 page(s) SILENTLY DROPPED (pages 7, 12, 31); overall
  123. confidence 71%, coverage 68% across 40 page(s).
  124. ❌ 3 page(s) SILENTLY DROPPED: 7, 12, 31
  125. ```
  126. Per page you get a verdict (`pass` / `review` / `fail`), confidence, coverage, alignment, hallucination-risk, and table/heading integrity. Batch mode rolls that up into a single **"N pages silently dropped across M documents"** line — the report you run on 100 of your own PDFs to find the silent failures already in your pipeline.
  127. ### It works on any engine's output
  128. `--extracted` accepts JSON, Markdown, or plain text (`--extracted-format auto | json | markdown | text`). When the extraction exposes real per-page structure, pdfmux compares page-by-page; when it's a single blob, it falls back to content-presence checks so it never fabricates a "silent drop" from a pagination mismatch.
  129. ### Python API
  130. ```python
  131. from pdfmux import verify_extraction, verify_batch
  132. # Single document → a CertificationManifest
  133. manifest = verify_extraction("report.pdf", "reducto.json", engine="reducto")
  134. print(manifest.verdict) # "PASS" | "REVIEW" | "FAIL"
  135. print(manifest.silent_drops) # e.g. (7, 12, 31) — 1-indexed page numbers
  136. print(manifest.coverage) # 0.0–1.0
  137. # Many documents → a BatchCertification ("M pages dropped across N docs")
  138. batch = verify_batch([("a.pdf", "a.json"), ("b.pdf", "b.json")], engine="llamaparse")
  139. print(batch.total_silent_drops, "pages dropped across", batch.doc_count, "docs")
  140. ```
  141. Each manifest carries a tamper-evident SHA-256 content signature over its canonical body and an embedded, honest limitations list: the certifier is **lexical, not linguistic** — it detects missing and garbled content, not faithful paraphrase or translation.
  142. ### MCP
  143. `verify_extraction` is exposed as an MCP tool (the 7th — see [MCP Server](#mcp-server-ai-agents)), so an agent can certify an engine's output in the same session it extracts.
  144. ### Free, MIT, patent-clean
  145. Certify Anything reuses only pdfmux's shipped MIT audit layer. It does **not** include, and does not require, the patent-pending decision-trace method — that stays in [pdfmux Cloud/Pro](#license). `pip install pdfmux` gives you the full `verify` command at no cost.
  146. Full reference: **[docs/CERTIFY-ANYTHING.md](docs/CERTIFY-ANYTHING.md)**.
  147. ### When you need to prove it to someone else
  148. A local install can audit an extraction, but it cannot *attest* to one — anything it signs, anyone could forge. [pdfmux Cloud](https://app.pdfmux.com/pricing) returns an **Ed25519-signed manifest** over the extraction: your auditor verifies it **offline**, against a published public key, without an account and without trusting pdfmux.
  149. ```bash
  150. pdfmux verify-manifest manifest.json # free, MIT, offline — no account
  151. ```
  152. Verification is free and open forever; only *generation* is paid ($49/mo). That asymmetry is deliberate — you should never need our permission to check our work.
  153. Free tool, no signup: **[app.pdfmux.com/audit](https://app.pdfmux.com/audit)** — upload a PDF and see which pages your current extractor silently dropped. Measured accuracy (and its blind spots) published in [pdfmux-bench](https://github.com/NameetP/pdfmux/blob/feat/pdfmux-bench/pdfmux-bench/VERIFIER-VALIDATION.md).
  154. ## Architecture
  155. ```
  156. ┌─────────────────────────────┐
  157. │ Segment Detector │
  158. │ text / tables / images / │
  159. │ formulas / headers per page │
  160. └─────────────┬───────────────┘
  161. │
  162. ┌────────────────────────────────────────┐
  163. │ Router Engine │
  164. │ │
  165. │ economy ── balanced ── premium │
  166. │ (minimize $) (default) (max quality)│
  167. │ budget caps: --budget 0.50 │
  168. └────────────────────┬───────────────────┘
  169. │
  170. ┌──────────┬──────────┬────────┴────────┬──────────┐
  171. │ │ │ │ │
  172. PyMuPDF OpenData RapidOCR Docling LLM
  173. digital Loader scanned tables (BYOK)
  174. 0.01s/pg complex CPU-only 97.9% any provider
  175. layouts TEDS
  176. │ │ │ │ │
  177. └──────────┴──────────┴────────┬────────┴──────────┘
  178. │
  179. ┌────────────────────────────────────────┐
  180. │ Quality Auditor │
  181. │ │
  182. │ 4-signal dynamic confidence scoring │
  183. │ per-page: good / bad / empty │
  184. │ if bad -> re-extract with next backend│
  185. └────────────────────┬───────────────────┘
  186. │
  187. ┌────────────────────────────────────────┐
  188. │ Output Pipeline │
  189. │ │
  190. │ heading injection (font-size analysis)│
  191. │ table extraction + normalization │
  192. │ text cleanup + merge │
  193. │ confidence score (honest, not inflated)│
  194. └────────────────────────────────────────┘
  195. ```
  196. ### Key design decisions
  197. - **Router, not extractor.** pdfmux does not compete with PyMuPDF or Docling. It picks the best one per page.
  198. - **Agentic multi-pass.** Extract, audit confidence, re-extract failures with a stronger backend. Bad pages get retried automatically.
  199. - **Segment-level detection.** Each page is classified by content type (text, tables, images, formulas, headers) before routing.
  200. - **4-signal confidence.** Dynamic quality scoring from character density, OCR noise ratio, table integrity, and heading structure. Not hardcoded thresholds.
  201. - **Document cache.** Each PDF is opened once, not once per extractor. Shared across the full pipeline.
  202. - **Data flywheel.** Local telemetry tracks which extractors win per document type. Routing improves with usage.
  203. ## Features
  204. | Feature | What it does | Command |
  205. |---------|-------------|---------|
  206. | Zero-config extraction | Routes to best backend automatically | `pdfmux convert file.pdf` |
  207. | RAG chunking | Section-aware chunks with token estimates | `pdfmux convert file.pdf --chunk --max-tokens 500` |
  208. | Cost modes | economy / balanced / premium with budget caps | `pdfmux convert file.pdf --mode economy --budget 0.50` |
  209. | Schema extraction | 5 built-in presets (invoice, receipt, contract, resume, paper) | `pdfmux convert file.pdf --schema invoice` |
  210. | Profiles | Save and re-use config; built-ins for invoices/receipts/papers/contracts/bulk-rag | `pdfmux convert file.pdf --profile invoices` |
  211. | BYOK LLM | Gemini, Gemma 3, Claude, GPT-4o, Ollama, Mistral, any OpenAI-compatible API | `pdfmux convert file.pdf --llm-provider claude` |
  212. | Cost estimate | Predict spend before running | `pdfmux estimate file.pdf --llm-provider gemini` |
  213. | Streaming output | NDJSON events page-by-page for long docs | `pdfmux stream file.pdf` |
  214. | Smart cache | Hash-keyed result cache, 30-day TTL, 1 GB LRU | `pdfmux convert file.pdf` (auto), `--no-cache` to bypass |
  215. | Watch mode | Auto-convert any PDF added to a folder | `pdfmux watch ./inbox/` |
  216. | Diff | Compare two extractions | `pdfmux diff a.pdf b.pdf` |
  217. | Benchmark | Eval all installed extractors against ground truth | `pdfmux benchmark` |
  218. | Doctor | Show installed backends, coverage gaps, recommendations | `pdfmux doctor` |
  219. | MCP server | AI agents read PDFs via stdio or HTTP | `pdfmux serve` |
  220. | Batch processing | Convert entire directories | `pdfmux convert ./docs/` |
  221. | Page-level streaming API | Bounded-memory page iteration for large files | `for page in ext.extract("500pg.pdf")` |
  222. | Retry with backoff | Every LLM provider auto-retries with exponential backoff + `Retry-After` | (built-in) |
  223. ## CLI Reference
  224. ### `pdfmux convert`
  225. ```bash
  226. pdfmux convert <file-or-dir> [options]
  227. Options:
  228. -o, --output PATH Output file or directory
  229. -f, --format FORMAT markdown | json | csv | llm (default: markdown)
  230. -q, --quality QUALITY fast | standard | high (default: standard)
  231. -s, --schema SCHEMA JSON schema file or preset (invoice, receipt, contract, resume, paper)
  232. --chunk Output RAG-ready chunks
  233. --max-tokens N Max tokens per chunk (default: 500)
  234. --mode MODE economy | balanced | premium (default: balanced)
  235. --budget AMOUNT Max spend per document in USD
  236. --llm-provider PROVIDER LLM backend: gemini | claude | openai | ollama
  237. --confidence Include confidence score in output
  238. --stdout Print to stdout instead of file
  239. ```
  240. ### `pdfmux serve`
  241. Start the MCP server for AI agent integration.
  242. ```bash
  243. pdfmux serve # stdio mode (Claude Desktop, Cursor)
  244. pdfmux serve --http 8080 # HTTP mode
  245. ```
  246. ### `pdfmux doctor`
  247. ```bash
  248. pdfmux doctor
  249. # ┌──────────────────┬─────────────┬─────────┬──────────────────────────────────┐
  250. # │ Extractor │ Status │ Version │ Install │
  251. # ├──────────────────┼─────────────┼─────────┼──────────────────────────────────┤
  252. # │ PyMuPDF │ installed │ 1.25.3 │ │
  253. # │ OpenDataLoader │ installed │ 0.3.1 │ │
  254. # │ RapidOCR │ installed │ 3.0.6 │ │
  255. # │ Docling │ missing │ -- │ pip install pdfmux[tables] │
  256. # │ Surya │ missing │ -- │ pip install pdfmux[ocr-heavy] │
  257. # │ LLM (Gemini) │ configured │ -- │ GEMINI_API_KEY set │
  258. # └──────────────────┴─────────────┴─────────┴──────────────────────────────────┘
  259. ```
  260. ### `pdfmux benchmark`
  261. ```bash
  262. pdfmux benchmark report.pdf
  263. # ┌──────────────────┬────────┬────────────┬─────────────┬──────────────────────┐
  264. # │ Extractor │ Time │ Confidence │ Output │ Status │
  265. # ├──────────────────┼────────┼────────────┼─────────────┼──────────────────────┤
  266. # │ PyMuPDF │ 0.02s │ 95% │ 3,241 chars │ all pages good │
  267. # │ Multi-pass │ 0.03s │ 95% │ 3,241 chars │ all pages good │
  268. # │ RapidOCR │ 4.20s │ 88% │ 2,891 chars │ ok │
  269. # │ OpenDataLoader │ 0.12s │ 97% │ 3,310 chars │ best │
  270. # └──────────────────┴────────┴────────────┴─────────────┴──────────────────────┘
  271. ```
  272. ### `pdfmux estimate`
  273. Predict spend (and which backends will run) before processing.
  274. ```bash
  275. pdfmux estimate report.pdf --quality high --llm-provider gemini
  276. # Pages : 47
  277. # Extractors : pymupdf4llm + gemini-2.5-flash on 9 pages
  278. # Estimated : $0.0234
  279. # Cache hit? : no (first run for this file)
  280. ```
  281. ### `pdfmux stream`
  282. Emit NDJSON events as pages complete — useful for very long PDFs and live UIs.
  283. ```bash
  284. pdfmux stream long.pdf --quality high
  285. # {"event":"classified","page_count":312,"plan":"pymupdf+gemini-fallback"}
  286. # {"event":"page","page_num":0,"confidence":0.97,"chars":1842}
  287. # {"event":"page","page_num":1,"confidence":0.92,"chars":1611,"ocr":true}
  288. # ...
  289. # {"event":"complete","confidence":0.94,"cost_usd":0.0712}
  290. ```
  291. ### `pdfmux watch`
  292. Auto-convert any PDFs that land in a directory. Survives until Ctrl+C.
  293. ```bash
  294. pdfmux watch ./inbox/ -o ./output/ --profile bulk-rag
  295. ```
  296. ### `pdfmux diff`
  297. Side-by-side extraction comparison (quality, content, cost).
  298. ```bash
  299. pdfmux diff a.pdf b.pdf --quality standard
  300. ```
  301. ### `pdfmux profiles`
  302. Saved configs at `~/.config/pdfmux/profiles.yaml`. Built-ins ship for the
  303. common shapes; save your own for project defaults.
  304. ```bash
  305. pdfmux profiles list
  306. # invoices quality=standard, schema=invoice, format=json
  307. # receipts quality=fast, schema=receipt, format=json
  308. # papers quality=high, chunk=true, max_tokens=500
  309. # contracts quality=high, schema=contract
  310. # bulk-rag quality=standard, format=llm, chunk=true
  311. pdfmux profiles show invoices
  312. pdfmux profiles save my-default --quality high --format llm --chunk
  313. pdfmux profiles delete my-default
  314. # use a profile when converting
  315. pdfmux convert file.pdf --profile invoices
  316. ```
  317. ## Python API
  318. ### Text extraction
  319. ```python
  320. import pdfmux
  321. text = pdfmux.extract_text("report.pdf") # -> str (markdown)
  322. text = pdfmux.extract_text("report.pdf", quality="fast") # PyMuPDF only, instant
  323. text = pdfmux.extract_text("report.pdf", quality="high") # LLM-assisted
  324. ```
  325. ### Structured extraction
  326. ```python
  327. data = pdfmux.extract_json("report.pdf")
  328. # data["page_count"] -> 12
  329. # data["confidence"] -> 0.91
  330. # data["ocr_pages"] -> [2, 5, 8]
  331. # data["pages"][0]["key_values"] -> [{"key": "Date", "value": "2026-02-28"}]
  332. # data["pages"][0]["tables"] -> [{"headers": [...], "rows": [...]}]
  333. ```
  334. ### RAG chunking
  335. ```python
  336. chunks = pdfmux.chunk("report.pdf", max_tokens=500)
  337. for c in chunks:
  338. print(f"{c['title']}: {c['tokens']} tokens (pages {c['page_start']}-{c['page_end']})")
  339. ```
  340. ### Schema-guided extraction
  341. ```python
  342. data = pdfmux.extract_json("invoice.pdf", schema="invoice")
  343. # Uses built-in invoice preset: extracts date, vendor, line items, totals
  344. # Also accepts a path to a custom JSON Schema file
  345. ```
  346. ### Streaming (bounded memory)
  347. ```python
  348. from pdfmux.extractors import get_extractor
  349. ext = get_extractor("fast")
  350. for page in ext.extract("large-500-pages.pdf"): # Iterator[PageResult]
  351. process(page.text) # constant memory, even on 500-page PDFs
  352. ```
  353. ### Types and errors
  354. ```python
  355. from pdfmux import (
  356. # Enums
  357. Quality, # FAST, STANDARD, HIGH
  358. OutputFormat, # MARKDOWN, JSON, CSV, LLM
  359. PageQuality, # GOOD, BAD, EMPTY
  360. # Data objects (frozen dataclasses)
  361. PageResult, # page: text, page_num, confidence, quality, extractor
  362. DocumentResult, # document: pages, source, confidence, extractor_used
  363. Chunk, # chunk: title, text, page_start, page_end, tokens
  364. # Errors
  365. PdfmuxError, # base -- catch this for all pdfmux errors
  366. FileError, # file not found, unreadable, not a PDF
  367. ExtractionError, # extraction failed
  368. ExtractorNotAvailable,# requested backend not installed
  369. FormatError, # invalid output format
  370. AuditError, # audit could not complete
  371. )
  372. ```
  373. ## Framework Integrations
  374. ### LangChain
  375. ```bash
  376. pip install langchain-pdfmux
  377. ```
  378. ```python
  379. from langchain_pdfmux import PDFMuxLoader
  380. loader = PDFMuxLoader("report.pdf", quality="standard")
  381. docs = loader.load() # -> list[Document] with confidence metadata
  382. ```
  383. ### LlamaIndex
  384. ```bash
  385. pip install llama-index-readers-pdfmux
  386. ```
  387. ```python
  388. from llama_index.readers.pdfmux import PDFMuxReader
  389. reader = PDFMuxReader(quality="standard")
  390. docs = reader.load_data("report.pdf") # -> list[Document]
  391. ```
  392. ### MCP Server (AI Agents)
  393. Listed on [mcpservers.org](https://mcpservers.org). One-line setup:
  394. ```json
  395. {
  396. "mcpServers": {
  397. "pdfmux": {
  398. "command": "npx",
  399. "args": ["-y", "pdfmux-mcp"]
  400. }
  401. }
  402. }
  403. ```
  404. Or via Claude Code:
  405. ```bash
  406. claude mcp add pdfmux -- npx -y pdfmux-mcp
  407. ```
  408. Tools exposed: `convert_pdf`, `analyze_pdf`, `extract_structured`,
  409. `extract_streaming`, `get_pdf_metadata`, `batch_convert`.
  410. ## BYOK LLM Configuration
  411. pdfmux supports any LLM via 5 lines of YAML. Bring your own keys -- nothing leaves your machine unless you configure it to.
  412. ```yaml
  413. # ~/.pdfmux/llm.yaml
  414. provider: claude # gemini | claude | openai | ollama | any OpenAI-compatible
  415. model: claude-sonnet-4-20250514
  416. api_key: ${ANTHROPIC_API_KEY}
  417. base_url: https://api.anthropic.com # optional, for custom endpoints
  418. max_cost_per_page: 0.02 # budget cap
  419. ```
  420. Supported providers:
  421. | Provider | Models | Local? | Cost |
  422. |----------|--------|--------|------|
  423. | Gemini | 2.5 Flash, 2.5 Pro | No | ~$0.01/page |
  424. | Gemma 3 | 27B IT, 12B IT (great for Arabic) | No (via Gemini key) | ~$0.0002/page |
  425. | Claude | Sonnet, Opus | No | ~$0.015/page |
  426. | GPT-4o | GPT-4o, GPT-4o-mini | No | ~$0.01/page |
  427. | Mistral | `mistral-ocr-latest` | No | $0.002/page |
  428. | Ollama | Any local model | Yes | Free |
  429. | Custom | Any OpenAI-compatible API | Configurable | Varies |
  430. Every provider's `extract_page()` is wrapped in `@with_retry(max_attempts=3,
  431. backoff_base=2.0)`, which honors `Retry-After` headers on 429s and skips
  432. retries on auth failures so a bad key fails fast.
  433. ## Arabic & RTL Support
  434. pdfmux ships first-class support for Arabic, Persian, Urdu, and Hebrew.
  435. Out of the box, RTL detection runs on every PDF and PyMuPDF-extracted
  436. pages are passed through the Unicode Bidirectional Algorithm so glyphs
  437. that were stored in left-to-right order render in correct reading order.
  438. ```bash
  439. # Default install — already includes python-bidi for RTL reordering
  440. pip install pdfmux
  441. # Recommended for Arabic-heavy docs — adds Gemma vision OCR
  442. # (Gemma speaks the OpenAI protocol, so it needs the openai SDK)
  443. pip install "pdfmux[llm-openai]"
  444. # One credential covers Gemma + Gemini (same Google endpoint)
  445. export GEMINI_API_KEY=...
  446. ```
  447. What happens automatically:
  448. - `pdfmux convert` detects Arabic content and routes pages with >5%
  449. Arabic characters through the Arabic-aware extractor chain.
  450. - PyMuPDF, RapidOCR, and Docling outputs are post-processed with the
  451. Bidi algorithm — markdown headings (`#`) and pipe-table rows preserve
  452. structure, only inner text is reordered.
  453. - `DocumentResult.has_arabic` is set to `True` whenever any page contains
  454. Arabic script.
  455. What requires opt-in:
  456. - Vision LLM extraction. Set `--llm-provider gemma` (or any vision
  457. provider) to route Arabic pages through Gemma instead of PyMuPDF.
  458. - Aggressive normalization (Tatweel removal, Alef/Yeh unification,
  459. Tashkeel stripping) — call `pdfmux.arabic.normalize_arabic(text)`
  460. on extracted strings if you need canonicalized output for search or
  461. embedding.
  462. ```python
  463. from pdfmux.arabic import (
  464. is_arabic_text,
  465. is_rtl_dominant,
  466. fix_bidi_order,
  467. normalize_arabic,
  468. )
  469. text = "مرحبا بالعالم"
  470. assert is_arabic_text(text)
  471. assert is_rtl_dominant(text)
  472. # Fix glyph order from PyMuPDF / OCR engines
  473. visual = fix_bidi_order(text)
  474. # Canonicalize for indexing — strip Tatweel, unify Alef variants, drop diacritics
  475. indexable = normalize_arabic("أَحْمَدْ") # → "احمد"
  476. ```
  477. ## Proof: a real customer batch
  478. We measured pdfmux on **433 real customer documents** — technical and safety data sheets, mixed digital and scanned, some encoding-corrupted. Run the naive way first (an early pdfmux CLI in a subprocess, pypdf fallback, no OCR), the pipeline **silently dropped 16 documents — 11 of them with no log line at all.** That was our own tool failing at the exact thing it promises.
  479. Rebuilt with the per-page audit + budgeted OCR cascade: **433 of 433 processed, zero silent failures.** Every unrecoverable page is flagged, not dropped.
  480. *(A small internal confidence-calibration set also ships under `eval/` — it's a regression guard on the confidence gate, not a competitive benchmark; see [`eval/README.md`](eval/README.md).)*
  481. ## Benchmark
  482. On [opendataloader-bench](https://github.com/opendataloader-project/opendataloader-bench) — 200 real-world PDFs (financial filings, academic papers, legal contracts, government reports) — pdfmux scores **0.903 overall — #2 of the 8 engines measured**, behind `opendataloader-hybrid` (0.909). Re-run 2026-07-16 (reproduction below).
  483. | Rank | Engine | Overall | Reading order | Tables (TEDS) | License | GPU |
  484. |---:|---|---:|---:|---:|---|---|
  485. | 1 | opendataloader-hybrid | 0.909 | 0.935 | 0.928 | Apache-2.0 | No |
  486. | **2** | **pdfmux** | **0.903** | **0.920** | **0.911** | **MIT** | **No** |
  487. | 3 | Docling | 0.877 | 0.900 | 0.887 | MIT | Optional |
  488. | 4 | marker | 0.861 | 0.890 | 0.808 | free | GPU |
  489. | 5 | mineru | 0.831 | 0.857 | 0.873 | free | GPU |
  490. Full per-document scores: the [200-PDF head-to-head](https://pdfmux.com/blog/pdfmux-vs-pymupdf-vs-marker-vs-docling/) · methodology: [best PDF extraction library, benchmarked](https://pdfmux.com/blog/best-pdf-extraction-library-python/).
  491. ## Smart Result Cache
  492. Re-running the same extraction is instant. pdfmux hashes every input PDF
  493. (SHA-256) and keys results on `(file_hash, quality, format, schema)`. Cache
  494. files live under `~/.cache/pdfmux/results/`, expire after 30 days, and are
  495. LRU-evicted at 1 GB.
  496. ```bash
  497. pdfmux convert big-report.pdf # first run: 14.2s
  498. pdfmux convert big-report.pdf # cache hit: 0.05s
  499. pdfmux convert big-report.pdf --no-cache # bypass cache (still writes back)
  500. pdfmux convert big-report.pdf --clear-cache # purge and re-run
  501. ```
  502. The cache also speeds up `--profile`, `--schema`, and `--format` switches —
  503. each combination is keyed independently, so you can flip between Markdown
  504. and JSON for the same document for free after the first extraction.
  505. ## Confidence Scoring
  506. Every result includes a 4-signal confidence score:
  507. - **95-100%** -- clean digital text, fully extractable
  508. - **80-95%** -- good extraction, minor OCR noise on some pages
  509. - **50-80%** -- partial extraction, some pages unrecoverable
  510. - **<50%** -- significant content missing, warnings included
  511. When confidence drops below 80%, pdfmux tells you exactly what went wrong and how to fix it:
  512. ```
  513. Page 4: 32% confidence. 0 chars extracted from image-heavy page.
  514. -> Install pdfmux[ocr] for RapidOCR support on 6 image-heavy pages.
  515. ```
  516. ## Cost Modes
  517. | Mode | Behavior | Typical cost |
  518. |------|----------|-------------|
  519. | economy | Rule-based backends only. No LLM calls. | $0/page |
  520. | balanced | LLM only for pages that fail rule-based extraction. | ~$0.002/page avg |
  521. | premium | LLM on every page for maximum quality. | ~$0.01/page |
  522. Set a hard budget cap: `--budget 0.50` stops LLM calls when spend reaches $0.50 per document.
  523. ## Why pdfmux?
  524. pdfmux is not another PDF extractor. It is the orchestration layer that picks the right extractor per page, verifies the result, and retries failures.
  525. | Tool | Good at | Limitation |
  526. |------|---------|-----------|
  527. | PyMuPDF | Fast digital text | Cannot handle scans or image layouts |
  528. | Docling | Tables (97.9% accuracy) | Slow on non-table documents |
  529. | Marker | Neural extraction for academic papers | Needs GPU for speed; overkill for digital PDFs |
  530. | Mistral OCR | Tables (96.6% TEDS), $0.002/page | Cloud-only API |
  531. | Unstructured | Enterprise platform | Complex setup, paid tiers |
  532. | LlamaParse | Cloud-native | Requires API keys, not local |
  533. | Reducto | High accuracy | $0.015/page, closed source |
  534. | **pdfmux** | **Orchestrates all of the above** | Routes per page, audits, re-extracts |
  535. Open source Reducto alternative: what costs $0.015/page elsewhere is free with pdfmux's rule-based backends, or ~$0.002/page average with BYOK LLM fallback.
  536. ## Development
  537. ```bash
  538. git clone https://github.com/NameetP/pdfmux.git
  539. cd pdfmux
  540. python3.12 -m venv .venv && source .venv/bin/activate
  541. pip install -e ".[dev]"
  542. pytest # 659 tests
  543. ruff check src/ tests/
  544. ruff format src/ tests/
  545. ```
  546. ## Contributing
  547. 1. Fork the repo
  548. 2. Create a branch (`git checkout -b feature/your-feature`)
  549. 3. Write tests for new functionality
  550. 4. Ensure `pytest` and `ruff check` pass
  551. 5. Open a PR
  552. ## License
  553. The pdfmux library and MCP server in this repository are **[MIT](LICENSE)** licensed — free for any use, and every released version stays MIT.
  554. The confidence-budgeted **decision-trace** method (the persisted per-page decision trace with retained rejected candidates, and the monotonic repair guard) is **patent-pending** (US Provisional App No. 64/106,302) and is reserved for pdfmux Cloud/Pro under a separate commercial license — it is not part of the MIT grant. See **[LICENSING.md](LICENSING.md)** and **[NOTICE](NOTICE)**.
  555. <!-- mcp-name: io.github.NameetP/pdfmux -->

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.