Skip to content
Back to search

docpilot

Agentic documentation assistant: FastAPI + RAG over product docs, LangGraph agent, and an MCP server for AI coding agents

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/pranay202--docpilot/badge.svg)](/listings/pranay202--docpilot)

HTML

<a href="/listings/pranay202--docpilot"><img src="/api/listings/pranay202--docpilot/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

python -m venv .venv && source .venv/bin/activate
pip install -r requirements-dev.txt
./scripts/fetch_libreoffice_help.sh          # Writer, Calc, Impress, Draw guides → brand_docs/libreoffice
uvicorn app.main:app --reload

Index the four products and ask a question:

for p in writer calc impress draw; do
  curl -s -X POST localhost:8000/ingest -H 'content-type: application/json' \
    -d "{\"product\": \"$p\", \"source\": \"brand_docs/libreoffice/$p\"}"
done
curl -X POST localhost:8000/ask -H 'content-type: application/json' \
  -d '{"question": "How do I refresh a pivot table after the source data changes?", "product": "calc"}'

Open http://localhost:8000 for the chat UI. To index the four products automatically at startup, run `PRELOAD_DOCS=brand_docs/libreoffice uvicorn app.main:app` (Docker Compose sets this).

With an LLM (Gemini, OpenAI, OpenRouter, Anthropic, ...)

Put one API key in `.env` (copy `.env.example`). DocPilot picks the provider from whichever key is set:

cp .env.example .env
echo 'GEMINI_API_KEY=AIza...' >> .env      # or OPENAI_API_KEY / OPENROUTER_API_KEY / ANTHROPIC_API_KEY / ...
uvicorn app.main:app                       # the log line "LLM: Google Gemini, model ..." confirms it
curl localhost:8000/stats                  # {"llm": true, "llm_provider": "gemini", ...}

| `LLM_PROVIDER` | Key env var | Default model | |---|---|---| | `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` | | `gemini` | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `gemini-2.5-flash` | | `anthropic` | `ANTHROPIC_API_KEY` | `claude-haiku-5-5` | | `openrouter` | `OPENROUTER_API_KEY` | `openai/gpt-4o-mini` | | `groq` | `GROQ_API_KEY` | `llama-3.3-70b-versatile` | | `deepseek` | `DEEPSEEK_API_KEY` | `deepseek-chat` | | `together` | `TOGETHER_API_KEY` | `meta-llama/Llama-3.3-70B-Instruct-Turbo` | | `mistral` | `MISTRAL_API_KEY` | `mistral-small-latest` | | `ollama` / `lmstudio` | none (local) | `llama3.1` / `local-model` | | `custom` | `LLM_API_KEY` + `LLM_BASE_URL` | set `LLM_MODEL` |

All of them speak the OpenAI chat API, so one client (`langchain-openai`) serves every provider and no extra SDK is installed. An OpenRouter or Gemini key pasted into `OPENAI_API_KEY` by mistake is recognised by its prefix and routed to the right provider.

With PostgreSQL

docker compose up --build           # api on :8000, MCP over HTTP on :8001, pgvector on :5433; reads .env

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 206 lines
  1. # DocPilot
  2. An agentic help-centre assistant for a brand with several products. A **FastAPI** backend answers user questions over the official documentation with **RAG** and cited sources. An **MCP server** gives AI assistants and agents the same knowledge, including exact how-to steps for a task.
  3. It is built and evaluated on a real, well-known brand: **LibreOffice**, across four products: **Writer, Calc, Impress and Draw**. That is 291 official help guides and 923 indexed sections.
  4. ![CI](https://github.com/pranay202/docpilot/actions/workflows/ci.yml/badge.svg)
  5. ## Results on LibreOffice help
  6. 50 hand-written user questions (`evals/libreoffice.jsonl`): 15 Writer, 15 Calc, 12 Impress, 8 Draw. Each question names the help page and section that answers it and the exact phrase the answer must contain. All of them were checked against the pinned help text.
  7. | Metric | Offline mode (no API key) |
  8. |---|---|
  9. | Retrieval hit@5 (right section in top 5) | **0.92** |
  10. | Answer accuracy (expected phrase in answer) | **0.78** |
  11. | Citation precision (cited sections that are the right one) | **0.69** |
  12. | Average latency per question | ~23 ms |
  13. Offline mode uses a deterministic hashing embedder and extractive answers, so these numbers are a floor that needs no API key or network. CI recomputes them on every push and fails the build if hit@5 < 0.85 or accuracy < 0.70.
  14. Example (Writer):
  15. ```
  16. Q: How do I add line numbers to the whole document?
  17. A: 1. Choose Tools - Line Numbering.
  18. 2. Select Show numbering, and then select the options that you want.
  19. 3. Click OK.
  20. [Adding Line Numbers > To Add Line Numbers to an Entire Document — writer/numbering_lines.xhp]
  21. ```
  22. ## Features
  23. - **Ingestion:** LibreOffice help XML (`.xhp`), Markdown/MDX folders, or a sitemap URL.
  24. - Heading-aware chunking keeps the section path (`Page Numbers > To Start With a Defined Page Number`) for precise citations.
  25. - The `.xhp` parser keeps ordered steps, tips and tables, takes the default (non-macOS) shortcut variants, and drops index bookmarks, comments and images.
  26. - **Hybrid retrieval:** vector recall (6× candidates), then a lexical re-rank that rewards query terms in the section heading. Help guides put the task in the heading ("To Insert a Table of Contents"), which makes this a strong, cheap signal.
  27. - **Vector store:** PostgreSQL + pgvector (HNSW, cosine) in production, in-memory for development and tests.
  28. - **LangGraph agent:** `plan → retrieve → answer → verify`.
  29. - The model must return strict JSON (`answer`, `citations`, `confidence`), validated with Pydantic.
  30. - Invalid JSON, or citations that point at no retrieved section, are rejected and retried with the error as feedback.
  31. - After the retry budget, or when the LLM provider fails, the agent falls back to an extractive, cited answer instead of erroring.
  32. - For "how do I…" questions it returns the documented steps in order, stitching a step list back together when chunking split it.
  33. - **MCP server (FastMCP):** `list_products`, `search_docs`, `ask_docs`, `get_steps`, over stdio or HTTP.
  34. - **Streaming:** `GET /ask/stream` streams each agent step and the final answer over Server-Sent Events; the bundled chat UI uses it.
  35. ## Architecture
  36. ```
  37. Browser (chat UI) ──SSE──▶ FastAPI ─┐
  38. ├─▶ LangGraph agent ──▶ LLM (any provider, optional)
  39. AI assistant ──MCP──▶ FastMCP ──────┘ │ plan → retrieve → answer → verify → (fallback)
  40. ▼
  41. hybrid retrieval (vector recall + heading-aware re-rank)
  42. ▼
  43. PostgreSQL + pgvector (help sections + embeddings)
  44. ```
  45. | Path | What it does |
  46. |---|---|
  47. | `app/ingest/xhp.py` | LibreOffice help XML → Markdown |
  48. | `app/ingest/loader.py` | Folder loader (`.xhp`, `.md`, `.mdx`) and sitemap loader |
  49. | `app/ingest/chunker.py` | Heading-aware chunking with overlap, code-fence safe |
  50. | `app/retrieval.py` | Hybrid search: vector recall + lexical re-rank |
  51. | `app/llm.py` | Provider registry: picks the LLM and embeddings endpoint from env/.env |
  52. | `app/embeddings.py` | Hashing embedder (offline) and OpenAI-compatible embeddings |
  53. | `app/store.py` | In-memory and pgvector stores behind one interface |
  54. | `app/agent.py` | LangGraph agent, JSON validation, retries, step extraction, extractive fallback |
  55. | `app/service.py` | Wiring, ingestion pipeline, streaming |
  56. | `app/main.py` | REST + SSE API and chat UI |
  57. | `app/mcp_server.py` | MCP tools |
  58. | `evals/` | LibreOffice and sample datasets, evaluation runner |
  59. | `scripts/fetch_libreoffice_help.sh` | Fetches the help guides at a pinned commit |
  60. ## Quickstart
  61. ```bash
  62. python -m venv .venv && source .venv/bin/activate
  63. pip install -r requirements-dev.txt
  64. ./scripts/fetch_libreoffice_help.sh # Writer, Calc, Impress, Draw guides → brand_docs/libreoffice
  65. uvicorn app.main:app --reload
  66. ```
  67. Index the four products and ask a question:
  68. ```bash
  69. for p in writer calc impress draw; do
  70. curl -s -X POST localhost:8000/ingest -H 'content-type: application/json' \
  71. -d "{\"product\": \"$p\", \"source\": \"brand_docs/libreoffice/$p\"}"
  72. done
  73. curl -X POST localhost:8000/ask -H 'content-type: application/json' \
  74. -d '{"question": "How do I refresh a pivot table after the source data changes?", "product": "calc"}'
  75. ```
  76. Open http://localhost:8000 for the chat UI. To index the four products automatically at startup, run `PRELOAD_DOCS=brand_docs/libreoffice uvicorn app.main:app` (Docker Compose sets this).
  77. ### With an LLM (Gemini, OpenAI, OpenRouter, Anthropic, ...)
  78. Put one API key in `.env` (copy `.env.example`). DocPilot picks the provider from whichever key is set:
  79. ```bash
  80. cp .env.example .env
  81. echo 'GEMINI_API_KEY=AIza...' >> .env # or OPENAI_API_KEY / OPENROUTER_API_KEY / ANTHROPIC_API_KEY / ...
  82. uvicorn app.main:app # the log line "LLM: Google Gemini, model ..." confirms it
  83. curl localhost:8000/stats # {"llm": true, "llm_provider": "gemini", ...}
  84. ```
  85. | `LLM_PROVIDER` | Key env var | Default model |
  86. |---|---|---|
  87. | `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` |
  88. | `gemini` | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | `gemini-2.5-flash` |
  89. | `anthropic` | `ANTHROPIC_API_KEY` | `claude-haiku-5-5` |
  90. | `openrouter` | `OPENROUTER_API_KEY` | `openai/gpt-4o-mini` |
  91. | `groq` | `GROQ_API_KEY` | `llama-3.3-70b-versatile` |
  92. | `deepseek` | `DEEPSEEK_API_KEY` | `deepseek-chat` |
  93. | `together` | `TOGETHER_API_KEY` | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
  94. | `mistral` | `MISTRAL_API_KEY` | `mistral-small-latest` |
  95. | `ollama` / `lmstudio` | none (local) | `llama3.1` / `local-model` |
  96. | `custom` | `LLM_API_KEY` + `LLM_BASE_URL` | set `LLM_MODEL` |
  97. All of them speak the OpenAI chat API, so one client (`langchain-openai`) serves every provider and no extra SDK is installed. An OpenRouter or Gemini key pasted into `OPENAI_API_KEY` by mistake is recognised by its prefix and routed to the right provider.
  98. ### With PostgreSQL
  99. ```bash
  100. docker compose up --build # api on :8000, MCP over HTTP on :8001, pgvector on :5433; reads .env
  101. ```
  102. ## API
  103. | Method | Path | Purpose |
  104. |---|---|---|
  105. | GET | `/health` | Liveness |
  106. | GET | `/stats` | Indexed section count, products, LLM provider and model (or `llm: false`) |
  107. | POST | `/ingest` | Index a docs folder (inside `DOCS_ROOT`) or a sitemap URL for a product |
  108. | POST | `/ask` | Question → `{answer, citations[], confidence, mode, grounded}` |
  109. | GET | `/ask/stream` | Same, streamed over SSE (`step` events, then `answer`) |
  110. ## MCP server
  111. ```bash
  112. python -m app.mcp_server # stdio
  113. python -m app.mcp_server --http --port 8001
  114. ```
  115. Example client config (`.mcp.json`):
  116. ```json
  117. {
  118. "mcpServers": {
  119. "docpilot": { "command": "python", "args": ["-m", "app.mcp_server"], "cwd": "/path/to/docpilot" }
  120. }
  121. }
  122. ```
  123. | Tool | Returns |
  124. |---|---|
  125. | `list_products()` | Indexed products |
  126. | `search_docs(query, product?, k?)` | Matching sections with source and score |
  127. | `ask_docs(question, product?)` | Cited answer |
  128. | `get_steps(task, product?)` | Ordered steps for a task, verbatim from the docs, with sources |
  129. ## Configuration
  130. | Variable | Default | Meaning |
  131. |---|---|---|
  132. | `DATABASE_URL` | unset (in-memory) | PostgreSQL with pgvector |
  133. | `LLM_PROVIDER` | `auto` | `auto` (first key found), a provider id from the table above, or `none` |
  134. | `LLM_API_KEY` | unset | Generic key for the named provider (else its own key env var) |
  135. | `OPENAI_API_KEY`, `GEMINI_API_KEY`, `OPENROUTER_API_KEY`, ... | unset (offline mode) | Provider keys; any one enables LLM answers |
  136. | `LLM_MODEL` | provider default | Chat model |
  137. | `LLM_BASE_URL` | provider default | Any OpenAI-compatible `/v1` URL |
  138. | `LLM_TIMEOUT` | `30` | Seconds per LLM call |
  139. | `EMBEDDING_PROVIDER` | `auto` | `auto` (OpenAI if `OPENAI_API_KEY` is set, else hashing), `hashing`, `openai`, `gemini`, `mistral`, `ollama` |
  140. | `EMBEDDING_MODEL` / `EMBEDDING_DIM` | provider default / `384` | Embeddings; re-ingest after changing them |
  141. | `LOG_LEVEL` | `INFO` | Provider errors are logged as warnings before the extractive fallback |
  142. | `DOCS_ROOT` | working directory | Only folders inside it can be ingested |
  143. | `CHUNK_SIZE` / `CHUNK_OVERLAP` | `800` / `120` | Chunking |
  144. | `MAX_ANSWER_RETRIES` | `2` | Retries before the extractive fallback |
  145. ## Tests and evals
  146. ```bash
  147. pytest -q # 38 tests
  148. python -m evals.run_eval --dataset evals/libreoffice.jsonl --docs brand_docs/libreoffice
  149. python -m evals.run_eval # small bundled sample set
  150. ```
  151. The pgvector test runs when `DATABASE_URL` is set, and the LibreOffice threshold test runs once the help docs are fetched. CI does both: it runs against a pgvector service container and the pinned help docs.
  152. Eval rows name the expected help page and section, the phrase the answer must contain, and any equally correct alternative the docs give (for example *Insert - Page Numbers* as well as *Insert - Field - Page Number*).
  153. ## Data and licensing
  154. The LibreOffice help content is © The Document Foundation and contributors, under the Mozilla Public License 2.0. It is not stored in this repository: `scripts/fetch_libreoffice_help.sh` downloads it at commit `72e318c` of [LibreOffice/help](https://github.com/LibreOffice/help), so every eval run uses the same text. LibreOffice is a trademark of The Document Foundation; this project is independent and not affiliated with it.
  155. ## Roadmap
  156. - LLM-mode eval with a model-graded rubric
  157. - Cross-encoder re-ranker
  158. - Incremental re-ingestion by content hash
  159. - Per-tenant API keys and rate limiting
  160. ## License
  161. MIT (code). Documentation content: see *Data and licensing*.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.