Skip to content
Back to search

dataroom

Give a query, get a dataroom. Pi + self-hosted Qwen3.6 research harness on a single L4.

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/hanxiao--dataroom/badge.svg)](/listings/hanxiao--dataroom)

HTML

<a href="/listings/hanxiao--dataroom"><img src="/api/listings/hanxiao--dataroom/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 301 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Dataroom
  2. Give it a query. A local model in a [Pi](https://pi.dev) harness loops search-read-write until it has built a comprehensive, fully-cited **dataroom** on disk - a `.zip` you hand to a frontier model for the long-horizon task.
  3. <p align="center">
  4. <img src="assets/banner.png" width="860"
  5. alt="Give a query to a self-hosted pi + harness + local model loop; it loops search-read-write to build a dataroom and hands you a .zip" />
  6. </p>
  7. <p align="center">
  8. <b>Live demo → <a href="https://dataroom.hanxiao.io">dataroom.hanxiao.io</a></b>
  9. </p>
  10. ## Why
  11. [For long-horizon task you need a grounded, well-organized knowledge dump before the real work can start.](https://x.com/hxiao/status/2044765001370701981?s=20) That upfront research is mostly a search-read-write loop, and few things are usually wrong with how it gets done today.
  12. - **Research is mechanical, so don't pay frontier tokens for it.** Gathering and organizing sources is tool-calling, not deep reasoning - a small local model in a disciplined harness (search, dedup, cite, verify) does it fine. And because it runs on your own GPU at near-zero marginal token cost, it can keep going for hours until the dataroom is actually comprehensive, instead of stopping when a metered budget runs out.
  13. - **The output is context for a machine, not a report for a human.** A 2025-style deep-research run ends in a long PDF nobody reads. Dataroom ends in a structured `.zip` - `topics/`, `sources/`, `data/`, a `SUMMARY.md`, every claim cited - built to be consumed by the next agent, not skimmed.
  14. - **It is stage one of a two-stage pipeline.** Unzip the dataroom into a frontier model's context and let it do the expensive second stage (usually implementation). The research does not have to be perfect - its consumer is intelligent and can spot gaps - it has to be comprehensive and grounded.
  15. Everything runs locally on your own GPU: the model is self-hosted (llama.cpp), and the only thing that leaves the box is the web search/read the agent chooses to do.
  16. ## How it works
  17. <p align="center">
  18. <img src="assets/screenshot-home.png" width="800"
  19. alt="Dataroom homepage: a query box and a live list of jobs with status, file counts, and pause/resume/download controls" />
  20. </p>
  21. Submit a query and an async job spins up a headless Pi coding agent backed by a self-hosted Qwen3.6-35B-A3B (llama.cpp). The agent runs its own research loop: `pi --mode json --continue` resumes the same per-cwd session across turns, and on each turn it searches, reads, reranks, and writes sourced files into a `dataroom/` directory on disk.
  22. - Autonomous loop: the agent is not micromanaged. It is handed tools and a one-page methodology, then drives itself - search, read, dedup, write, verify - until the work is done.
  23. - Outcome-based stopping: `DONE` is honored only once the dataroom holds enough substantive sourced files, all sub-questions are closed, and a `SUMMARY.md` exists. Turns / seconds / Jina-call caps are only hard backstops, and a premature `DONE` is rejected so the agent keeps going. The reason it stopped is surfaced on the dashboard.
  24. - jina CLI: the `jina` CLI is on PATH (search / read / rerank / embed / dedup), driven from bash and composable via pipes (`jina search Q | jina rerank R`, `cat urls.txt | jina read`, `xargs -P 8` for parallel fan-out) so bulky intermediates stay out of the LLM context.
  25. - Embedding dedup index: `jina-embeddings-v5-nano` is preloaded for the dataroom index (embed / semantic search / dedup), with server-side reconciliation so it never drifts from disk. The agent must search the index before adding anything, to avoid duplicates and keep structure.
  26. - Live dashboard: real-time context utilization, throughput, tool-call distribution, live activity feed, warnings/errors, progress-to-floor, a stop-reason banner, and the dataroom file tree, at `GET /jobs/{id}/dashboard`.
  27. The [live dashboard](https://dataroom.hanxiao.io) for a finished job - progress-to-floor, total tokens, tool-call distribution, throughput, the activity feed, and the dataroom file tree:
  28. <p align="center">
  29. <img src="assets/screenshot-dashboard.png" width="800"
  30. alt="Dataroom job dashboard: progress-to-floor, total tokens, tool-call distribution, throughput, live activity feed, and the dataroom file tree" />
  31. </p>
  32. ## Get Started
  33. An NVIDIA Docker host runs two containers (llama-server + the app). `scripts/setup.sh` installs Docker + the NVIDIA toolkit, downloads the model, and brings the stack up. The only value you must set is `JINA_API_KEY`; everything else in `.env.example` ships with working defaults. No NVIDIA GPU? Option C runs it on an Apple Silicon Mac.
  34. Clone and set the key once:
  35. ```bash
  36. git clone https://github.com/hanxiao/dataroom.git
  37. cd dataroom
  38. cp .env.example .env
  39. sed -i 's/^JINA_API_KEY=.*/JINA_API_KEY=jina_your_real_key/' .env
  40. ```
  41. Then pick one (each brings the stack up and prints the API URL when done):
  42. <details open>
  43. <summary><b>Option A: prebuilt image (fastest)</b></summary>
  44. Pull the published app image from GHCR instead of building it locally (skips the ~14GB build). `setup.sh` still installs Docker + the toolkit and downloads the model, then pulls + starts the stack:
  45. ```bash
  46. DAAS_PULL=1 bash scripts/setup.sh
  47. ```
  48. Pulls `ghcr.io/hanxiao/dataroom:latest`.
  49. </details>
  50. <details>
  51. <summary><b>Option B: build from source</b></summary>
  52. Build the app image locally (no pull). Same one-shot, just slower the first time:
  53. ```bash
  54. bash scripts/setup.sh
  55. ```
  56. </details>
  57. <details>
  58. <summary><b>Option C: Apple Silicon (Mac, Metal)</b></summary>
  59. No Docker, no NVIDIA. The model runs on Metal via Homebrew `llama.cpp`; the app, Pi agent, and embedder run in a local `uv` venv. Needs 32 GB+ unified memory (the Q4 model wires ~22 GB).
  60. ```bash
  61. brew install llama.cpp
  62. npm install -g @earendil-works/pi-coding-agent@0.78.0
  63. uv venv --python 3.11 .venv
  64. uv pip install --python .venv/bin/python torch -r server/requirements.txt jina-cli huggingface-hub hf_transfer
  65. mkdir -p models/mtp
  66. HF_HUB_ENABLE_HF_TRANSFER=1 HF_TOKEN=hf_... .venv/bin/python -c "from huggingface_hub import hf_hub_download; \
  67. hf_hub_download('unsloth/Qwen3.6-35B-A3B-MTP-GGUF','Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf',local_dir='models/mtp')"
  68. cp .env.example .env && sed -i '' 's/^JINA_API_KEY=.*/JINA_API_KEY=jina_your_real_key/' .env
  69. bash scripts/mac-run.sh
  70. ```
  71. GGUF and Metal-flag details (MTP needs `llama.cpp` >= 9430): [`docs/MAC.md`](docs/MAC.md).
  72. </details>
  73. <details>
  74. <summary><b>Option D: Windows (WSL2 + Docker Desktop)</b></summary>
  75. The same Docker stack as A/B, run from a WSL2 shell with Docker Desktop (GPU via WSL integration, no NVIDIA toolkit). It validates Docker + GPU, downloads the model, and starts the stack:
  76. ```bash
  77. bash scripts/setup-win.sh # or DAAS_PULL=1 bash scripts/setup-win.sh to pull the prebuilt image
  78. ```
  79. </details>
  80. Prereqs:
  81. - An NVIDIA GPU with the driver installed (`nvidia-smi` must work) and the `nvidia-container-toolkit` (Options A/B/D). `setup.sh` installs the toolkit on Debian/Ubuntu hosts; on RHEL-family hosts install it yourself first. The llama-server needs the GPU; the app's v5-nano embedder runs on CPU by default (`EMBED_DEVICE=cpu`) to leave VRAM for the Q4 model (set `EMBED_DEVICE=cuda` to move it onto the GPU).
  82. - A Jina API key: https://jina.ai/api-dashboard/
  83. - Disk for a ~22GB model download plus the CUDA + pytorch base images and job data under `./data`. The model download alone can take several minutes on a slow link; it resumes from the Hugging Face cache if interrupted.
  84. ## Skill & API usage
  85. ### Skill
  86. Another LLM/agent can commission a dataroom from a deployed instance with the `use-dataroom` skill ([`skills/use-dataroom/SKILL.md`](skills/use-dataroom/SKILL.md)): submit a query with a **minutes time-box** (like handing an intern a time-boxed task), poll until it finishes, then download and unzip the result. One-shot:
  87. ```bash
  88. BASE="https://dataroom.hanxiao.io" # the deployed instance
  89. QUERY="Competitive landscape of self-hosted small embedding models in 2026"
  90. MINUTES=30 # time box: works up to this long, then hands over
  91. JOB=$(curl -s -X POST "$BASE/jobs" -H 'content-type: application/json' \
  92. -d "{\"query\": $(python3 -c 'import json,sys;print(json.dumps(sys.argv[1]))' "$QUERY"), \"max_seconds\": $((MINUTES*60))}" \
  93. | python3 -c 'import sys,json; print(json.load(sys.stdin)["job_id"])')
  94. echo "watch: $BASE/jobs/$JOB/dashboard"
  95. while :; do
  96. S=$(curl -s "$BASE/jobs/$JOB" | python3 -c 'import sys,json; print(json.load(sys.stdin).get("status","?"))')
  97. case "$S" in done|stopped|failed) break;; esac; sleep 30
  98. done
  99. curl -s -OJ "$BASE/jobs/$JOB/result" && unzip -oq "dataroom-$JOB.zip" -d "$JOB" # -> $JOB/dataroom/
  100. ```
  101. `stopped` (time box reached) is a success, not an error - you still get the dataroom built so far. See the skill file for status meanings, partial `/snapshot` downloads, and the full endpoint table.
  102. ### API
  103. Once the stack is up, the API is on port 8000 (open, no auth). `{JOB}` is the 12-hex id returned by `POST /jobs`.
  104. ```bash
  105. # submit a job -> {"job_id":"<12hex>","status":"queued"}
  106. curl -s -X POST localhost:8000/jobs -H 'content-type: application/json' \
  107. -d '{"query":"Competitive landscape of self-hosted small embedding models in 2026"}'
  108. # optionally cap work: -d '{"query":"...","max_turns":50,"max_seconds":3600}'
  109. JOB=abc123def456
  110. # list all jobs (live + on-disk), newest first
  111. curl -s localhost:8000/jobs
  112. # list one page, newest first (large backlogs): ?limit=100&offset=0 -> {jobs,total,limit,offset}
  113. curl -s 'localhost:8000/jobs?limit=50&offset=0'
  114. # single job status
  115. curl -s localhost:8000/jobs/$JOB
  116. # pause (sticky: 'held' - never auto-resumed) / resume a job
  117. curl -s -X POST localhost:8000/jobs/$JOB/pause
  118. curl -s -X POST localhost:8000/jobs/$JOB/resume
  119. # per-job metrics feed (drives the dashboard)
  120. curl -s localhost:8000/jobs/$JOB/stats
  121. # tail of the Pi agent log (last 8000 chars)
  122. curl -s localhost:8000/jobs/$JOB/log
  123. # download the dataroom AS-IT-IS-NOW (works mid-run)
  124. curl -s -OJ localhost:8000/jobs/$JOB/snapshot
  125. # download the FINAL dataroom zip (409 until the job stops)
  126. curl -s -OJ localhost:8000/jobs/$JOB/result
  127. # read one dataroom file (path is relative to the job's dataroom/ dir)
  128. curl -s 'localhost:8000/jobs/'$JOB'/file?path=SUMMARY.md'
  129. # open the live dashboard in a browser
  130. open http://localhost:8000/jobs/$JOB/dashboard
  131. ```
  132. There is also a minimal submit page at `GET /` and a liveness probe at `GET /health` (`{"ok":true}`).
  133. ### Scheduling (single GPU, one slot)
  134. The L4 serves one llama slot, so jobs run **serially**. The scheduler is:
  135. - **Foreground FIFO.** Fresh submits and explicit resumes are foreground work and run oldest-first.
  136. - **Preemption.** A new foreground job preempts whatever is running - including another foreground
  137. job - and takes the slot immediately (`PREEMPT_FOREGROUND=1`, the default). The preempted job
  138. returns to the pool and resumes ahead of bulk backfill. Set `PREEMPT_FOREGROUND=0` to make a new
  139. job wait for the running one instead.
  140. - **Auto-backfill.** When no foreground work is queued, the oldest paused job is auto-resumed to
  141. keep the GPU busy (`AUTO_BACKFILL=1`). A backfill run is preemptible by any foreground job.
  142. - **Pause is sticky.** A user `POST /pause` puts the job in `held` - it is *not* auto-resumed; only
  143. an explicit `POST /resume` revives it. (System preemption uses a separate, preemptible `paused`
  144. state.)
  145. - **Cumulative budget.** A job that never reaches its coverage floor would otherwise be backfilled
  146. forever. Once its lifetime GPU time crosses `MAX_CUMULATIVE_SECONDS` it retires
  147. (`stopped`, `stop_reason=budget_exhausted`) and stops being re-picked. An explicit resume grants
  148. a fresh budget. Per-run caps (`MAX_SECONDS` etc.) still apply to each individual run.
  149. State survives an app restart: in-flight jobs re-queue in submit order, user-held jobs stay held.
  150. ## Architecture
  151. Two containers on a single GPU host: `daas-llama` serves the model, `daas-app` runs the FastAPI orchestrator + the Pi agent + the embedding index. The agent loops turns until the dataroom meets the outcome floor, then the orchestrator zips it.
  152. ```mermaid
  153. flowchart LR
  154. user([User / curl / Web UI]) -->|POST /jobs query| api
  155. subgraph host["NVIDIA Docker host (single L4 24GB)"]
  156. subgraph appc["container: daas-app (:8000, GPU)"]
  157. api["FastAPI app.py<br/>/jobs /result /snapshot<br/>/file /stats /log /dashboard /health"]
  158. orch["orchestrator<br/>run_dataroom.py<br/>floor + ceiling guard"]
  159. pi["Pi coding agent<br/>pi --mode json --continue"]
  160. jina["jina CLI on PATH<br/>search / read / rerank / embed"]
  161. emb["v5-nano embedder<br/>EMBED_DEVICE=cpu (off-GPU)<br/>dataroom_index"]
  162. disk[("/data/jobs/&lt;id&gt;/<br/>dataroom/ + meta + logs")]
  163. end
  164. subgraph llamac["container: daas-llama (:8080, GPU)"]
  165. llama["llama-server<br/>Qwen3.6-35B-A3B UD-Q4_K_XL<br/>+ MTP draft, ctx 131072"]
  166. end
  167. end
  168. jinacloud([Jina API<br/>jina.ai]):::ext
  169. api -->|spawn thread + subprocess| orch
  170. orch -->|loop turns| pi
  171. pi -->|OpenAI-compat /v1<br/>LLAMA_URL| llama
  172. pi -->|bash| jina
  173. jina -->|JINA_API_KEY| jinacloud
  174. pi -->|index / search / dedup| emb
  175. pi -->|read / write / edit| disk
  176. emb -.reconcile.- disk
  177. orch -->|zip dataroom/| disk
  178. disk -->|GET /result or /snapshot .zip| user
  179. classDef ext fill:#eee,stroke:#999,stroke-dasharray:4 3;
  180. ```
  181. By default `llama-server` serves `Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf` (repo `unsloth/Qwen3.6-35B-A3B-MTP-GGUF`) with MTP draft flags, and the agent model id is `qwen3.6`.
  182. ### Switching the model
  183. **One knob.** Set `MODEL=<hf_repo>/<file.gguf>` in `.env` and re-run `scripts/setup.sh`. It derives the download repo and filename from `MODEL`, pulls the GGUF, and persists the filename so `docker-compose` serves the same file - download and serve stay in sync.
  184. ```bash
  185. # .env (default)
  186. MODEL=unsloth/Qwen3.6-35B-A3B-MTP-GGUF/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf
  187. ```
  188. That is all you change to swap the LLM. The rest are **advanced overrides**, rarely needed (leave unset to use the defaults derived from `MODEL`):
  189. | Env var | Default | Role |
  190. | --- | --- | --- |
  191. | `MODEL_ID` | `qwen3.6` | Agent-facing model id (Pi `models.json` / `defaultModel`). A free label; need not match the GGUF. |
  192. | `CHAT_TEMPLATE_FILE` | `/templates/chat_template.jinja` | Jinja chat template inside the llama-server container. |
  193. | `SPEC_ARGS` | `--spec-type draft-mtp --spec-draft-n-max 2` | MTP / speculative-draft flags appended to `llama-server`. |
  194. Non-Qwen caveat: switching to a non-Qwen GGUF is not just a filename swap. The bundled chat template is Qwen3.6-specific - point `CHAT_TEMPLATE_FILE` at the new model's Jinja template (a wrong template silently corrupts tool-calling), or drop the flag to use the GGUF's embedded template. `--spec-type draft-mtp` needs a GGUF that ships an MTP draft head (the `...-MTP-GGUF` repo does); for a plain GGUF set `SPEC_ARGS=` (empty). The `CTX_SIZE` default of 131072 is tuned to Qwen3.6's hybrid GDN+MoE KV math; a dense model of similar size uses far more KV per token, so lower `CTX_SIZE` or it may OOM on the L4. Re-measure VRAM with `nvidia-smi` for any other weights. See `docs/DEPLOY.md` for the deeper reproducibility detail.
  195. **The default L4 tune.** The shipped llama-server settings are the best we could squeeze out of a low-budget L4 (24GB VRAM) without sacrificing generation quality, tuned in [`Qwen3.6-35B-A3B-MTP-L4`](https://github.com/hanxiao/Qwen3.6-35B-A3B-MTP-L4):
  196. | Setting | Value | Why |
  197. | --- | --- | --- |
  198. | Quant | `Qwen3.6-35B-A3B` **UD-Q4_K_XL** (~22GB) | best quality that still fits 24GB alongside MTP + KV |
  199. | MTP draft | `--spec-type draft-mtp --spec-draft-n-max 2` (no `--spec-draft-p-min`) | ~80-90% draft acceptance; `n-max 2` is the sweet spot on this MoE, `p-min` hurts MoE |
  200. | KV cache | `--cache-type-k/v q4_0` | this hybrid GDN+MoE has only 10/40 KV-bearing layers, so q4_0 KV is tiny (~0.65GB at 131072) |
  201. | Batch | `-ub 256 -b 2048` | measured-best prefill throughput |
  202. | Context | `--ctx-size 131072` | full native window; fits with q4_0 KV |
  203. | Offload | `-ngl` unset (auto-fit) + mmap on | forcing all layers to GPU OOMs once MTP + KV load; auto-fit spills compute-light expert layers to CPU |
  204. | Cache reuse | omitted | GDN recurrent-state drift can silently corrupt digits (llama.cpp#21681) |
  205. Measured ~22.2GB used at the full 131072 window, no OOM. A smaller **Q3_K_XL** (~17GB) would free enough VRAM to also put the v5-nano embedder on the GPU - but embedding is not the bottleneck (LLM decode is), so we keep the embedder on CPU and spend the freed headroom on **Q4 for slightly better generation quality** instead.
  206. ## Local dev (no GPU)
  207. Point the agent at any OpenAI-compatible endpoint (or a remote Qwen box) via `LLAMA_URL`, then run the harness directly:
  208. ```bash
  209. uv venv && uv pip install -r server/requirements.txt
  210. JINA_API_KEY=... LLAMA_URL=http://<host>:8080 \
  211. uv run python -m server.run_dataroom --query "your query" --out ./out
  212. ```
  213. ## License
  214. MIT

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.