Skip to content
Back to search

mazzap

Mazzap, part of the Mazzstack: essentials for the Singularity Slowlife

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today
Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/zymazza--mazzap/badge.svg)](/listings/zymazza--mazzap)

HTML

<a href="/listings/zymazza--mazzap"><img src="/api/listings/zymazza--mazzap/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

Three ways in, easiest first — each writes a self-contained twin you then view with `npm start`:

# 1. From a map, in the browser: draw an area, press "Set area & scan layers".
npm run init                                            # -> http://127.0.0.1:4173/init.html

# 2. From an AOI polygon (US): one command fetches terrain, imagery,
#    vegetation, and soils for that footprint.
npm run build-from-aoi -- --aoi my_area.shp --data-dir ./twins/mine/data --name "My Place"

# 3. Anywhere on Earth, from global open data (see "Build a twin anywhere" below):
python3 packs/nato/fetch_nato.py --country FR --aoi 2.6,48.4,2.7,48.5 \
  --aoi-crs EPSG:4326 --data-dir ./twins/mine/data --name "My Place"

# then view it:
TWIN_DATA_DIR=./twins/mine/data npm start               # -> http://127.0.0.1:4173

Everything below is the detail behind those three paths.

**First-time setup from a map** — launch the initiation page, optionally search for a U.S. street address to jump the map to that location, draw an AOI on the basemap, then press **Set area & scan layers**:

npm run init

By default this builds into `./twins/init/data` and opens `http://127.0.0.1:4173/init.html`. Set `TWIN_DATA_DIR` first to build somewhere else:

TWIN_DATA_DIR=./twins/mine/data npm run init

The init page writes the drawn polygon to `<data>/init/aoi.geojson`, then runs the same national AOI builder as the CLI path below. The nationally available layers currently implemented in this repo are 3DEP elevation, 3DEP LiDAR Point Cloud tiles where USGS has coverage, NAIP Plus orthoimagery, LANDFIRE EVT, and additional LANDFIRE forest/fire ecology rasters, and gSSURGO soils, plus `us-national` vegetation typing from LANDFIRE. LiDAR, the extra LANDFIRE rasters, and gSSURGO are best-effort: if a source is unavailable, the twin still builds from the remaining national sources. When the build finishes, open the viewer link shown on the setup page.

**US, fetch everything from one AOI** — hand it a small AOI polygon (shapefile, GeoJSON, GeoPackage…) and it queries 3DEP elevation, 3DEP LiDAR Point Cloud tiles where available, NAIP Plus orthoimagery, LANDFIRE land cover/vegetation, additional LANDFIRE forest/fire ecology rasters, and gSSURGO soils for that footprint and builds a complete, typed twin:

npm run build-from-aoi -- --aoi my_area.shp --data-dir ./twins/mine/data --name "My Place"
TWIN_DATA_DIR=./twins/mine/data PORT=4174 npm start    # -> http://127.0.0.1:4174

# the bundled demo is exactly this, from one committed shapefile:
npm run demo && npm run serve-demo

**Build a twin anywhere (global open data)** — the paths above are US-only. The bundled `nato` pack (`packs/nato/`) extends Mazzap to **any of the 32 NATO member nations**, assembling a twin from worldwide open datasets with no national account or portal needed. Give it an ISO country code and a lon/lat AOI:

python3 packs/nato/fetch_nato.py --country FR --aoi 2.6,48.4,2.7,48.5 \
  --aoi-crs EPSG:4326 --data-dir ./twins/mine/data --name "My Place"
TWIN_DATA_DIR=./twins/mine/data npm start

Coverage is **tiered, and the pack is honest about which tier you get**. 14 members ship a national adapter, but the tier they deliver varies. Seven pull their own higher-resolution national terrain today (Tier A) — **BE, CZ, ES, FR, LU, NL, NO**; the other seven adapters (**DK, EE, FI, LV, PL, SE, SK**) are wired up and probe their national services, but currently fall back to the **global 30 m stack** where an anonymous/tokenless high-res route wasn't reachable (per-country specifics in [`packs/nato/README.md`](packs/nato/README.md)). Every remaining member also resolves through that global stack, which works for any land AOI on Earth. So every member resolves to a working fetch path — but *builds a twin* is not the same as *builds a great twin*: where a member relies on the global fallback, the terrain (30 m) and the canopy are **modeled, not

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 857 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # Mazzap v2 — a georeferenced 3D digital-twin engine
  2. https://github.com/user-attachments/assets/deb9e860-abf8-4e34-ab9b-88b3c7a2643f
  3. Mazzap v2 turns a patch of real ground into a **VEIL — a Virtually Embodied
  4. Intelligent Land**: a standalone, fully **georeferenced** 3D digital twin that
  5. Mazzap models and instantiates from open geospatial data. Open it in a browser,
  6. click to read true GPS coordinates, drape your own map layers onto the terrain,
  7. **simulate** the processes that move water and fire across it, and ask questions
  8. about it in natural language.
  9. No database, no cloud, no build step at view time: one tiny zero-dependency Node
  10. static server serves a Three.js viewer over a self-contained bundle of geospatial
  11. data. Nothing is fetched from the network when you view it.
  12. ```bash
  13. # A fresh clone ships the engine, not a place — build a twin first:
  14. npm run demo # build the bundled Flatirons demo twin (needs internet + GDAL)
  15. npm run serve-demo # -> http://127.0.0.1:4174
  16. # ...or build your own area interactively, then serve it:
  17. npm run init # guided setup: draw an AOI, fetch data, build the twin
  18. npm start # -> http://127.0.0.1:4173
  19. ```
  20. (`npm start` with no twin built yet just tells you to run one of the above.)
  21. Requires Node ≥ 18 (the server uses the built-in `fetch`); the data pipeline
  22. scripts need Python 3 with GDAL (`osgeo`), numpy, pyproj, and Pillow. The
  23. MCP/chat path also needs the Python `mcp` SDK from `requirements.txt`. If you'd
  24. rather not assemble that toolchain yourself, **[run it in a container](#run-with-docker)** —
  25. GDAL, numpy, Node, and the rest come pinned and pre-built.
  26. Point it at **your own** DEM and imagery (see "Build your own twin" below) — the
  27. engine is region-agnostic. The coordinate system, the vegetation knowledge, the
  28. map-layer styling, and any source-acquisition scripts all live in data and in an
  29. optional **regional pack**, never hardcoded in the engine.
  30. ## What you get
  31. - **3D terrain** from any DEM (a LiDAR DTM, USGS 3DEP, a national survey grid),
  32. rendered as a lit mesh with selectable surface modes (aerial / false-color /
  33. hillshade / elevation).
  34. - **Aerial imagery** draped on the terrain, aligned to the grid so it conforms
  35. to the topography instead of floating.
  36. - **Classified point-cloud view** — switch independently between the normal
  37. textured DEM/rendered-tree mesh scene and a PLY-style LiDAR viewer. Point
  38. presets expose ground, ground + vegetation, vegetation, or every return;
  39. points can be colored by the orthophoto, ASPRS class, elevation, or intensity.
  40. - **Click-to-identify + GPS readout** — click anywhere to pin the real-world
  41. lat/lon (validated against proj4js), elevation, and every map feature true at
  42. that spot.
  43. - **Atlas layers** — import any vector or raster geospatial file and it is
  44. reprojected to the scene, clipped to the terrain footprint, auto-styled, and
  45. made clickable. Bring soils, wetlands, zoning, habitat, hydrology, land cover —
  46. whatever you have.
  47. - **Vegetation** — a capability ladder that derives trees from the best signal
  48. available (LiDAR segmentation stems, a canopy-height model, or an NDVI canopy),
  49. rendered as low-LOD 3D forms. Anywhere in the continental US, the bundled
  50. **`us-national`** pack types them evergreen/deciduous and names their
  51. community from LANDFIRE — no regional setup (see "Vegetation" under
  52. [docs/make-a-twin.md](docs/make-a-twin.md)).
  53. - **Simulation** — model the processes that move across the land, not just what's
  54. on it: terrain **hydrology** (flow, wetness, ponding, seeps, and snowmelt/storm
  55. runoff scenarios), **evapotranspiration & water balance** (FAO-56 reference and
  56. actual ET, closing the land's water budget), **wildfire** behavior scenarios,
  57. and **solar siting** (panel radiation, PV yield, shade/cloud losses, and angle
  58. recommendations). Each renders as draped, clickable layers in the viewer's Simulation
  59. window, and is honest about its uncertainty. See ["Simulation"](#simulation).
  60. - **Solar panel planning** — combine astronomy, viewshed horizons, and Daymet
  61. climate normals to rank panel sites, answer proposed-site questions, and
  62. visualize annual/winter PV potential. See ["Solar siting"](#solar-siting).
  63. - **Astronomy** — a local ephemeris-driven sky, lighting model, and time scrubber
  64. for sun, moon, planets, stars, constellations, eclipses, rise/set, golden hour,
  65. and terrain-aware solar context. See ["Astronomy"](#astronomy).
  66. - **Viewshed & distant terrain** — compute what is visible from an observer point,
  67. rank viewpoints, query visibility through MCP, and render 3DEP distant terrain
  68. with NAIP drapes around the twin. See ["Viewshed & distant terrain"](#viewshed--distant-terrain).
  69. - **Ask the land chat** — a collapsible panel that talks to an LLM wired to the
  70. twin's read-only query tools (the MCP server), scoped to the whole twin, a
  71. polygon you draw, or a point you pick. See [docs/mcp.md](docs/mcp.md).
  72. - **QField survey companion** — generate a QField project from the active twin,
  73. collect trails / stream centerlines / photo points / observations in the
  74. field, then upload the zipped project folder back through the viewer. Uploads
  75. become journaled `survey_*` entities, rendered as survey layers and queryable
  76. through MCP. See [docs/survey.md](docs/survey.md).
  77. - **Live telemetry** — register Meshtastic/LoRa gateways from the viewer,
  78. render live tracker positions, replay recorded days with realtime/speed
  79. controls and tracker POV, then append selected samples into the curated twin
  80. store. See [docs/live-inputs.md](docs/live-inputs.md).
  81. - **Plan** — a smooth, interactive 3D workspace with live terrain/vegetation
  82. feedback for tree/shrub removal, free-text species-aware planting, and
  83. depressions/mounds. Hold Ctrl/Command to navigate without leaving a brush;
  84. G.A.I.A. composes swales, orchards, and gardens from the same primitives.
  85. Plans autosave as immutable revisions, support named versions and branches,
  86. can be safely discarded, and rerun hydrology, wildfire, ET, solar, and
  87. viewshed against the effective land. G.A.I.A. can draft and visualize proposals
  88. before an explicitly confirmed apply. See [docs/plan.md](docs/plan.md).
  89. ## Plan land alternatives
  90. Plan is a non-destructive workspace for trying changes to the land before making
  91. decisions in the field. The baseline twin never changes: each completed brush
  92. gesture becomes an immutable plan revision, and every simulation result stays
  93. scoped to the exact plan and revision that produced it.
  94. ![VEIL Plan workspace showing the Flatirons demo, live Difference view, terrain and vegetation brushes, version controls, and change summary](docs/img/plan-workspace-flatirons.png)
  95. ### Use Plan by hand
  96. 1. Open **Plan** from the left rail, then click **New** or select an existing
  97. plan. Choose **Baseline**, **Planned**, or **Difference** at any time to
  98. compare the original land with the active alternative.
  99. 2. Select **Remove plants**, **Add tree**, **Add bush**, **Depression**, or
  100. **Mound**. Set brush radius and, where relevant, species, modeled stage,
  101. spacing, and earth depth/height. Click once to place a plant; drag to paint
  102. plantings or earthworks. The 3D terrain and vegetation update while you draw.
  103. 3. Hold **Ctrl** (or **Command** on macOS) while dragging to move the camera
  104. without leaving the brush. **Esc** cancels an unfinished gesture;
  105. **Ctrl/Command+Z** undoes the last completed one.
  106. 4. Watch the save indicator and change summary. Finished gestures autosave as
  107. immutable revisions, so **Save version** is for naming a checkpoint—not for
  108. protecting ordinary work. Select an older version to inspect or simulate it;
  109. use **Branch** to turn that read-only version into a new editable alternative.
  110. **Discard** archives the whole plan without deleting its revision history.
  111. 5. Open **Simulation** and run hydrology, wildfire, Water & ET, solar, or
  112. viewshed normally. With a plan selected, VEIL automatically runs against that
  113. revision's effective terrain and vegetation instead of the baseline.
  114. ### Plan with G.A.I.A.
  115. 1. Open **Ask G.A.I.A.** and describe the outcome and location—for example,
  116. “Draft a swale along the main flow path with a capture basin; show me the
  117. difference, but do not apply it.” Use **Draw region** or **Pick point** first
  118. when the request should be spatially constrained.
  119. 2. G.A.I.A. creates or extends a plan, validates its proposed edits, estimates
  120. quantities, and opens the proposal in the already-running viewer's
  121. **Difference** view. This is a preview: it does not alter the plan head or the
  122. baseline.
  123. 3. Inspect the live terrain, switch among **Baseline**, **Planned**, and
  124. **Difference**, and read the proposal quantities. Ask G.A.I.A. to revise the
  125. draft if needed. When it is right, click **Approve as new revision** (or give
  126. an equally explicit approval in chat). Approval creates one immutable
  127. revision; stale proposals are rejected rather than overwriting newer work.
  128. 4. Ask G.A.I.A. to run a plan simulation, or use the Simulation pane yourself,
  129. then compare results across saved versions or branches.
  130. ![Ask G.A.I.A. showing an unapplied Flatirons land-plan proposal, screening quantities, the live Difference overlay, and the explicit Approve as new revision action](docs/img/plan-gaia-proposal-flatirons.png)
  131. The detailed edit model, simulation coupling, REST endpoints, MCP tools, and
  132. current limits are documented in [docs/plan.md](docs/plan.md).
  133. ## Simulation
  134. Mapping shows what's *on* the land; **simulation** models the processes that move
  135. *across* it. Mazzap runs these inside the viewer's collapsible **Simulation**
  136. window, each rendered as draped, clickable layers that conform to the topography —
  137. and each is explicit about what it can and cannot tell you.
  138. ### Hydrology
  139. Mazzap models where water moves and pools across the land, in two tiers. **Tier 1**
  140. (`analyze_hydrology.py`) works the bare LiDAR/DEM surface: priority-flood depression
  141. filling, D8 flow direction with O(n) flow accumulation, Horn slope, and a
  142. Topographic Wetness Index. It produces draped, clickable layers for flow paths
  143. (upslope contributing area, ha), wetness index (TWI percentile), ponding depth in
  144. closed depressions (m), and spring/seep candidates (wetness × flow convergence ×
  145. slope-break × shallow restrictive layer from SSURGO soils). **Tier 2**
  146. (`hydro_scenario.py`) runs event scenarios — snowmelt (inches of SWE over N melt
  147. days, optional rain-on-snow, antecedent moisture, frozen ground) or a rainstorm
  148. (depth over a duration) — partitioning each soil cell into runoff versus
  149. infiltration via the SCS Curve-Number method (SSURGO hydrologic soil groups, AMC
  150. I/II/III), then routing down the Tier-1 D8 graph for per-cell runoff, routed
  151. event-flow volumes, and a peak-discharge estimate at the AOI outlet. Honest
  152. framing: the geometry — *where* water concentrates — is the reliable output;
  153. discharge magnitude carries a ±50% class band because the catchment is ungauged.
  154. ![Mazzap hydrology simulation: topographic wetness index and flow paths draped on the terrain of the Flatirons demo VEIL](docs/img/sim-hydrology.png)
  155. **Use cases:**
  156. - **Agriculture** — locate wet-index hollows and ponding cells that drown crops or bog down equipment, to guide tile-drainage and field-traffic planning.
  157. - **Ecology** — map spring/seep candidates and convergent flow paths that mark seasonal wetland and riparian habitat across the land.
  158. - **Conservation** — target erosion-control and buffer plantings along the routed high-runoff channels a modeled storm concentrates.
  159. ### Wildfire
  160. Mazzap runs fire-behavior scenarios on top of the LANDFIRE fuel layers a US VEIL
  161. already carries — surface fuel models (FBFM13/40), canopy cover, height, base
  162. height, and bulk density. Combined with the land's terrain (slope, aspect), local
  163. hydrology influence, an ignition point, and a chosen weather/wind scenario, it
  164. models arrival time, flame length, fireline intensity, ember exposure, and
  165. crown-fire class across the land, reusing the same Simulation-window and
  166. draped-layer machinery as the hydrology scenarios. Treat the result as
  167. scenario-grade exploration, not an operational forecast: where fire spreads is
  168. more reliable than exact times, flame lengths, or intensities.
  169. ![Mazzap wildfire simulation: modeled fire arrival footprint draped on the terrain of the Sawtooth demo VEIL](docs/img/sim-wildfire.png)
  170. **Use cases:**
  171. - **Agriculture** — compare wind scenarios to see where crown fire could reach a woodlot or shelterbelt bordering cropland, informing defensible-space clearing.
  172. - **Ecology** — explore how fuel and canopy structure shape modeled fire intensity across a habitat mosaic.
  173. - **Conservation** — rehearse where a prescribed burn might spread or spot before committing to a burn plan.
  174. ### Evapotranspiration & water balance
  175. Mazzap simulates how much water the land actually loses to the atmosphere. From the
  176. VEIL's Daymet climate forcing it derives daily reference ET (ET0) with an ensemble
  177. of standard methods — FAO-56 Penman-Monteith (reduced-data form), Priestley-Taylor,
  178. Hargreaves-Samani, and Oudin (`derive_et0_daily.py`) — and reports the spread
  179. between methods as an explicit uncertainty band rather than a single false-precise
  180. number. A FAO-56 root-zone soil-water-balance (`et_water_balance.py`) then converts
  181. that reference demand into **actual** ET (AET) using basal crop coefficients from
  182. land cover and canopy, a water-stress coefficient from SSURGO available-water
  183. capacity, canopy interception, and a self-contained temperature-index snow model
  184. (robust where gridded SWE is unreliable). This closes the land's water balance —
  185. P = ET + runoff + Δstorage + recharge — the loss term the hydrology model previously
  186. left open, and feeds antecedent soil moisture back into the runoff scenarios. Honest
  187. framing: ET0 is FAO-56-correct, but absolute annual AET is only ±20–35% without
  188. local flux-tower or gauge validation; relative timing, seasonality, and wet/dry
  189. antecedent state are far more trustworthy than the absolute totals. Queryable over
  190. MCP via `et_summary`, `et_at`, and `water_balance`.
  191. ![Mazzap evapotranspiration simulation: modeled annual actual ET draped across the Flatirons demo VEIL](docs/img/sim-evapotranspiration.png)
  192. **Use cases:**
  193. - **Agriculture** — read the modeled crop-coefficient AET and soil-water-stress signal to see when and where the root zone draws down, flagging the seasonal windows most exposed to irrigation deficit.
  194. - **Ecology** — use the daily AET and antecedent soil-moisture series to characterize drought stress and growing-season water availability across the VEIL's vegetation communities.
  195. - **Conservation** — combine the closed water balance's runoff and recharge terms with wet/dry antecedent state to compare how land-cover scenarios shift where water leaves versus infiltrates.
  196. ### Solar siting
  197. Mazzap plans fixed solar-panel sites inside the Simulation window. It combines
  198. the local astronomy model, terrain/canopy horizons from viewshed, the VEIL
  199. vegetation inventory, and Daymet shortwave climate normals when available to
  200. estimate annual panel radiation, winter panel radiation, PV yield per installed
  201. kWdc, shade loss, cloud loss, and whether the assumed panel footprint is clear
  202. of tree/shrub crowns. The Solar sidebar shows the three best as-is
  203. vegetation-aware sites and the three best bare-earth/cleared-potential sites
  204. side by side. Pick any point to ask what tilt and azimuth a panel should use
  205. there, or let G.A.I.A. rank candidate sites and draw them on the live map.
  206. Honest framing: this is a planning-grade PVWatts-style estimate for strategy and
  207. comparison, not a bankable production report; validate promising sites with
  208. NSRDB/PVGIS or measurements before spending money. See
  209. [docs/solar.md](docs/solar.md).
  210. ![Mazzap solar siting simulation: annual PV yield heatmap and a picked panel-site recommendation over the Flatirons demo VEIL](docs/img/solar-flatirons.png)
  211. ## Astronomy
  212. Mazzap's Astronomy pane runs a local ephemeris in the browser and in MCP. It
  213. scrubs the VEIL clock, renders physical sky and lighting, overlays sun, moon,
  214. planets, stars, and constellations, and answers site-specific sky questions such
  215. as rise/set, eclipses, supermoons, alignments, solstice/equinox times, and golden
  216. hour. The screenshot below uses the bundled Flatirons demo VEIL.
  217. ![Mazzap astronomy simulation: physical night sky, stars, constellations, moon and planet markers over the Flatirons demo VEIL](docs/img/astronomy-flatirons.png)
  218. ## Viewshed & distant terrain
  219. Viewshed analysis now lives in the Simulation window alongside hydrology, ET, and
  220. wildfire. Pick an observer, choose eye level or tower height, select bare-earth
  221. or canopy blockers, and VEIL computes the visible area, horizon, farthest visible
  222. terrain, and sky-open azimuths. For US twins, `fetch_distant_terrain.py` builds
  223. real USGS 3DEP rings and NAIPPlus imagery around the twin so distant ridges and
  224. valleys participate in the analysis instead of being a painted backdrop. The
  225. screenshot below is the bundled Flatirons demo VEIL with generated 3DEP/NAIP
  226. distant terrain.
  227. ![Mazzap viewshed simulation: a tower-height observer, computed visible area, and 3DEP distant terrain around the Flatirons demo VEIL](docs/img/viewshed-flatirons.png)
  228. ## Run with Docker
  229. The static server has **zero npm dependencies**, but the data pipeline needs
  230. Python with GDAL, numpy, pyproj, and Pillow — fiddly to assemble by hand because
  231. GDAL's Python (`osgeo`) bindings must match the system GDAL. The container does
  232. it for you: one image with Node and the whole Python pipeline, all versions
  233. pinned (GDAL via `GDAL_VERSION`, the pip deps in `requirements.txt`, Node via
  234. `NODE_MAJOR`). It's built on the official [OSGeo GDAL](https://github.com/OSGeo/gdal)
  235. image, so GDAL + bindings + numpy already line up.
  236. ```bash
  237. docker compose up --build # build + serve ./data at http://127.0.0.1:4173
  238. docker compose --profile live-hardware up veil-live --build
  239. # serve with Bluetooth/USB serial access for Meshtastic
  240. # the same image runs every pipeline step — twins persist to ./twins on the host:
  241. docker compose run --rm veil npm run demo # build the Flatirons demo twin
  242. TWIN_DATA_DIR=/app/twins/demo/data docker compose up # serve it
  243. # build your own twin (see "Build your own twin" below for the commands):
  244. docker compose run --rm -e TWIN_DATA_DIR=/app/twins/mine/data \
  245. veil npm run build-from-aoi -- --aoi /app/twins/mine/my_area.shp --name "My Place"
  246. ```
  247. `./data` and `./twins` are bind-mounted, so anything the pipeline builds inside
  248. the container lands on the host and stays private/gitignored. Those files are
  249. written as `root` by default; to own them as your host user, prefix the commands
  250. with `VEIL_UID=$(id -u) VEIL_GID=$(id -g)`. The image includes
  251. the live Meshtastic bridge dependencies in `/opt/veil-live`; the
  252. `live-hardware` profile additionally mounts host D-Bus and `/dev` with elevated
  253. device access so Bluetooth and USB serial gateways can be discovered. Use the
  254. plain `veil` service when you only need the viewer, data pipeline, chat, or
  255. network/TCP telemetry. PDAL is optional and is not installed by this image; if
  256. the `pdal` command is absent, LiDAR-derived DSM/DTM generation is skipped as
  257. described in `requirements.txt`. For the **G.A.I.A.** chat panel, pass
  258. `OPENAI_API_KEY` (env var or a `.env` file) — see the chat section below. For an
  259. editable dev environment with the same pinned toolchain, open the repo in a
  260. **Dev Container** (`.devcontainer/`) in VS Code or any devcontainer-aware
  261. editor.
  262. Prefer a local toolchain? Everything below works the same with `npm` and
  263. `python3` directly.
  264. ## Hosted mode
  265. For a public website where each visitor gets an isolated session twin and brings
  266. their own OpenAI key, run with `VEIL_HOSTED=1` and
  267. `OPENAI_REQUIRE_USER_KEY=1`. Hosted mode scopes `/data`, chat/MCP tools,
  268. annotations, simulations, and sidebar geospatial uploads to a signed browser
  269. session. Setup details and production caveats are in [docs/hosted.md](docs/hosted.md).
  270. ## Build your own twin
  271. **The repo ships only code** (plus one ~600-byte demo AOI). You either fetch
  272. national data live for your area, or bring your own files. One checkout hosts
  273. many twins — point `--data-dir` / `TWIN_DATA_DIR` at a folder and your other
  274. twins are untouched. Full walkthrough: [docs/make-a-twin.md](docs/make-a-twin.md).
  275. ### Quick start
  276. Three ways in, easiest first — each writes a self-contained twin you then view
  277. with `npm start`:
  278. ```bash
  279. # 1. From a map, in the browser: draw an area, press "Set area & scan layers".
  280. npm run init # -> http://127.0.0.1:4173/init.html
  281. # 2. From an AOI polygon (US): one command fetches terrain, imagery,
  282. # vegetation, and soils for that footprint.
  283. npm run build-from-aoi -- --aoi my_area.shp --data-dir ./twins/mine/data --name "My Place"
  284. # 3. Anywhere on Earth, from global open data (see "Build a twin anywhere" below):
  285. python3 packs/nato/fetch_nato.py --country FR --aoi 2.6,48.4,2.7,48.5 \
  286. --aoi-crs EPSG:4326 --data-dir ./twins/mine/data --name "My Place"
  287. # then view it:
  288. TWIN_DATA_DIR=./twins/mine/data npm start # -> http://127.0.0.1:4173
  289. ```
  290. Everything below is the detail behind those three paths.
  291. **First-time setup from a map** — launch the initiation page, optionally search
  292. for a U.S. street address to jump the map to that location, draw an AOI on the
  293. basemap, then press **Set area & scan layers**:
  294. ```bash
  295. npm run init
  296. ```
  297. By default this builds into `./twins/init/data` and opens
  298. `http://127.0.0.1:4173/init.html`. Set `TWIN_DATA_DIR` first to build somewhere
  299. else:
  300. ```bash
  301. TWIN_DATA_DIR=./twins/mine/data npm run init
  302. ```
  303. The init page writes the drawn polygon to `<data>/init/aoi.geojson`, then runs
  304. the same national AOI builder as the CLI path below. The nationally available
  305. layers currently implemented in this repo are 3DEP elevation, 3DEP LiDAR Point
  306. Cloud tiles where USGS has coverage, NAIP Plus orthoimagery, LANDFIRE EVT, and
  307. additional LANDFIRE forest/fire ecology rasters, and gSSURGO soils, plus
  308. `us-national` vegetation typing from LANDFIRE. LiDAR, the extra LANDFIRE rasters,
  309. and gSSURGO are best-effort: if a source is unavailable, the twin still builds
  310. from the remaining national sources. When the build finishes, open the viewer
  311. link shown on the setup page.
  312. **US, fetch everything from one AOI** — hand it a small AOI polygon (shapefile,
  313. GeoJSON, GeoPackage…) and it queries 3DEP elevation, 3DEP LiDAR Point Cloud
  314. tiles where available, NAIP Plus orthoimagery, LANDFIRE land cover/vegetation,
  315. additional LANDFIRE forest/fire ecology rasters, and gSSURGO soils for that
  316. footprint and builds a complete, typed twin:
  317. ```bash
  318. npm run build-from-aoi -- --aoi my_area.shp --data-dir ./twins/mine/data --name "My Place"
  319. TWIN_DATA_DIR=./twins/mine/data PORT=4174 npm start # -> http://127.0.0.1:4174
  320. # the bundled demo is exactly this, from one committed shapefile:
  321. npm run demo && npm run serve-demo
  322. ```
  323. **Build a twin anywhere (global open data)** — the paths above are US-only. The
  324. bundled `nato` pack (`packs/nato/`) extends Mazzap to **any of the 32 NATO member
  325. nations**, assembling a twin from worldwide open datasets with no national
  326. account or portal needed. Give it an ISO country code and a lon/lat AOI:
  327. ```bash
  328. python3 packs/nato/fetch_nato.py --country FR --aoi 2.6,48.4,2.7,48.5 \
  329. --aoi-crs EPSG:4326 --data-dir ./twins/mine/data --name "My Place"
  330. TWIN_DATA_DIR=./twins/mine/data npm start
  331. ```
  332. Coverage is **tiered, and the pack is honest about which tier you get**. 14
  333. members ship a national adapter, but the tier they deliver varies. Seven pull
  334. their own higher-resolution national terrain today (Tier A) — **BE, CZ, ES, FR,
  335. LU, NL, NO**; the other seven adapters (**DK, EE, FI, LV, PL, SE, SK**) are
  336. wired up and probe their national services, but currently fall back to the
  337. **global 30 m stack** where an anonymous/tokenless high-res route wasn't
  338. reachable (per-country specifics in [`packs/nato/README.md`](packs/nato/README.md)).
  339. Every remaining member also resolves through that global stack, which works for
  340. any land AOI on Earth. So every member resolves to a working fetch path — but
  341. *builds a twin* is not the same as *builds a great twin*: where a member relies
  342. on the global fallback, the terrain (30 m) and the canopy are **modeled, not
  343. surveyed**, and the quality of the free aerial imagery varies by location and date (see the
  344. note below). The US is served by its own richer `us-national` pack (3DEP LiDAR +
  345. NAIP + LANDFIRE), not this one. What the pack pulls, all as clickable atlas
  346. layers:
  347. - **Terrain** — Copernicus GLO-30, or national LiDAR (e.g. Netherlands AHN,
  348. Spain PNOA) where a country publishes it.
  349. - **Canopy height → trees** — **Meta / WRI Global 1 m Canopy Height** (Tolan et
  350. al. 2024, CC-BY 4.0), falling back to ETH Global Canopy Height (10 m). Trees
  351. are detected as local maxima of this canopy surface.
  352. - **Soil** — ISRIC SoilGrids 250 m: pH, organic carbon, clay %, sand %.
  353. - **Hydrology** — HydroRIVERS + HydroLAKES (WWF) and JRC Global Surface Water.
  354. Lakes and rivers render as real **water surfaces**, not just flat overlays.
  355. - **Species** — GBIF occurrence density and a GBIF **species-richness** grid (an
  356. open-data analogue of a GAP richness layer), plus EEA Article 17
  357. protected-species distributions across the EU.
  358. - **Land cover / leaf type** — Copernicus HRL Dominant Leaf Type and CLC+ across
  359. Europe, CGLS-LC100 globally, used to type vegetation and mask the canopy to
  360. forest.
  361. > **The Meta canopy fallback is imperfect — treat it as modeled, not measured.**
  362. > Meta/WRI 1 m canopy height is a *model prediction from satellite imagery*, not
  363. > a survey: ~2.8 m mean absolute error, it saturates (under-estimates) tall
  364. > canopy above ~25–30 m, and carries a US/NEON training-domain bias. It gives
  365. > realistic per-tree structure where no LiDAR exists, but a twin built on a
  366. > country's real national LiDAR is materially more accurate. Likewise, where only
  367. > a hazy or low-sun free Sentinel-2 scene is available for an area, the draped
  368. > aerial imagery can come out dark or tinted. See `packs/nato/README.md` for the
  369. > per-country data tiers and full source list.
  370. During `npm run init`, Mazzap also probes an optional national-layer catalog
  371. against the drawn AOI before the build starts. The setup dialog lists only
  372. interactive sources that report an AOI intersection, explains what each layer is
  373. useful for, and passes the checked layer IDs into the build. Those selected
  374. layers are fetched after terrain/georeferencing exists and are registered as
  375. ordinary clickable atlas layers. The catalog includes live fetchers for sources
  376. such as NWI, NHDPlus/WBD, FEMA NFHL, USFWS critical habitat, EPA ecoregions, BLM
  377. surface management and grazing allotments, USFS/NPS boundaries/access layers,
  378. current/historic WFIGS fire perimeters, NCED easements, National Inventory of
  379. Dams, USDA CDL, NRCS MLRA/LRR, PAD-US, current U.S. Drought Monitor polygons,
  380. USDA plant hardiness zones, USFS Wildfire Hazard Potential, and LCMS landscape
  381. change layers, plus selected USGS RCMAP WCS rasters. Some sources use direct
  382. archive downloads instead of service-side AOI clipping: Mazzap checks the
  383. advertised file size, requires 20 GB of local free headroom, refuses sources
  384. over 1 GB, downloads the source archive, clips it locally, and discards the raw
  385. archive. Direct downloads are tagged in the setup menu as light under 100 MB,
  386. medium from 100 MB to 1 GB, and heavy at 1 GB or more. Moderate downloads such
  387. as PAD-US are offered as normal guarded downloads, not manual work.
  388. Multi-purpose sources such as PAD-US, LCMS, and RCMAP show child choices in the
  389. menu with their own explanations, so users can choose the interpretation they
  390. need without unnecessary duplicate downloads. The menu keeps a live total of
  391. selected download bytes and the estimated clipped output size for sources that
  392. can report it, including GAP species masks.
  393. The catalog also records self-service download sources such as NLCD, GAP species
  394. habitat species beyond the starter menu, TreeMap, PRISM/gridMET, POLARIS/SOLUS,
  395. MTBS/RAVG, and others. Those are not one-click AOI services yet, but the catalog
  396. includes official download links and notes so users can fetch a product, drop it
  397. into `manual_layers/`, and let Mazzap clip/register it locally.
  398. CLI equivalents:
  399. ```bash
  400. npm run --silent fetch-national-layers -- catalog
  401. npm run --silent fetch-national-layers -- probe --aoi my_area.geojson
  402. npm run --silent fetch-national-layers -- fetch --aoi my_area.geojson \
  403. --data-dir ./twins/mine/data --layers nwi_wetlands,nhdplus_flowlines
  404. ```
  405. Show the full national catalog, including download/manual links:
  406. ```bash
  407. npm run --silent fetch-national-layers -- catalog
  408. ```
  409. Check the direct-download endpoints and local disk headroom without downloading:
  410. ```bash
  411. npm run --silent fetch-national-layers -- check-downloads
  412. ```
  413. **Bring your own data** — anywhere, any format. Drop files in and ingest them:
  414. ```bash
  415. # terrain + georeferencing from any DEM (the CRS is the DEM's, or the AOI's UTM zone)
  416. npm run ingest-dem -- mydem.tif --aoi boundary.geojson --data-dir ./twins/mine/data
  417. # aerial imagery, aligned to the terrain footprint
  418. npm run ingest-imagery -- myaerial.tif --data-dir ./twins/mine/data
  419. # optional LAS/LAZ display derivative (auto-discovers <data>/lidar/raw by default)
  420. npm run build-pointcloud -- survey.laz --data-dir ./twins/mine/data
  421. # any vector/raster layer, any format, any CRS
  422. npm run add-layer -- soils.shp --id soils --label "Soils" --data-dir ./twins/mine/data
  423. npm run add-layer -- wetlands.gpkg --id wetlands --layer NWI --data-dir ./twins/mine/data
  424. npm run add-layer -- landcover.tif --id landcover --data-dir ./twins/mine/data
  425. ```
  426. The point-cloud build keeps the normal textured DEM/tree mesh as a separate
  427. viewer representation. In **LiDAR points**, high-level presets show ground,
  428. ground + vegetation, vegetation, or every return; individual point classes
  429. remain available under the advanced disclosure. The default classifier
  430. preserves authoritative source classes, runs PDAL SMRF at full cropped density,
  431. and only recovers generic source returns as ground when they also agree with
  432. the twin DEM. The optional `npm run classify-pointcloud` stage fuses open
  433. FRACTAL/Myria3D RGB+NIR semantics for vegetation and other above-ground classes,
  434. with entropy gating and a deterministic fallback; see
  435. [point-cloud classification](docs/pointcloud-classification.md) for the pinned
  436. model, local proxy benchmark, licenses, and reproducible commands. Use
  437. `--ground-classifier source` to disable the open ground-recovery pass.
  438. `add_layer` accepts anything GDAL/OGR can read — GeoJSON, Shapefile (`.shp`),
  439. GeoPackage (`.gpkg`), KML/KMZ, GPX, CSV with coordinates, File Geodatabase,
  440. GeoTIFF and other rasters — in any coordinate system. Multi-layer sources take a
  441. `--layer NAME` selector. National raster layers such as LANDFIRE fetch straight
  442. into a twin and register themselves as draped, clickable atlas layers. 3DEP LPC
  443. LiDAR is handled separately by `scripts/fetch_lidar.py`, which queries TNMAccess
  444. for LAZ tiles and derives `terrain/dsm.tif` + `terrain/dtm.tif` for canopy-height
  445. vegetation detection; it does not become an atlas drape. The extra LANDFIRE
  446. forest/fire layers are handled by `scripts/fetch_forest_ecology.py` and include
  447. existing vegetation cover/height, biophysical settings, succession class,
  448. vegetation condition/departure, fire behavior fuel models, canopy cover/height,
  449. canopy base height, canopy bulk density, fuel disturbance, and fire regime group.
  450. ### Downloadable and Manual Layers
  451. Many public products are available, but not as small AOI clipping services.
  452. For those, use the drop-directory workflow:
  453. 1. Download the source product you need. Prefer one AOI, state, year, metric,
  454. species, or fire event instead of whole national archives when possible.
  455. 2. Put the downloaded file or extracted dataset in `manual_layers/`.
  456. 3. Ingest everything GDAL/OGR can read:
  457. ```bash
  458. npm run ingest-manual-layers -- --data-dir ./twins/mine/data
  459. ```
  460. 4. If a GeoPackage, FileGDB, or zip has multiple vector layers, ingest each layer
  461. with derived ids:
  462. ```bash
  463. npm run ingest-manual-layers -- --data-dir ./twins/mine/data --all-layers
  464. ```
  465. Supported inputs include GeoTIFF, GeoJSON, Shapefile, GeoPackage, KML/KMZ, GPX,
  466. CSV with coordinates, File Geodatabase directories, and many zip archives. Mazzap
  467. clips each layer to the twin terrain footprint, reprojects it, styles it, and
  468. registers it in the viewer.
  469. Common download/manual sources:
  470. | Layer | Link | Note |
  471. | --- | --- | --- |
  472. | NLCD / Annual NLCD | <https://www.mrlc.gov/data> | Pick product and year. |
  473. | PAD-US | <https://www.usgs.gov/programs/gap-analysis-project/science/pad-us-data-download> | Direct download path offers protection status, public access, manager/owner, and designation type choices. |
  474. | GAP species habitat | <https://www.usgs.gov/programs/gap-analysis-project/science/species-data-download> | Starter species are automated as selectable child downloads; additional species can be added from ScienceBase metadata. |
  475. | USFS TreeMap | <https://data.fs.usda.gov/geodata/rastergateway/treemap/index.php> | Pick year and attribute raster. |
  476. | USFS LCMS | <https://data.fs.usda.gov/geodata/rastergateway/LCMS/index.php> | Fetched automatically for default land-cover/change/loss/gain views. |
  477. | USFS FIA DataMart | <https://apps.fs.usda.gov/fia/datamart/> | Tabular/statistical; not a direct atlas layer yet. |
  478. | USFS Aerial Detection Survey | <https://www.fs.usda.gov/science-technology/data-tools-products/fhp-mapping-reporting/detection-surveys> | Download region/year geospatial data. |
  479. | Wildfire Hazard Potential | <https://research.fs.usda.gov/firelab/products/dataandtools/wildfire-hazard-potential> | Fetched automatically from the 2023 classified image service. |
  480. | MTBS burn severity | <https://www.mtbs.gov/direct-download> | Pick fire/state/national products. |
  481. | RAVG post-fire condition | <https://data.fs.usda.gov/geodata/rastergateway/ravg/index.php> | Pick fire and product raster. |
  482. | gNATSGO | <https://www.nrcs.usda.gov/resources/data-and-reports/gridded-national-soil-survey-geographic-database-gnatsgo> | Bulk soils source; base builds already fetch SDA/gSSURGO. |
  483. | POLARIS soils | <http://hydrology.cee.duke.edu/POLARIS/> | Pick property, depth, and statistic raster. |
  484. | SOLUS100 soils | <https://www.nrcs.usda.gov/resources/data-and-reports/soil-landscapes-of-the-united-states-solus> | Pick property/depth raster. |
  485. | NRCS MLRA/LRR | <https://www.nrcs.usda.gov/resources/data-and-reports/major-land-resource-area-mlra> | Fetched automatically from the hosted feature service; downloadable database also available. |
  486. | RAP rangelands | <https://rangelands.app/> | Export AOI/year/metric outputs. |
  487. | USGS RCMAP | <https://www.mrlc.gov/data/type/rcmap-time-series-trends> | Fetched automatically from MRLC WCS for selected 2025 component views. |
  488. | LANID irrigation | <https://zenodo.org/records/5548555> | Pick annual irrigation or frequency raster. |
  489. | MIrAD-US irrigation | <https://data.usgs.gov/datacatalog/data/USGS:5db08e84e4b0b0c58b56e04f> | Pick year/resolution raster. |
  490. | OpenET | <https://etdata.org/api/> | API/date/product workflow; export before ingest. |
  491. | PRISM | <https://prism.oregonstate.edu/> | Pick variable and period. |
  492. | gridMET | <https://www.climatologylab.org/gridmet.html> | Pick variable/year or AOI subset. |
  493. | SNODAS | <https://nsidc.org/data/g02158/versions/1> | Convert flat-binary products to GeoTIFF first. |
  494. | U.S. Drought Monitor | <https://droughtmonitor.unl.edu/DmData/GISData.aspx> | Direct download path fetches the current weekly shapefile automatically. |
  495. | USGS geology / NGMDB | <https://ngmdb.usgs.gov/mapview/> | Pick map/product and download geospatial files if offered. |
  496. | EPA ATTAINS | <https://www.epa.gov/waterdata/get-data-access-public-attains-data> | API/table/geospatial service; adapter still needed. |
  497. | Water Quality Portal | <https://www.waterqualitydata.us/> | Download stations/results; rich observation adapter still needed. |
  498. | NOAA C-CAP | <https://coast.noaa.gov/digitalcoast/data/ccapregional.html> | Pick regional/high-resolution product and year. |
  499. | NOAA Essential Fish Habitat | <https://www.habitat.noaa.gov/application/efhinventory/> | Pick council/species/life-stage GIS data. |
  500. | USDA plant hardiness zones | <https://prism.oregonstate.edu/phzm/> | Direct download path fetches the 2023 CONUS shapefile automatically. |
  501. | NatureServe biodiversity | <https://www.natureserve.org/access-data> | Download open GIS layers. |
  502. | TNC Resilient and Connected Network | <https://crcs.tnc.org/pages/data-terrestrial-resilience> | Download state/network component data. |
  503. The terrain grid that `ingest-dem` writes conforms to a frozen interface — see
  504. [docs/grid-contract.md](docs/grid-contract.md). It is the one contract every
  505. terrain consumer depends on; don't change it silently.
  506. ## QField survey companion
  507. Mazzap has a built-in field loop for QField: the app generates the survey package
  508. from the current twin, QField records edits against that package, and the viewer
  509. uploads the finished package back into the twin store.
  510. Build a package for the twin you are serving:
  511. ```bash
  512. npm run build-survey-package -- --data-dir ./twins/mine/data --name "My Place"
  513. TWIN_DATA_DIR=./twins/mine/data npm start
  514. ```
  515. Then open the viewer's **Survey companion** panel and download
  516. `survey-package.zip`. Unzip or sideload the `survey/` project folder into
  517. QField and open `project.qgs`. The generated package contains:
  518. - `survey.gpkg` with four editable layers: `trails`, `stream_centerlines`,
  519. `photo_points`, and `observations`.
  520. - `project.qgs`, with forms wired for stable UUIDs, active / retired / removed
  521. status, capture time, GPS accuracy, notes, and camera attachments.
  522. - `basemap.tif` when the twin has georeferenced imagery.
  523. After fieldwork, zip the whole QField project folder, including its `DCIM/`
  524. photos, and upload it from the same **Survey companion** panel. The server saves
  525. the raw zip under `<data>/surveys/incoming/`, logs it in
  526. `<data>/surveys/uploads.log.jsonl`, runs `scripts/ingest_survey.py --pending`,
  527. and refreshes the viewer's survey layers. If Python ingest is unavailable, the
  528. upload is kept and `npm run export` will process pending uploads later.
  529. Survey features are stored as ordinary twin-store entities with stable IDs like
  530. `survey_trails:<uuid>`. Re-uploading the same project is safe: unchanged
  531. features are skipped, moved features keep identity, and retirement is explicit
  532. through the `status` field rather than inferred from a missing feature. To gate
  533. uploads on a shared LAN, create a gitignored `.survey_token`; the viewer prompts
  534. for the token and sends it as `X-Survey-Token`.
  535. Full details: [docs/survey.md](docs/survey.md).
  536. ## MCP server and app chat
  537. `scripts/mcp_server.py` is the MCP surface over the twin store. It exposes
  538. the same facts the viewer uses — terrain, entities, atlas layers, survey
  539. layers, provenance, and region summaries — as structured tools for LLM
  540. agents. Query tools are read-only. Map drawings and viewer directives only
  541. write the flat `annotations.json` that the viewer polls. Plan lifecycle tools
  542. intentionally journal non-destructive plan roots, branches, checkpoints, and
  543. confirmed revisions; plan simulations write revision-scoped results. None of
  544. these paths edits the baseline twin. The server reads the active twin from
  545. `TWIN_DATA_DIR` or `./data`.
  546. Install the Python side once:
  547. ```bash
  548. pip install -r requirements.txt
  549. TWIN_DATA_DIR=./twins/mine/data npm run rebuild-store # only if twin.gpkg is missing/stale
  550. TWIN_DATA_DIR=./twins/mine/data python3 scripts/twin_query.py describe_twin
  551. ```
  552. You do **not** start the MCP server separately for the browser app. Open **Ask
  553. G.A.I.A.**, and the Node server lazily spawns `scripts/mcp_server.py` on the
  554. first chat request. Provide an OpenAI key either way:
  555. ```bash
  556. # Bring-your-own-key (recommended, esp. for a shared/public twin): start with no
  557. # key, then click "Key" in the chat panel and paste yours. It is stored only in
  558. # your browser (localStorage) and sent per request as X-OpenAI-Key — never on the
  559. # server or in the repo.
  560. TWIN_DATA_DIR=./twins/mine/data npm start
  561. # Server-side key (single-user/local convenience): the server uses this for any
  562. # request that doesn't bring its own.
  563. OPENAI_API_KEY=sk-... TWIN_DATA_DIR=./twins/mine/data npm start
  564. # or put the key in a gitignored .openai_key file
  565. # Public deployment: forbid the server-key fallback so every request must BYOK.
  566. OPENAI_REQUIRE_USER_KEY=1 TWIN_DATA_DIR=./twins/mine/data npm start
  567. # Fully local, no key, nothing leaves the machine: point the chat at an Ollama
  568. # model. `npm run start:local` is the shortcut (CHAT_PROVIDER=ollama,
  569. # OLLAMA_MODEL=gpt-oss:20b) and also opens the viewer in an integrated-GPU Firefox
  570. # profile so CUDA VRAM stays free for the model; use `npm run start:local:server`
  571. # (or VEIL_OPEN_BROWSER=0) for server-only. CHAT_PROVIDER=ollama is also inferred
  572. # whenever OLLAMA_MODEL is set. Needs `ollama serve` running with a tool-calling
  573. # model pulled. gpt-oss:20b is a good 24 GB-GPU default (~16 GB on the GPU at the
  574. # 96k default context — its KV cache is cheap; even the full 131072 fits ~17 GB).
  575. TWIN_DATA_DIR=./twins/mine/data npm run start:local
  576. # tune with OLLAMA_HOST (default http://127.0.0.1:11434), OLLAMA_NUM_CTX (default
  577. # 98304), OLLAMA_TEMPERATURE (default 0), OLLAMA_MAX_TOOL_ROUNDS (default 24)
  578. ```
  579. In the app, questions can target:
  580. - **Whole land** — the default scope.
  581. - **Drawn region** — click **Draw region**, place 3+ terrain points, then ask
  582. about "this area".
  583. - **Picked point** — click **Pick point**, choose a terrain point, then ask
  584. about "here" with nearby context preloaded.
  585. The chat transcript shows the MCP tools the model called, so answers can be
  586. checked against store data and layer provenance.
  587. For an external MCP client, register the stdio server directly:
  588. ```bash
  589. claude mcp add veil-twin -- env TWIN_DATA_DIR=/ABS/PATH/TO/twins/mine/data \
  590. python3 /ABS/PATH/TO/veil/scripts/mcp_server.py
  591. ```
  592. Useful direct sanity checks:
  593. ```bash
  594. TWIN_DATA_DIR=./twins/mine/data python3 scripts/twin_query.py describe_twin
  595. TWIN_DATA_DIR=./twins/mine/data python3 scripts/twin_query.py identify_at '{"point":{"x":50,"y":100}}'
  596. npm test
  597. ```
  598. Core MCP tools include `describe_twin`, `find_entities`, `get_entity`,
  599. `entity_history`, `identify_at`, `sample_raster`, `list_layers`,
  600. `layer_summary`, `summarize_region`, `aggregate_entities`,
  601. `canopy_change`, and `list_survey_layers`; planning adds `list_plans`,
  602. `planning_catalog`, `create_plan`, `branch_plan`, `propose_plan_edits`, the
  603. semantic `propose_swale` / `propose_orchard` / `propose_garden` helpers,
  604. `apply_plan_proposal`, `visualize_plan`, and `run_plan_simulation`. The
  605. map-drawing trio remains
  606. `draw_polygon` / `draw_point` / `clear_drawings` — answers can point at places
  607. with labeled orange shapes in the viewer (built-in chat and external MCP
  608. clients alike; the chat panel's **Clear drawings** button removes them).
  609. Survey uploads appear automatically as `survey_*` kinds for `find_entities`,
  610. `aggregate_entities`, and `summarize_region`, are catalogued by
  611. `list_survey_layers`, and are now included in point `identify_at` (with photo
  612. and status). Full tool semantics and examples: [docs/mcp.md](docs/mcp.md).
  613. ## Architecture: engine vs. regional pack
  614. Mazzap is a **region-agnostic engine** (`scripts/`, `public/`, `server.js`) plus an
  615. optional **regional content pack** (`packs/<name>/`). It models and instantiates a
  616. VEIL from data; nothing in the engine names a CRS, a layer, or a species.
  617. - **Coordinates are data.** `<data>/georef.json` carries the projected CRS (EPSG +
  618. a proj4 string), the geographic CRS, and the scene origin. Python reads it
  619. through `scripts/twin_georef.py`; the viewer through vendored **proj4js**
  620. (`public/vendor/proj4.js`) in `public/viewer/georef.js`. Scene coordinates are
  621. local meters (x = east, y = north) offset from the origin; the store keeps the
  622. same convention.
  623. - **Packs** load via `scripts/twin_pack.py` (chosen by `TWIN_PACK`, else
  624. `<data>/pack.txt`, else none). A pack is a folder with `pack.json` and optional
  625. `load(context)` hook modules: `vegetation.py` (species/community/type knowledge)
  626. and `layers.py` (atlas styles, attribute enrichment, named raster renderings),
  627. plus its own source-acquisition scripts. Without a pack the engine auto-styles
  628. every layer and emits trees with `type:"unknown"` — it never guesses local
  629. botany.
  630. - **`packs/us-national`** is the region-agnostic exception: it encodes no single
  631. place, just CONUS-wide LANDFIRE EVT. Any US twin can use it
  632. (`TWIN_PACK=us-national`) to type vegetation evergreen/deciduous and name
  633. communities, after fetching LANDFIRE for the twin's footprint
  634. (`packs/us-national/fetch_landfire.py --data-dir <data>`). A single-region
  635. pack adds what no national dataset has: curated local species, your own atlas
  636. styling, regional attribute enrichment. National datasets
  637. (3DEP/NAIP/NLCD/LANDFIRE/GAP/gSSURGO) belong in `us-national`-style packs,
  638. distinct from both the engine core and any one region.
  639. ```
  640. server.js zero-dependency static server (+ /api/chat, /api/* )
  641. public/
  642. index.html app.js chat.js plan.js UI, boot, chat, land planning
  643. viewer/ scene.js terrain.js vegetation.js overlays.js buildings3d.js
  644. georef.js scene-local meters <-> lon/lat (proj4js)
  645. vendor/ three.min.js OrbitControls.js proj4.js
  646. scripts/ the region-agnostic engine
  647. twin_georef.py twin_pack.py read georef.json / load the active pack
  648. ingest_dem.py ingest_imagery.py genesis: DEM + imagery -> a twin
  649. build_pointcloud.py LAS/LAZ -> classified ortho-color display payload
  650. add_layer.py import any geospatial file as a layer
  651. analyze_vegetation.py veg_detect.py capability-gated vegetation
  652. build_viewer_layers.py generic atlas localization + styling
  653. twin_store.py migrate_to_store.py rebuild_store.py the twin store
  654. plan_engine.py plan_cli.py revisions, materialization, simulations
  655. mcp_server.py twin_query.py query + map-drawing tools for the chat/agent
  656. live/ live telemetry bridges + side store
  657. packs/<name>/ optional regional content pack
  658. pack.json vegetation.py layers.py knowledge + styling hooks
  659. *.py its own source-acquisition scripts
  660. data/ (or any --data-dir) one twin instance: georef, terrain,
  661. imagery, vegetation, atlas, store
  662. ```
  663. ## The twin store
  664. The system of record is the **twin store** — an append-only write journal in
  665. `<data>/journal/`, materialized as a GeoPackage at `<data>/twin.gpkg`. The
  666. journal lives inside the twin's data dir (private, gitignored along with the
  667. rest of it); `npm run rebuild-store` reconstructs the GeoPackage from it
  668. exactly. The flat JSON the viewer loads is an *export* of the store. Origin
  669. and CRS live in the store's `meta`; coordinates are scene-local meters. See
  670. `CLAUDE.md` and `docs/mcp.md` for the store model and the query/agent layer.
  671. ## Tests
  672. ```bash
  673. npm test # offline: the committed fixture twin
  674. npm run test:demo # real data: the Flatirons demo twin
  675. ```
  676. `npm test` runs `scripts/twin_query_test.py` against a tiny **committed fixture
  677. twin** (`tests/fixtures/mini-twin/data`) — a synthetic, network-free twin built
  678. by `scripts/build_test_fixture.py`. It needs no internet and no GDAL (just
  679. Python + `pyproj`; Node for the proj4js cross-check), so the full query suite
  680. runs offline anywhere — in CI or on a fresh clone. Every expectation is derived
  681. from the twin under test, never hardcoded to a place.
  682. `npm run test:demo` runs the same suite against the **Flatirons demo twin**
  683. (`twins/demo/data`), building it first from live national services if it isn't
  684. there (needs internet + GDAL once) — the real-data check. Point `TWIN_DATA_DIR`
  685. at any other twin to run the suite against it. Regenerate the fixture with
  686. `npm run build-test-fixture` after a store schema change, then commit it.
  687. ## Privacy
  688. A twin's `data/` (and any `--data-dir`) holds real coordinates, parcel/owner
  689. attributes, building footprints, and imagery for a specific place — it is **not**
  690. tracked by git in this repo (see `.gitignore`). Share the engine and your pack;
  691. keep your ground to yourself. If you previously committed a twin and want it gone
  692. from history (not just future commits), rewrite history with `git filter-repo` —
  693. or, safer, start a fresh repo from the current tree, since a rewritten history
  694. can still leave reachable objects in clones and forks.
  695. ## Security posture
  696. The server is meant for localhost or a trusted LAN/Tailscale bind, not the
  697. open internet:
  698. - **`POST /api/chat` spends on an OpenAI key.** Each request uses the caller's
  699. own key if the viewer supplies one (the chat panel's **"Key"** button stores it
  700. in the browser's `localStorage` and sends it as the `X-OpenAI-Key` header — it
  701. never touches the repo or the server's disk); otherwise it falls back to the
  702. server's `OPENAI_API_KEY` / `.openai_key`. That fallback is **unauthenticated
  703. spend**: anyone who can reach the port can call OpenAI on the server's key. For
  704. a public deployment set **`OPENAI_REQUIRE_USER_KEY=1`** so the server never uses
  705. its own key and every request must bring its own.
  706. - Survey uploads can be token-gated (a `.survey_token` file at the repo root
  707. enforces an `X-Survey-Token` header); without the file the route is open.
  708. - **The live-telemetry API (`/api/live/*`) is unauthenticated by default and can
  709. spawn local processes.** Unless you set a `.live_token` (or `VEIL_LIVE_TOKEN`),
  710. anyone who can reach the port can register/start Meshtastic bridge processes
  711. and trigger serial/Bluetooth device scans. It is disabled entirely in hosted
  712. mode; set a token before exposing the port on a LAN.
  713. - There is no TLS, no rate limiting, and no auth on the building-placement
  714. endpoint — by design, for a single-user posture.
  715. - State-changing routes require the request's `Origin`/`Referer` to match `Host`
  716. (blocking ordinary cross-site drive-by POSTs). That check alone does **not**
  717. stop DNS rebinding; on any exposed deployment set **`VEIL_ALLOWED_HOSTS`**
  718. (comma-separated hostnames) to pin the acceptable `Host` values.

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.