Skip to content
Back to search

ucam-eo

MarketplaceWorks with Claude CodeGitHub

Python library for the Tessera embeddings

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026. Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/ucam-eo/geotessera with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/ucam-eo/geotessera'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/ucam-eo--geotessera/badge.svg)](/listings/ucam-eo--geotessera)

HTML

<a href="/listings/ucam-eo--geotessera"><img src="/api/listings/ucam-eo--geotessera/badge.svg" alt="findsafeskills safety rating" /></a>

Install

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

GeoTessera requires Python 3.12 or later.

pip install geotessera

For development:

git clone https://github.com/ucam-eo/geotessera
cd geotessera
uv sync --all-extras --dev   # or: pip install -e ".[docs]"

Sphinx lives in the `docs` extra and the test harness in the `dev` group, so a plain `pip install geotessera` pulls in neither.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json · Manifest · 12 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. {
  2. "name": "ucam-eo",
  3. "owner": { "name": "Anil Madhavapeddy" },
  4. "plugins": [
  5. {
  6. "name": "geotessera",
  7. "source": "./",
  8. "description": "Teach coding agents to read Tessera satellite embeddings via the geotessera zarr interface",
  9. "version": "0.11.0"
  10. }
  11. ]
  12. }
README.md · README · 806 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

  1. # GeoTessera
  2. GeoTessera is a Python library for reading, exporting, and displaying Tessera embeddings.
  3. > ## 🚀 TESSERA v2 is here
  4. >
  5. > **The TESSERA v2 code is now live** — model weights and inference code are
  6. > available in the [`ucam-eo/tessera`](https://github.com/ucam-eo/tessera)
  7. > repository. v2 is our next-generation pixel-wise Earth foundation model; see
  8. > the preprint, [*TESSERA v2: Scaling Pixel-wise Earth Foundation Models*](https://arxiv.org/abs/2607.03949).
  9. >
  10. > **Want to try v2 embeddings early?** You can **pre-request** v2 embeddings for
  11. > your region and become an early tester:
  12. >
  13. > - 👉 **[Submit a v2 Embedding Pre-Request](../../issues/new?template=v2-embedding-prerequest.yml&labels=v2-embedding-prerequest)**
  14. >
  15. > ⚠️ **Heads-up:** we are still ramping up the compute, storage, and release
  16. > infrastructure for v2, so v2 embeddings will be produced **slowly** at first
  17. > and there is **no guaranteed turnaround time**. **If you need embeddings now,
  18. > use v1.1 instead** — [wall-to-wall global coverage](#global-coverage) for
  19. > 2017–2025 is available today.
  20. ## Overview
  21. GeoTessera provides access to geospatial embeddings from the [Tessera
  22. foundation model](https://github.com/ucam-eo/tessera), which processes
  23. Sentinel-1 and Sentinel-2 satellite imagery to generate 128-channel
  24. representation maps at 10m resolution. These embeddings compress a full year of
  25. temporal-spectral features into dense representations optimized for downstream
  26. geospatial analysis tasks. Read more details about [the model](https://github.com/ucam-eo/tessera).
  27. The [Zarr API](#cloud-native-zarr-access) reads selected points and regions
  28. from the cloud store and returns dequantized values on their native UTM
  29. grid. The [tile API](#python-api) downloads individual NPY or GeoTIFF files
  30. for offline use; NPY tiles are deprecated and will be removed.
  31. ### Stream regions to GeoTIFF or a web map
  32. `download` reads from Zarr by default and writes one GeoTIFF per UTM zone.
  33. `webmap` reads three embedding bands and creates an RGB web map.
  34. ```bash
  35. geotessera download --bbox '-3.0,53.4,-2.9,53.5' --year 2024 --output region/
  36. geotessera webmap --bbox '-3.0,53.4,-2.9,53.5' --year 2024 --output map/ --serve
  37. ```
  38. GeoTIFFs contain float32 embeddings on the native grid, with NaN nodata.
  39. Use `--bands` to select zero-based bands, `--depth` to select a published
  40. embedding prefix, or `--store-url` to read another Zarr store.
  41. Country and vector regions select their bounding boxes.
  42. Rerunning a Zarr export replaces its completed files. Use `--source tiles`
  43. for individual tiles that skip existing files on rerun. `--format npy`
  44. also selects individual tiles.
  45. Every download records its dataset version and variant in
  46. `tessera_metadata.json` in the output directory, and GeoTIFFs record them
  47. in their tags. Downloading into a directory that holds another dataset,
  48. or merging GeoTIFFs of different datasets, fails.
  49. Web maps reuse matching completed mosaics and tiles. Keep the output
  50. directory and its JSON completion files. Use `--force` to refresh a map
  51. when the source changes at the same URL, or `geotessera serve map/` to
  52. view an existing map without regenerating it.
  53. See the [CLI reference](https://geotessera.readthedocs.io/en/latest/cli_reference.html)
  54. for options, output files, and restart behavior.
  55. ### Global coverage
  56. Since GeoTessera 0.11, the default `v1.1-dclimate` dataset provides
  57. **wall-to-wall global v1.1 embeddings for every year from 2017 to 2025**.
  58. No request is needed; `GeoTesseraZarr()` and `geotessera download` read
  59. any land area directly.
  60. We recommend v1.1 for all users. Earlier versions remain selectable to
  61. reproduce existing results.
  62. If you find missing or defective embeddings, please
  63. **[open an issue](../../issues/new?template=embedding-request.yml&labels=embedding-request)**
  64. with the bounding box (lon,lat, 4 decimals) and year(s).
  65. ## Table of Contents
  66. - [Installation](#installation)
  67. - [Cloud-Native Zarr Access](#cloud-native-zarr-access)
  68. - [Architecture](#architecture)
  69. - [Quick Start](#quick-start)
  70. - [Python API](#python-api)
  71. - [CLI Reference](#cli-reference)
  72. - [Registry System](#registry-system)
  73. - [Data Organization](#data-organization)
  74. - [Contributing](#contributing)
  75. ## Installation
  76. GeoTessera requires Python 3.12 or later.
  77. ```bash
  78. pip install geotessera
  79. ```
  80. For development:
  81. ```bash
  82. git clone https://github.com/ucam-eo/geotessera
  83. cd geotessera
  84. uv sync --all-extras --dev # or: pip install -e ".[docs]"
  85. ```
  86. Sphinx lives in the `docs` extra and the test harness in the `dev` group, so a
  87. plain `pip install geotessera` pulls in neither.
  88. ## Cloud-Native Zarr Access
  89. The Zarr API reads selected pixels from the public store and returns
  90. dequantized float32 embeddings. Point and region reads use the native UTM
  91. grid. Patches can combine pixels across zone boundaries.
  92. ```python
  93. from geotessera import GeoTesseraZarr
  94. gt = GeoTesseraZarr() # v1.1 dClimate Icechunk store
  95. print(gt.years) # [2017, ..., 2025]
  96. # One embedding, with a status explaining any missing value
  97. vec, status = gt.probe(0.12, 52.20, year=2024)
  98. print(status) # 'valid', 'water', 'nodata', or 'outside'
  99. # Many points, one bulk read per UTM zone
  100. X = gt.sample_points([(0.12, 52.20), (-2.97, 53.44)], year=2024) # (2, 128)
  101. # A lon/lat bounding box as a mosaic on the native UTM grid
  102. bbox = (0.05, 52.15, 0.20, 52.25)
  103. mosaic, transform, crs = gt.read_region(bbox, year=2024)
  104. # A fixed-size patch centred on a point, merged across UTM zones when needed
  105. patch, transform, crs = gt.read_patch(0.12, 52.20, year=2024, size_px=256)
  106. # Stream a large region in row strips rather than holding it in memory
  107. for block, transform, crs in gt.iter_region(bbox, year=2024, strip_rows=512):
  108. predictions = model.predict(block.reshape(-1, 128))
  109. ```
  110. Export a region without holding the full array in memory:
  111. ```python
  112. files = gt.export_geotiffs(bbox, 2024, "region/", bands=[0, 1, 2])
  113. ```
  114. `gt.dataset` names the published dataset being read, such as
  115. `1.1-dclimate`. Exports record it in their tags and in the directory's
  116. `tessera_metadata.json`, and exporting into a directory that holds
  117. another dataset fails.
  118. Other dataset versions are selected by store URL. v2 stores also publish
  119. matryoshka prefixes. `depth=16` reads the first 16 dimensions; the bytes
  120. transferred depend on the store's chunk layout:
  121. ```python
  122. from geotessera.registry import zarr_store_url
  123. gt = GeoTesseraZarr(zarr_store_url("v2"))
  124. X16 = gt.sample_points([(0.12, 52.20)], year=2024, depth=16) # (N, 16)
  125. ```
  126. A location ending in `.icechunk` opens an Icechunk repository. Its UTM zone
  127. and hemisphere groups read as one `utmNN` zone on the northern CRS, with
  128. negative northings south of the equator; NaN scales mark unembedded pixels,
  129. reported as `nodata`.
  130. Set `cache_dir` to persist Zarr metadata between runs. Byte-range reads
  131. of sharded embeddings are cached within the process. Icechunk stores,
  132. including the default, ignore it.
  133. ```python
  134. gt = GeoTesseraZarr(zarr_store_url("v2"), cache_dir="tessera-cache")
  135. ```
  136. For direct access to one UTM zone, `gt.open_zone(lon=0.15)` returns an xarray
  137. dataset with a `.tessera` accessor that works in that zone's own eastings and
  138. northings. The store layout follows the `geoemb:` convention for geospatial
  139. embedding data; the
  140. [zarr quickstart](https://geotessera.readthedocs.io/en/latest/zarr_quickstart.html)
  141. and the [examples repository](https://github.com/ucam-eo/geotessera-examples)
  142. walk through complete workflows.
  143. ### Agent Skill
  144. The repository ships an [Agent Skill](https://agentskills.io) that teaches
  145. coding agents the zarr interface. Claude Code users can install it as a
  146. plugin:
  147. ```
  148. /plugin marketplace add ucam-eo/geotessera
  149. /plugin install geotessera@ucam-eo
  150. ```
  151. The skill itself is [`skills/geotessera/SKILL.md`](skills/geotessera/SKILL.md),
  152. in the open Agent Skills format, so it also works with other
  153. SKILL.md-compatible agents when copied into their skills directory.
  154. ## Architecture
  155. ### Core Concepts
  156. This section describes the tile download interface; the
  157. [zarr backend](#cloud-native-zarr-access) above streams the same embeddings
  158. without downloading files. The tile workflow has two steps:
  159. 1. **Retrieve embeddings**: Fetch raw numpy arrays for a geographic bounding box
  160. 2. **Export to desired format**: Save as raw numpy arrays or convert to georeferenced GeoTIFF files
  161. ### Coordinate System and Tile Grid
  162. The Tessera embeddings use a **0.1-degree grid system**:
  163. - **Tile size**: Each tile covers 0.1° × 0.1° (approximately 11km × 11km at the equator)
  164. - **Tile naming**: Tiles are named by their **center coordinates** (e.g., `grid_0.15_52.05`)
  165. - **Tile bounds**: A tile at center (lon, lat) covers:
  166. - Longitude: [lon - 0.05°, lon + 0.05°]
  167. - Latitude: [lat - 0.05°, lat + 0.05°]
  168. - **Resolution**: 10m per pixel (variable number of pixels per tile depending on latitude)
  169. ### File Structure and Downloads
  170. When you request embeddings, GeoTessera downloads files over HTTPS from the
  171. public Source Cooperative repository into the output directory you specify,
  172. where they persist for re-use:
  173. #### Embedding Files (via `fetch_embedding`)
  174. 1. **Quantized embeddings** (`grid_X.XX_Y.YY.npy`):
  175. - Shape: `(height, width, 128)`
  176. - Data type: int8 (quantized for storage efficiency)
  177. - Contains the compressed embedding values
  178. 2. **Scale files** (`grid_X.XX_Y.YY_scales.npy`):
  179. - Shape: `(height, width)` or `(height, width, 128)`
  180. - Data type: float32
  181. - Contains scale factors for dequantization
  182. 3. Use `geotessera.dequantize_embedding(quantized_embedding, scales)` to
  183. dequantize arrays. The helper handles scale broadcasting and missing values.
  184. 4. **Persistent Storage**: Files are downloaded into your chosen output
  185. directory and skipped on rerun, so interrupted downloads resume cleanly
  186. #### Landmask Files (for GeoTIFF export)
  187. When exporting to GeoTIFF, additional landmask files are fetched:
  188. - **Landmask tiles** (`grid_X.XX_Y.YY.tiff`):
  189. - Provide UTM projection information
  190. - Define precise geospatial transforms
  191. - Contain land/water masks
  192. - Cached alongside the embedding tiles for re-use
  193. ### Data Flow
  194. ```
  195. User Request (lat/lon bbox)
  196. ↓
  197. Parquet Registry Lookup (find available tiles from manifest.parquet)
  198. ↓
  199. HTTPS Downloads from Source Cooperative to Output Directory (integrity verified)
  200. ├── embedding.npy (quantized) → output dir
  201. └── embedding_scales.npy → output dir
  202. ↓
  203. Dequantization (multiply arrays)
  204. ↓
  205. Output Format
  206. ├── NumPy arrays → Direct analysis
  207. └── GeoTIFF → GIS integration
  208. ```
  209. **Storage Note**: Only the Parquet manifests (tens to a couple of hundred
  210. MB per dataset) are cached under `~/.cache/geotessera`. Embedding tiles are
  211. downloaded on demand into the output directory you specify and persist there
  212. for re-use across runs.
  213. ## Quick Start
  214. ### Check Available Data
  215. Before downloading, check what data is available:
  216. ```bash
  217. # Map the default v1.1 dclimate dataset from its tile registry
  218. geotessera coverage --output coverage_map.png
  219. # Map it for the UK, or for one year
  220. geotessera coverage --country uk
  221. geotessera coverage --year 2024 --output coverage_2024.png
  222. # Map NPY tile coverage, with coverage.json and globe.html
  223. geotessera coverage --dataset-variant cambridge --output tiles_coverage.png
  224. ```
  225. ### Download Embeddings
  226. Download individual tiles as NumPy arrays or GeoTIFF files. Tiles are
  227. deprecated and will be removed. v1.1 tiles exist only for the `cambridge`
  228. variant, which these commands use with a warning; add
  229. `--dataset-variant cambridge` to select it explicitly.
  230. ```bash
  231. # Download as GeoTIFF (default, with georeferencing)
  232. geotessera download --source tiles \
  233. --bbox "-0.2,51.4,0.1,51.6" \
  234. --year 2024 \
  235. --output ./london_tiffs
  236. # Download quantized arrays with scales and landmasks.
  237. geotessera download --source tiles \
  238. --bbox "-0.2,51.4,0.1,51.6" \
  239. --format npy \
  240. --year 2024 \
  241. --output ./london_arrays
  242. # Download using a GeoJSON/Shapefile region
  243. geotessera download --source tiles \
  244. --region-file cambridge.geojson \
  245. --format tiff \
  246. --year 2024 \
  247. --output ./cambridge_tiles
  248. # Download specific bands only
  249. geotessera download --source tiles \
  250. --bbox "-0.2,51.4,0.1,51.6" \
  251. --bands "0,1,2" \
  252. --year 2024 \
  253. --output ./london_rgb
  254. ```
  255. ### Create Visualizations
  256. Generate PCA visualizations and web maps from downloaded GeoTIFFs:
  257. ```bash
  258. # Create a PCA mosaic from downloaded tiles
  259. geotessera visualize ./london_tiffs pca_mosaic.tif
  260. # Use histogram equalization for maximum contrast
  261. geotessera visualize ./london_tiffs pca_balanced.tif --balance histogram
  262. # Create web tiles and serve interactively
  263. geotessera webmap pca_mosaic.tif --serve
  264. # Serve existing web visualizations locally
  265. geotessera serve ./london_web --open
  266. ```
  267. ## Python API
  268. ### Core Methods
  269. The `GeoTessera` class downloads embedding tiles as files; for streaming
  270. access use the [zarr backend](#cloud-native-zarr-access). NPY tiles are
  271. deprecated and will be removed. v1.1 tiles exist only for the `cambridge`
  272. variant, which `GeoTessera()` uses with a warning; pass
  273. `dataset_variant="cambridge"` to select it explicitly. The tile interface
  274. provides two main methods for retrieving embeddings:
  275. ```python
  276. from geotessera import GeoTessera
  277. # Initialize the client
  278. gt = GeoTessera()
  279. # Method 1: Fetch a single tile
  280. embedding, crs, transform = gt.fetch_embedding(lon=0.15, lat=52.05, year=2024)
  281. print(f"Shape: {embedding.shape}") # e.g., (1200, 1200, 128)
  282. print(f"CRS: {crs}") # Coordinate reference system from landmask
  283. # Method 2: Fetch all tiles in a bounding box
  284. bbox = (-0.2, 51.4, 0.1, 51.6) # (min_lon, min_lat, max_lon, max_lat)
  285. tiles_to_fetch = gt.registry.load_blocks_for_region(bounds=bbox, year=2024)
  286. embeddings = gt.fetch_embeddings(tiles_to_fetch)
  287. for year, tile_lon, tile_lat, embedding_array, crs, transform in embeddings:
  288. print(f"Tile ({tile_lat}, {tile_lon}): {embedding_array.shape}")
  289. ```
  290. ### Export Formats
  291. #### Export as GeoTIFF
  292. ```python
  293. # Export embeddings for a region as individual GeoTIFF files
  294. # Step 1: Get the tiles for the region
  295. bbox = (-0.2, 51.4, 0.1, 51.6)
  296. tiles_to_fetch = gt.registry.load_blocks_for_region(bounds=bbox, year=2024)
  297. # Step 2: Export those tiles as GeoTIFFs
  298. files = gt.export_embedding_geotiffs(
  299. tiles_to_fetch=tiles_to_fetch,
  300. output_dir="./output",
  301. bands=None, # Export all 128 bands (default)
  302. compress="lzw" # Compression method
  303. )
  304. print(f"Created {len(files)} GeoTIFF files")
  305. # Export specific bands only (e.g., first 3 for RGB visualization)
  306. files = gt.export_embedding_geotiffs(
  307. tiles_to_fetch=tiles_to_fetch,
  308. output_dir="./rgb_output",
  309. bands=[0, 1, 2] # Only export first 3 bands
  310. )
  311. ```
  312. #### Work with NumPy Arrays
  313. ```python
  314. # Fetch and process embeddings directly
  315. tiles_to_fetch = gt.registry.load_blocks_for_region(bounds=bbox, year=2024)
  316. embeddings = gt.fetch_embeddings(tiles_to_fetch)
  317. for year, tile_lon, tile_lat, embedding, crs, transform in embeddings:
  318. # Compute statistics
  319. mean_values = np.mean(embedding, axis=(0, 1)) # Mean per channel
  320. std_values = np.std(embedding, axis=(0, 1)) # Std per channel
  321. # Extract specific pixels
  322. center_pixel = embedding[embedding.shape[0]//2, embedding.shape[1]//2, :]
  323. # Apply custom processing
  324. processed = your_analysis_function(embedding)
  325. ```
  326. ### Visualization Functions
  327. ```python
  328. from geotessera.visualization import (
  329. create_rgb_mosaic,
  330. visualize_global_coverage
  331. )
  332. from geotessera.web import (
  333. create_coverage_summary_map,
  334. geotiff_to_web_tiles
  335. )
  336. # Create an RGB mosaic from multiple GeoTIFF files
  337. create_rgb_mosaic(
  338. geotiff_paths=["tile1.tif", "tile2.tif"],
  339. output_path="mosaic.tif",
  340. bands=(0, 1, 2) # RGB bands
  341. )
  342. # Generate web tiles for interactive maps
  343. geotiff_to_web_tiles(
  344. geotiff_path="mosaic.tif",
  345. output_dir="./web_tiles",
  346. zoom_levels=(8, 15)
  347. )
  348. # Create a global coverage visualization
  349. visualize_global_coverage(
  350. tessera_client=gt,
  351. output_path="global_coverage.png",
  352. year=2024, # Or None for all years
  353. width_pixels=2000,
  354. tile_color="red",
  355. tile_alpha=0.6
  356. )
  357. ```
  358. ## CLI Reference
  359. Use `geotessera COMMAND --help` for command options. The
  360. [command reference](https://geotessera.readthedocs.io/en/latest/cli_reference.html)
  361. describes output formats, region selection, caching, and repeated runs.
  362. ### download
  363. Export a region from Zarr as GeoTIFFs. Add `--source tiles` for individual
  364. GeoTIFF tiles, or `--format npy` for quantized arrays with scales and landmasks.
  365. ```bash
  366. geotessera download --bbox '-3.0,53.4,-2.9,53.5' --bands 0,1,2 --output region/
  367. ```
  368. ### visualize
  369. Create a PCA mosaic from a GeoTIFF file or a directory of GeoTIFF or NPY
  370. tiles. One sampled PCA model and color scale are applied across the inputs.
  371. The output contains display-scaled uint8 components; use the original
  372. embeddings for analysis.
  373. ```bash
  374. geotessera visualize region/ pca.tif
  375. ```
  376. ### webmap
  377. Create web tiles and a viewer from an RGB GeoTIFF or a Zarr region.
  378. Tile generation requires the GDAL command-line tools.
  379. ```bash
  380. geotessera webmap pca.tif --output map/ --serve
  381. geotessera webmap --bbox '-3.0,53.4,-2.9,53.5' --output region_map/
  382. ```
  383. ### coverage
  384. Show data availability as a PNG map and HTML globe. Icechunk datasets,
  385. including the default, are drawn from their tile registry as a PNG map
  386. alone. Use `--by-source` to compare the datasets with NPY tiles.
  387. ```bash
  388. geotessera coverage --country 'United Kingdom' --year 2024
  389. ```
  390. ### serve
  391. Serve an existing map directory over HTTP. An occupied port causes an error.
  392. ```bash
  393. geotessera serve map/ --port 8001 --html viewer.html
  394. ```
  395. ### info
  396. List every dataset, its formats and store URLs, and summarise the selected
  397. dataset; or inspect local GeoTIFF and NPY files.
  398. ```bash
  399. geotessera info
  400. geotessera info --tiles region/
  401. ```
  402. ## Registry System
  403. ### Overview
  404. GeoTessera uses a Parquet-based registry system to efficiently manage and access the large Tessera dataset:
  405. - **Per-version manifests**: Each dataset version has its own `manifest.parquet`
  406. listing every `(year, lon, lat)` tile available for that version's variants
  407. - **Fast queries**: Uses pandas DataFrames for efficient spatial and temporal filtering
  408. - **Block-based organization**: Internal 5×5 degree geographic blocks for efficient queries
  409. - **Minimal storage**: Only manifest files (tens to a couple of hundred MB per dataset) are cached locally
  410. - **Integrity checking**: Every download is verified against the response
  411. `Content-Length`, and against an MD5 computed over the streamed body whenever
  412. the server's `ETag` is a content MD5 (single-part uploads)
  413. - A mismatch rejects the download and triggers a retry, so corrupt or
  414. truncated files never reach the cache
  415. ### Dataset Versions and Variants
  416. Tessera embeddings are published as dataset *versions* (e.g. `v1`, `v1.1`,
  417. `v2`) and, within a version, as *variants*, each a separate inference run.
  418. Embeddings from different variants do not interoperate, even within one
  419. version: train and predict on the same `(version, variant)`. Each variant is
  420. published in one or more formats:
  421. | Version | Variant | NPY tiles (`npy/`) | Zarr (`zarr/`) | Icechunk |
  422. |---------|------------------------|----------------------|------------------|--------------------------------------------------|
  423. | `1.0` | `vultr` (default) | `v1/` | `v1/` | — |
  424. | `1.1` | `dclimate` (default) | — | `v1.1-dclimate/` | `s3://tessera-embeddings/v1.1/dclimate.icechunk` |
  425. | `1.1` | `cambridge` | `v1.1-cam/` | `v1.1/` | — |
  426. | `2.0` | `2B-L~beta1` (default) | `v2-2B-L~beta1/` | `v2-2B-L~beta1/` | — |
  427. | `2.0` | `2B-L~beta2` | `v2-2B-L~beta2/` | `v2-2B-L~beta2/` | — |
  428. The default version is `v1.1`. Streamed reads (`GeoTesseraZarr()`,
  429. `download`, `webmap`) use the version's default variant, `dclimate`, read
  430. from its Zarr store on Source Cooperative; the same run is also published as
  431. an [Icechunk store](https://github.com/dClimate/tessera-embeddings/blob/main/docs/global-store.md).
  432. NPY tiles of v1.1 exist only for `cambridge`, so `GeoTessera` and
  433. `download --format npy` fall back to it with a warning; pass
  434. `--dataset-variant cambridge` to select it explicitly. `coverage` and
  435. `info` read the Icechunk store's Parquet tile registry in place of an NPY
  436. manifest. The table is `DATASETS` in `geotessera/registry.py`.
  437. NPY tiles are deprecated and will be removed; use Zarr or Icechunk.
  438. List the datasets at any time with `geotessera info`, and select them on the
  439. CLI with `--dataset-version` and `--dataset-variant`, or in Python:
  440. ```python
  441. gt = GeoTessera(dataset_version="v1.1", dataset_variant="cambridge")
  442. ```
  443. Use `geotessera coverage --by-source` to render each `(version, variant)` source
  444. in a distinct colour on the coverage map and globe viewer.
  445. ### Registry Sources
  446. The registry can be loaded from multiple sources (in priority order):
  447. 1. **Local file** (via `registry_path` parameter)
  448. 2. **Local directory** (via `--registry-dir` or `registry_dir` parameter, looks for `manifest.parquet`, falling back to the legacy `registry.parquet`)
  449. 3. **Remote URL** (via `registry_url` parameter)
  450. 4. **Default remote** (from `https://data.source.coop/tessera/tessera/npy/{dataset}/manifest.parquet`, where `{dataset}` encodes the (version, variant) pair: `v1`, `v1.1-cam`, `v2-2B-L~beta1`)
  451. ```python
  452. # Use local manifest file
  453. gt = GeoTessera(registry_path="/path/to/manifest.parquet")
  454. # Use local registry directory
  455. gt = GeoTessera(registry_dir="/path/to/registry-dir")
  456. # Use default remote manifest (downloads and caches automatically)
  457. gt = GeoTessera() # Default behavior
  458. ```
  459. ### Registry Structure
  460. The Parquet manifest contains columns for:
  461. - **Coordinates**: `lon`, `lat` (tile center coordinates)
  462. - **Year**: `year` (data year, 2017-2025)
  463. - **Size**: `file_size` (file size in bytes for download planning)
  464. ```python
  465. # Example manifest query
  466. import pandas as pd
  467. manifest = pd.read_parquet("manifest.parquet")
  468. print(manifest.head())
  469. ```
  470. ### Regenerating Manifests (Maintainers)
  471. The per-version manifests can be rebuilt at any time by scanning the public
  472. Source Cooperative repository itself — no local copy of the data is needed:
  473. ```bash
  474. # Rescan every dataset and write one manifest per npy/ directory
  475. # (./manifests/{v1,v1.1-cam,v2-2B-L~beta1}/manifest.parquet) plus a
  476. # per-version landmasks.parquet (./manifests/{v1,v1.1,v2}/landmasks.parquet)
  477. geotessera-registry s3scan s3://tessera/tessera/npy/ \
  478. --landmasks-uri s3://tessera/tessera/landmasks/ \
  479. --output ./manifests
  480. # Or scope to a single dataset — the directory name encodes the
  481. # (version, variant) pair, so no --variant flag is needed
  482. geotessera-registry s3scan "s3://tessera/tessera/npy/v1.1-cam/" \
  483. --landmasks-uri s3://tessera/tessera/landmasks/v1.1/ \
  484. --output ./manifests
  485. ```
  486. The scan uses anonymous S3 `ListObjectsV2` calls against
  487. `https://data.source.coop`, so no credentials are required to regenerate.
  488. Uploading the results does require source.coop write access, and the two
  489. parquet files go to *different* trees (matching where clients fetch them —
  490. note the npy/ tree is keyed by dataset directory, the landmasks/ tree by
  491. plain version):
  492. ```bash
  493. aws s3 cp manifests/v1.1-cam/manifest.parquet \
  494. s3://tessera/tessera/npy/v1.1-cam/manifest.parquet \
  495. --endpoint-url https://data.source.coop
  496. aws s3 cp manifests/v1.1/landmasks.parquet \
  497. s3://tessera/tessera/landmasks/v1.1/landmasks.parquet \
  498. --endpoint-url https://data.source.coop
  499. ```
  500. The `s3scan` summary panel prints these per-file upload commands for you.
  501. Transient listing failures (Cloudflare 503s, timeouts) are retried with
  502. exponential backoff; if a shard still fails after retries, the affected
  503. manifest is not written and the command exits non-zero so an incomplete
  504. manifest can never be uploaded. See the
  505. [maintenance guide](https://geotessera.readthedocs.io/en/latest/maintenance.html)
  506. for the full workflow, including caching caveats.
  507. ### How Registry Loading Works
  508. 1. **Load Parquet manifest** → Download and cache the dataset's manifest (if not local)
  509. 2. **Request tiles for bbox** → Query DataFrame for tiles in region
  510. 3. **Filter by year and variant** → Select tiles matching the requested year/variant
  511. 4. **Find available tiles** → Return list of matching tiles
  512. 5. **HTTPS download** → Fetch tiles on demand from the Source Cooperative mirror into the output directory, with integrity checks
  513. 6. **Persist** → Downloaded tiles stay in the output directory and are skipped on rerun
  514. ## Data Organization
  515. ### Tessera Data Structure
  516. ```
  517. Remote Server (https://data.source.coop/tessera/tessera)
  518. ├── npy/ # NPY embeddings + scales
  519. │ │ # (one dir per (version, variant) dataset)
  520. │ ├── v1/ # 1.0 — all variants share this dir
  521. │ │ ├── manifest.parquet # Per-dataset tile manifest
  522. │ │ └── 2024/grid_0.15_52.05/grid_0.15_52.05{,_scales}.npy
  523. │ ├── v1.1-cam/ # 1.1 / cambridge
  524. │ │ ├── manifest.parquet
  525. │ │ └── 2024/grid_0.15_52.05/grid_0.15_52.05{,_scales}.npy
  526. │ └── v2-2B-L~beta1/ # 2.0 / 2B-L~beta1 (beta)
  527. │ └── 2024/grid_0.15_52.05/grid_0.15_52.05{,_scales}.npy
  528. ├── landmasks/ # Landmask TIFFs (per version)
  529. │ ├── v1/
  530. │ │ ├── landmasks.parquet # Landmask manifest
  531. │ │ └── grid_0.15_52.05.tiff # Landmask with projection info
  532. │ ├── v1.1/
  533. │ │ ├── landmasks.parquet
  534. │ │ └── grid_0.15_52.05.tiff
  535. │ └── v2/
  536. │ ├── landmasks.parquet
  537. │ └── grid_0.15_52.05.tiff
  538. └── zarr/ # Cloud-native zarr stores
  539. ├── v1/ # 1.0 / vultr: 60 UTM zone groups + RGB pyramid
  540. ├── v1.1/ # 1.1 / cambridge
  541. ├── v1.1-dclimate/ # 1.1 / dclimate, the default
  542. ├── v2-2B-L~beta1/ # v2 stores add matryoshka
  543. └── v2-2B-L~beta2/ # prefix arrays (d4, d16)
  544. Icechunk (s3://tessera-embeddings/v1.1)
  545. ├── dclimate.icechunk/ # 1.1 / dclimate, one group per zone and hemisphere
  546. └── dclimate.registry/parts/ # Parquet tile registry
  547. ```
  548. ### Local Cache Structure
  549. ```
  550. ~/.cache/geotessera/ # Default cache location (manifests only)
  551. ├── v1/ # 1.0 dataset dir + v1 landmask registry
  552. │ ├── manifest.parquet
  553. │ └── landmasks.parquet
  554. ├── v1.1-cam/ # 1.1/cambridge dataset dir
  555. │ └── manifest.parquet
  556. ├── v1.1/ # v1.1 landmask registry
  557. │ └── landmasks.parquet
  558. ├── v2-2B-L~beta1/ # 2.0 beta dataset dir
  559. │ └── manifest.parquet
  560. └── v2/ # v2 landmask registry
  561. └── landmasks.parquet
  562. # Note: Embedding and landmask tiles are NOT stored here. They are downloaded
  563. # into the output directory you specify and persist there for re-use.
  564. ```
  565. ### Coordinate Reference Systems
  566. - **Embeddings**: Stored in simple arrays, referenced by center coordinates
  567. - **GeoTIFF exports**: Use UTM projection from corresponding landmask tiles
  568. - **Web visualizations**: Reprojected to Web Mercator (EPSG:3857)
  569. ## Cache Configuration
  570. Tile workflows cache manifests and write embedding and landmask files to
  571. the output directory for reuse. Zarr reads persist metadata in `cache_dir`
  572. and cache byte ranges within the process. Keep exported files or completed
  573. web map directories to reuse their data between runs.
  574. ### Python API
  575. ```python
  576. from geotessera import GeoTessera
  577. # Use custom cache directory for registry
  578. gt = GeoTessera(cache_dir="/path/to/cache")
  579. # Use default cache location (recommended)
  580. gt = GeoTessera()
  581. ```
  582. ### CLI
  583. ```bash
  584. # Specify custom cache directory
  585. geotessera download --source tiles --cache-dir /path/to/cache ...
  586. # Use default cache location
  587. geotessera download --source tiles ...
  588. ```
  589. ### Default Cache Locations
  590. When `cache_dir` is not specified, the registry is cached in platform-appropriate locations:
  591. - **Linux/macOS**: `$XDG_CACHE_HOME/geotessera` or `~/.cache/geotessera`
  592. - **Windows**: `%LOCALAPPDATA%/geotessera`
  593. ## Hash Verification
  594. GeoTessera verifies every downloaded file (embeddings, scales, and landmasks) against the response `Content-Length`, and additionally against an MD5 computed over the streamed body whenever the server's `ETag` is a content MD5 (a single-part upload; this covers landmask TIFFs and scales files). Large multipart-uploaded embedding tiles carry a composite ETag that is not a content hash, so they are length-checked only. A mismatch rejects the download and triggers a retry with backoff, so corrupt or truncated files never reach the cache.
  595. ## Contributing
  596. Contributions are welcome! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
  597. This project is licensed under the MIT License - see the [LICENSE](LICENSE.md) file for details.
  598. ## Citation
  599. If you use Tessera in your research, please cite the [arXiv paper](https://arxiv.org/abs/2506.20380):
  600. ```bibtex
  601. @misc{feng2025tesseratemporalembeddingssurface,
  602. title={TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis},
  603. author={Zhengpeng Feng and Clement Atzberger and Sadiq Jaffer and Jovana Knezevic and Silja Sormunen and Robin Young and Madeline C Lisaius and Markus Immitzer and David A. Coomes and Anil Madhavapeddy and Andrew Blake and Srinivasan Keshav},
  604. year={2025},
  605. eprint={2506.20380},
  606. archivePrefix={arXiv},
  607. primaryClass={cs.LG},
  608. url={https://arxiv.org/abs/2506.20380},
  609. }
  610. ```
  611. ## Links
  612. - [Tessera Foundation Model](https://github.com/ucam-eo/tessera)
  613. - [Tessera Interactive Notebook](https://github.com/ucam-eo/tessera-interactive-map)
  614. - [Tessera Examples](https://github.com/ucam-eo/geotessera-examples)
  615. - [Documentation](https://geotessera.readthedocs.io/)
  616. - [PyPI Package](https://pypi.org/project/geotessera/)
  617. - [Issue Tracker](https://github.com/ucam-eo/geotessera/issues)
  618. ## Star History
  619. [![Star History Chart](https://star-history.dera.page/svg?repos=ucam-eo/geotessera&type=Date)](https://star-history.dera.page/#ucam-eo/geotessera&Date)

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.