Skip to content
Back to search

portaljs

πŸŒ€ AI-native framework for building data portals. Scaffold a full portal from a brief and load datasets in minutes with agentic skills β€” any backend (CKAN, GitHub, Frictionless).

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

Its scripts were only partly scanned.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/datopian/portaljs with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/datopian/portaljs'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/datopian--portaljs/badge.svg)](/listings/datopian--portaljs)

HTML

<a href="/listings/datopian--portaljs"><img src="/api/listings/datopian--portaljs/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

Install instructions from the README

As written by the repo's author. findsafeskills has not verified these steps.

These steps run a script from the network that findsafeskills does not scan.

**Create a portal** β€” one command, nothing to install beyond Node 22+:

npm create portaljs@latest my-portal
cd my-portal
npm run dev      # β†’ http://localhost:3000

You get the three surfaces β€” Home, a Catalog (`/search`), and a dataset Showcase (`/@<namespace>/<slug>`) β€” over sample data. Plain, editable Next.js, no lock-in. Add your own CSV/JSON to `datasets.json` and it renders automatically.

**Build it with your AI assistant** β€” PortalJS ships [Claude Code](https://claude.com/claude-code) skills that do the assembly. Install them once (into `~/.claude/commands`):

curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash

Then, in a Claude Code session from any directory:

/portaljs-architect    not sure what stack you need? start here
/portaljs-new-portal   "Auckland Council open data portal"
/portaljs-add-dataset  ./data/air-quality.csv

`/portaljs-new-portal` scaffolds the three surfaces; `/portaljs-add-dataset` (or `/portaljs-add-resource`) loads data; `/portaljs-connect-ckan` points it at a CKAN backend; `/portaljs-deploy` ships it. ([All skills + install β†’](.claude/INSTALL.md))

**Prefer the bare template** β€” plain Next.js, no AI, no lock-in:

npx tiged datopian/portaljs/examples/portaljs-catalog my-portal
cd my-portal && npm install && npm run dev      # β†’ http://localhost:3000

You get Home, a Catalog (`/search`), and a dataset Showcase (`/@<namespace>/<slug>`) over sample data. Add your own CSV/JSON to `datasets.json` and it renders automatically.

⭐ If it's useful, a star helps others find it.

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

.claude-plugin/marketplace.json Β· Manifest Β· 18 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. {
  2. "name": "datopian-portaljs",
  3. "owner": {
  4. "name": "Datopian",
  5. "url": "https://www.datopian.com"
  6. },
  7. "metadata": {
  8. "description": "PortalJS agentic skills β€” build data portals with AI.",
  9. "version": "0.1.0"
  10. },
  11. "plugins": [
  12. {
  13. "name": "portaljs",
  14. "source": ".",
  15. "description": "Scaffold and extend PortalJS data portals: /portaljs-new-portal, /portaljs-add-dataset, /portaljs-add-chart, /portaljs-add-map, /portaljs-deploy."
  16. }
  17. ]
  18. }
README.md Β· README Β· 263 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. <p align="center">
  2. <img src="assets/portaljs-logo-spin.svg" alt="PortalJS" width="96" height="96" />
  3. <h1 align="center">PortalJS</h1>
  4. <p align="center">
  5. <b>The AI-native framework for building data portals.</b>
  6. <br />
  7. Describe the portal you want β€” your agent helps you choose an architecture, scaffolds it, and loads your data.
  8. <br />
  9. <br />
  10. <a href="https://www.portaljs.com/docs">Docs</a>
  11. Β·
  12. <a href="https://github.com/datopian/portaljs/discussions">Discussions</a>
  13. Β·
  14. <a href="https://github.com/datopian/portaljs/issues/new">Report a bug</a>
  15. <br />
  16. <br />
  17. <a href="https://www.npmjs.com/package/create-portaljs"><img src="https://img.shields.io/npm/v/create-portaljs?logo=npm&logoColor=white&label=create-portaljs&color=cb3837" alt="npm version" /></a>
  18. <a href="https://github.com/datopian/portaljs/stargazers"><img src="https://img.shields.io/github/stars/datopian/portaljs?logo=github&logoColor=white&label=Stars&color=blue" alt="GitHub stars" /></a>
  19. <a href="https://discord.gg/krmj5HM6He"><img src="https://img.shields.io/badge/Discord-Join-5865F2?logo=discord&logoColor=white" alt="Join our Discord" /></a>
  20. <a href="license"><img src="https://img.shields.io/badge/License-MIT-blue" alt="MIT License" /></a>
  21. </p>
  22. </p>
  23. ---
  24. ## Quickstart
  25. **Create a portal** β€” one command, nothing to install beyond Node 22+:
  26. ```bash
  27. npm create portaljs@latest my-portal
  28. cd my-portal
  29. npm run dev # β†’ http://localhost:3000
  30. ```
  31. You get the three surfaces β€” Home, a Catalog (`/search`), and a dataset Showcase
  32. (`/@<namespace>/<slug>`) β€” over sample data. Plain, editable Next.js, no lock-in. Add your
  33. own CSV/JSON to `datasets.json` and it renders automatically.
  34. **Build it with your AI assistant** β€” PortalJS ships [Claude Code](https://claude.com/claude-code)
  35. skills that do the assembly. Install them once (into `~/.claude/commands`):
  36. ```bash
  37. curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash
  38. ```
  39. Then, in a Claude Code session from any directory:
  40. ```text
  41. /portaljs-architect not sure what stack you need? start here
  42. /portaljs-new-portal "Auckland Council open data portal"
  43. /portaljs-add-dataset ./data/air-quality.csv
  44. ```
  45. `/portaljs-new-portal` scaffolds the three surfaces; `/portaljs-add-dataset` (or `/portaljs-add-resource`) loads data;
  46. `/portaljs-connect-ckan` points it at a CKAN backend; `/portaljs-deploy` ships it. ([All skills + install β†’](.claude/INSTALL.md))
  47. **Prefer the bare template** β€” plain Next.js, no AI, no lock-in:
  48. ```bash
  49. npx tiged datopian/portaljs/examples/portaljs-catalog my-portal
  50. cd my-portal && npm install && npm run dev # β†’ http://localhost:3000
  51. ```
  52. You get Home, a Catalog (`/search`), and a dataset Showcase (`/@<namespace>/<slug>`) over
  53. sample data. Add your own CSV/JSON to `datasets.json` and it renders automatically.
  54. ⭐ If it's useful, a star helps others find it.
  55. ## Why PortalJS
  56. Building a data portal has always meant more than a website. You have to decide where
  57. the data lives, how it's versioned, how people search it, how it's served, and how it's
  58. governed β€” and then wire a frontend on top. Teams either over-build on a heavy data
  59. warehouse they don't need, or under-build on a pile of scripts that doesn't scale.
  60. PortalJS is an **open-source, agentic skills framework that helps data teams build,
  61. develop, and ship data portals β€” and the data infrastructure underneath them.** It isn't
  62. only a frontend. The skills do two jobs:
  63. - **Advise** β€” given _what you're building_, _what your data is_, and _what it's for_,
  64. they recommend an architecture: storage, compute, catalog, access, hosting, metadata.
  65. - **Build** β€” they scaffold that stack as plain, editable Next.js code with no lock-in.
  66. It is **opinionated but open**: the recommended modern path is git + object storage
  67. (Cloudflare R2) + Parquet, queried with [DuckDB](https://duckdb.org/) β€” an open lakehouse
  68. instead of a classic warehouse. For living, incremental tables you can layer on
  69. [DuckLake](https://ducklake.select/), and a traditional datastore (CKAN, a warehouse)
  70. stays a first-class option when you need it. You always own plain code.
  71. Built and maintained in the open by [Datopian](https://www.datopian.com/) and the
  72. PortalJS community.
  73. ## Architecture at a glance
  74. ```text
  75. πŸ§‘ you describe what you want to build
  76. β”‚
  77. β–Ό
  78. ╭─ πŸ€– AGENTIC SKILLS ────────────────────────────────── decide + build
  79. β”‚ /portaljs-architect Β· /portaljs-new-portal Β· /portaljs-add-dataset Β· /portaljs-add-chart Β· /portaljs-add-map …
  80. ╰─ generates plain, editable Next.js code β€” no lock-in
  81. β”‚
  82. β–Ό
  83. ╭─ πŸ–₯️ SURFACES ──────────────────────────────────────── what users see
  84. β”‚ 🏠 Home / πŸ”Ž Catalog /search πŸ“Š Showcase /@ns/slug
  85. ╰─ read data through one DataProvider contract
  86. β”‚
  87. β–Ό
  88. ╭─ πŸ”Œ PROVIDERS ─────────────────────────────────────── pluggable backends
  89. β”‚ πŸ“ staticΒ·git 🐘 CKAN πŸ”­ OpenMetadata πŸ—‚οΈ git-LFS + R2
  90. ╰─ swap the source without touching a page
  91. β”‚
  92. β–Ό
  93. πŸ“¦ STORAGE + COMPUTE β€” choose your point on the spectrum:
  94. flat files ─▢ Git-LFS + R2 ─▢ Parquet on R2 + πŸ¦† DuckDB ─▢ warehouse / CKAN
  95. simplest ⭐ open lakehouse (default) heaviest
  96. (+ DuckLake for living tables)
  97. ☁️ Substrate β€” Cloudflare R2 (storage) Β· Workers (runtime) Β· D1 (catalog) Β· Pages (static)
  98. object storage stays S3-compatible β€” R2 is the default, never a lock-in
  99. ```
  100. **Three surfaces.** Every data portal is built from three: a **Home** page that explains
  101. it and offers search, a **Catalog** (`/search`) to discover datasets, and a **Showcase**
  102. (`/@<namespace>/<slug>`) to explore one dataset β€” metadata, preview, download/API, and
  103. charts/maps. ([Core concepts β†’](https://www.portaljs.com/docs/core-concepts))
  104. **One seam.** The surfaces read data only through a `DataProvider`, so the source β€” static
  105. files today, a CKAN or lakehouse backend tomorrow β€” can change without touching a page.
  106. See [`ROADMAP.md`](ROADMAP.md) for the full model and the
  107. [architecture decision framework](https://www.portaljs.com/docs/architecture/decision-framework)
  108. for how `/portaljs-architect` turns your needs into a stack.
  109. ## Build a portal with your AI assistant
  110. PortalJS ships [Claude Code](https://claude.com/claude-code) skills that turn a brief into
  111. a working portal.
  112. ### Setup
  113. Install the skills once into your personal scope so they're available from **any**
  114. directory:
  115. ```bash
  116. curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash
  117. ```
  118. Restart Claude Code (or open a new session) and type `/` to see them. See
  119. [`.claude/INSTALL.md`](.claude/INSTALL.md) for other install options (versioned plugin, or
  120. running straight from a clone of this repo).
  121. ### Use
  122. If you're not sure how to set up your portal, start with the advisor, then build:
  123. ```text
  124. /portaljs-architect we have ~200 public CSVs, updated quarterly, and must publish DCAT-AP
  125. /portaljs-new-portal "Auckland Council open data portal"
  126. /portaljs-add-dataset ./data/air-quality.csv
  127. /portaljs-add-dataset https://example.com/parks.geojson
  128. ```
  129. The skills are **interactive** β€” if your brief is thin, they interview you in short rounds
  130. rather than erroring. `/portaljs-architect` recommends a stack and hands off; `/portaljs-new-portal`
  131. scaffolds the three surfaces; `/portaljs-add-dataset` appends to the `datasets.json` manifest and
  132. the showcase renders automatically at `/@<namespace>/<slug>`. Run `npm run dev` and you
  133. have a portal.
  134. **Prefer to build by hand?** The skills are a convenience, not a requirement β€” scaffold the
  135. template directly with the CLI:
  136. ```bash
  137. npm create portaljs@latest my-portal
  138. ```
  139. (Or grab the bare template with no prompts: `npx tiged datopian/portaljs/examples/portaljs-catalog my-portal`.)
  140. ### Available skills
  141. <!-- BEGIN:skills-table -->
  142. | Skill | What it does |
  143. | ----- | ------------ |
  144. | [`/portaljs-architect`](.claude/commands/portaljs-architect.md) | Advisory β€” turns your needs (data, scale, governance) into a recommended architecture before you build. Start here if you're unsure of the stack. |
  145. | [`/portaljs-new-portal`](.claude/commands/portaljs-new-portal.md) | Scaffold a new portal (Home + Catalog + Showcase) from a brief β€” copies the template, substitutes your project name and description, installs deps, verifies the build. |
  146. | [`/portaljs-add-dataset`](.claude/commands/portaljs-add-dataset.md) | Add a CSV, TSV, JSON, or GeoJSON dataset β€” registers it in the catalog and renders its showcase automatically; large local files are pushed to Cloudflare R2 via Git LFS for you. |
  147. | [`/portaljs-add-resource`](.claude/commands/portaljs-add-resource.md) | Attach another file (data dictionary, methodology, extra data) to an existing dataset β€” it becomes multi-resource and the showcase renders a section per file. |
  148. | [`/portaljs-add-chart`](.claude/commands/portaljs-add-chart.md) | Add a line, bar, area, pie, or scatter chart to a dataset's showcase. |
  149. | [`/portaljs-add-map`](.claude/commands/portaljs-add-map.md) | Render a GeoJSON dataset on an interactive map and register it on the home page. |
  150. | [`/portaljs-add-geo`](.claude/commands/portaljs-add-geo.md) | Auto-ingest a geospatial file (GeoJSON, Shapefile, GeoPackage, KML/KMZ, FlatGeobuf, CSV-with-geometry) on your own machine β€” no server: normalizes CRS to EPSG:4326, derives a PMTiles render tier and a GeoParquet query tier, pushes both plus the original to R2, and registers one dual-tier dataset the showcase maps and queries in place. |
  151. | [`/portaljs-define-schema`](.claude/commands/portaljs-define-schema.md) | Infer a Frictionless Table Schema from a dataset's data, add license/source/keyword metadata, and surface a typed field table on its showcase. |
  152. | [`/portaljs-add-dcat`](.claude/commands/portaljs-add-dcat.md) | Make the portal harvestable β€” emit standards-compliant DCAT feeds (DCAT 2/3, DCAT-AP, DCAT-US, national profiles) in JSON-LD, Turtle, and RDF/XML so national/EU/US open-data portals can harvest its datasets. |
  153. | [`/portaljs-connect-ckan`](.claude/commands/portaljs-connect-ckan.md) | Wire the portal to a CKAN backend over its API instead of static files. |
  154. | [`/portaljs-check-data-quality`](.claude/commands/portaljs-check-data-quality.md) | Validate a dataset against its schema and flag quality issues (type mismatches, missing values, constraint violations). |
  155. | [`/portaljs-migrate`](.claude/commands/portaljs-migrate.md) | Harvest or migrate a whole catalog into the portal from CKAN, Socrata, OpenDataSoft, ArcGIS, or DCAT-US, over a canonical Frictionless/DCAT model. |
  156. | [`/arcgis-to-portaljs`](.claude/commands/arcgis-to-portaljs.md) | Migrate a whole ArcGIS Hub site into the portal end-to-end β€” harvest its `/data.json`, export every FeatureService layer via the ArcGIS REST API, convert to the serverless dual tier (PMTiles + GeoParquet), push to R2, and write a source-vs-derived parity report. |
  157. | [`/portaljs-deploy`](.claude/commands/portaljs-deploy.md) | Build a static export and publish it to PortalJS Arc β€” Datopian-managed hosting on Cloudflare β€” with a live `<slug>.arc.portaljs.com` URL. |
  158. <!-- END:skills-table -->
  159. <!-- Generated from scripts/skills-manifest.mjs β€” edit there and run `npm run gen:skills`. -->
  160. Large-data scaling β€” big files pushed to Cloudflare R2 via Git LFS β€” already ships in
  161. `/portaljs-add-dataset`. More skill families β€” metadata schemas (Frictionless/DCAT), more
  162. backends (OpenMetadata), a browser DuckDB query layer, and access control β€” are on the
  163. [roadmap](ROADMAP.md). Write your own β€” see [`.claude/AUTHORING.md`](.claude/AUTHORING.md).
  164. ## What's in this repo
  165. ```text
  166. .claude/commands/ the agentic skills (slash commands)
  167. examples/ portaljs-catalog β€” the canonical template
  168. packages/
  169. ckan/ CKAN catalog UI + React (@portaljs/ckan)
  170. ckan-api-client-js/ pure CKAN API client (@portaljs/ckan-api-client-js)
  171. site/ portaljs.com β€” the marketing site + docs
  172. ROADMAP.md direction, the four contracts, sequencing
  173. ```
  174. The canonical template, [`examples/portaljs-catalog`](examples/portaljs-catalog/), is where
  175. the three surfaces and the `DataProvider` seam live β€” read it before building.
  176. ## What makes it different
  177. - 🌱 **Open source, MIT, no lock-in** β€” every skill emits plain Next.js you can fork and own.
  178. - 🧭 **Advisory, not just generative** β€” `/portaljs-architect` helps you _decide_ the infrastructure, not only scaffold a UI.
  179. - πŸ¦† **Open lakehouse by default** β€” git + R2 + Parquet queried with DuckDB, over a heavy warehouse; add DuckLake for living/incremental tables. A datastore/warehouse stays a supported choice.
  180. - ☁️ **Cloudflare-first, portable** β€” R2 / Workers / D1 / Pages as the default substrate, but object storage stays S3-compatible.
  181. - 🧩 **Decoupled, any backend** β€” one `DataProvider` contract in front of [CKAN](https://ckan.org/), [DKAN](https://getdkan.org/), [OpenMetadata](https://open-metadata.org/), [DataHub](https://datahubproject.io/), GitHub, [Frictionless](https://frictionlessdata.io/), plain files β€” or your own.
  182. - 🎨 **Bring your own stack** β€” adopt the template or lift the skills and the three-surface model into an app you already have.
  183. ## Examples
  184. The canonical template lives in [`examples/`](examples/); OpenSpending, built on PortalJS, has its own repo:
  185. | Example | What it is |
  186. |---------|------------|
  187. | [`portaljs-catalog`](examples/portaljs-catalog/) | **Canonical template** β€” Home + Catalog + Showcase over a static manifest. Scaffold it with `npm create portaljs` or `/portaljs-new-portal`. |
  188. | [`openspending.org`](https://github.com/datopian/openspending.org) | Real-world portal behind [OpenSpending](https://www.openspending.org/) β€” now in its own repo |
  189. The earlier 2023-era examples (`portaljs-template`, `ckan`, `ckan-ssg`,
  190. `github-backed-catalog`, `dataset-frictionless`, `fivethirtyeight`, `turing`, `learn`)
  191. were retired; their source is preserved at
  192. [`59d816c`](https://github.com/datopian/portaljs/tree/59d816c73aa3569733ce26cf6c471d24b155d760/examples).
  193. ## Community & support
  194. - πŸ’¬ **Discord** β€” live chat and help: [join the server](https://discord.gg/krmj5HM6He)
  195. - πŸ—£οΈ **Discussions** β€” questions, ideas, show-and-tell: [github.com/datopian/portaljs/discussions](https://github.com/datopian/portaljs/discussions)
  196. - πŸ› **Issues** β€” bugs and feature requests: [open an issue](https://github.com/datopian/portaljs/issues/new)
  197. - πŸ“– **Docs** β€” [portaljs.com/docs](https://www.portaljs.com/docs)
  198. ## Contributing
  199. PortalJS is built in the open and we welcome contributions of all sizes β€” new skills,
  200. examples, docs, and fixes. See [CONTRIBUTING.md](CONTRIBUTING.md) to get started, and read
  201. [ROADMAP.md](ROADMAP.md) and [VISION.md](VISION.md) for where the project is headed.
  202. ## License
  203. [MIT](license) Β© [Datopian](https://www.datopian.com/)
packages/create-portaljs/index.mjs Β· Bundled script Β· 411 lines Β· 1 finding marked
  1. #!/usr/bin/env node
  2. // create-portaljs β€” scaffold a PortalJS data portal.
  3. // npm create portaljs@latest my-portal
  4. // See SPEC.md. Plain ESM, Node >= 22, deps: giget + prompts.
  5. import { existsSync, readdirSync, readFileSync, writeFileSync, statSync, rmSync, mkdtempSync } from 'node:fs'
  6. import { join, basename, resolve } from 'node:path'
  7. import { tmpdir } from 'node:os'
  8. import { spawn, spawnSync } from 'node:child_process'
  9. const TEMPLATE = 'examples/portaljs-catalog'
  10. const TEXT_EXT = new Set(['.ts', '.tsx', '.js', '.mjs', '.json', '.css', '.md', '.html'])
  11. // Where the agentic skills live in the repo. We bundle them into the scaffold's
  12. // project-local .claude/commands/ (so `claude` in the new portal sees
  13. // /portaljs-add-dataset, /portaljs-connect-ckan, … with no separate install step).
  14. const SKILLS_SRC = '.claude/commands'
  15. // The allowlist of skills to bundle comes from the `commands` array of
  16. // .claude-plugin/plugin.json, fetched at the same ref as the template β€” that file
  17. // is the single source of truth (scripts/install-portaljs-skills.sh reads it too).
  18. // Only canonical names are listed there, so the bare alias stubs (kept one release
  19. // for existing installs) are pruned and new projects start clean.
  20. const PLUGIN_MANIFEST_SRC = '.claude-plugin'
  21. // Used only if plugin.json can't be fetched. Must match plugin.json's `commands`
  22. // exactly β€” `npm run skills:check` (CI) fails if it drifts.
  23. const FALLBACK_SKILLS = [
  24. 'portaljs-architect',
  25. 'portaljs-new-portal',
  26. 'portaljs-add-dataset',
  27. 'portaljs-add-resource',
  28. 'portaljs-add-chart',
  29. 'portaljs-add-map',
  30. 'portaljs-add-geo',
  31. 'portaljs-connect-ckan',
  32. 'portaljs-migrate',
  33. 'arcgis-to-portaljs',
  34. 'portaljs-define-schema',
  35. 'portaljs-deploy',
  36. 'portaljs-check-data-quality',
  37. 'portaljs-add-dcat',
  38. ]
  39. // `./.claude/commands/<name>.md` entries of a plugin.json `commands` array β†’ names.
  40. function skillNamesFromPlugin(manifest) {
  41. const names = []
  42. for (const entry of manifest?.commands ?? []) {
  43. const m = /^(?:\.\/)?\.claude\/commands\/([a-z0-9-]+)\.md$/.exec(entry)
  44. if (m) names.push(m[1])
  45. }
  46. return names
  47. }
  48. // Fetch plugin.json at `ref` and return its skill names, or null on any failure.
  49. async function fetchSkillAllowlist(downloadTemplate, ref) {
  50. const tmp = mkdtempSync(join(tmpdir(), 'create-portaljs-'))
  51. try {
  52. await downloadTemplate(`github:datopian/portaljs/${PLUGIN_MANIFEST_SRC}#${ref}`, {
  53. dir: tmp,
  54. force: true,
  55. })
  56. const names = skillNamesFromPlugin(JSON.parse(readFileSync(join(tmp, 'plugin.json'), 'utf8')))
  57. return names.length ? names : null
  58. } catch {
  59. return null
  60. } finally {
  61. rmSync(tmp, { recursive: true, force: true })
  62. }
  63. }
  64. function parseArgs(argv) {
  65. const o = { install: true, git: true, yes: false, ref: 'main' }
  66. for (let i = 0; i < argv.length; i++) {
  67. const a = argv[i]
  68. if (a === '-h' || a === '--help') o.help = true
  69. else if (a === '-y' || a === '--yes') o.yes = true
  70. else if (a === '--no-install') o.install = false
  71. else if (a === '--no-git') o.git = false
  72. else if (a === '--namespace') o.namespace = argv[++i]
  73. else if (a === '--name') o.name = argv[++i]
  74. else if (a === '--description') o.description = argv[++i]
  75. else if (a === '--ref') o.ref = argv[++i]
  76. else if (a.startsWith('--')) {
  77. const [k, v] = a.slice(2).split('=')
  78. if (v !== undefined) o[k] = v
  79. } else if (!o.dir) o.dir = a
  80. }
  81. return o
  82. }
  83. const HELP = `
  84. create-portaljs β€” scaffold a PortalJS data portal
  85. Usage:
  86. npm create portaljs@latest [directory] [options]
  87. Options:
  88. --namespace <theme|owner> Namespace mode (default: theme)
  89. --name <string> Human project name (default: from directory)
  90. --description <string> One-line description
  91. --ref <git-ref> Template ref to fetch (default: main)
  92. --no-install Skip npm install
  93. --no-git Skip git init
  94. -y, --yes Accept defaults, no prompts
  95. -h, --help Show this help
  96. `
  97. const slugify = (s) =>
  98. s
  99. .toLowerCase()
  100. .replace(/[^a-z0-9]+/g, '-')
  101. .replace(/^-+|-+$/g, '') || 'my-portal'
  102. function fail(msg) {
  103. console.error(`\nβœ– ${msg}\n`)
  104. process.exit(1)
  105. }
  106. // Recursively substitute the template's placeholder tokens in text files.
  107. function substitute(dir, replacements) {
  108. for (const entry of readdirSync(dir)) {
  109. if (entry === 'node_modules' || entry === '.git') continue
  110. const p = join(dir, entry)
  111. const st = statSync(p)
  112. if (st.isDirectory()) {
  113. substitute(p, replacements)
  114. } else if (TEXT_EXT.has(p.slice(p.lastIndexOf('.')))) {
  115. let s = readFileSync(p, 'utf8')
  116. let changed = false
  117. for (const [token, value] of replacements) {
  118. if (s.includes(token)) {
  119. s = s.split(token).join(value)
  120. changed = true
  121. }
  122. }
  123. if (changed) writeFileSync(p, s)
  124. }
  125. }
  126. }
  127. function run(cmd, args, cwd) {
  128. const r = spawnSync(cmd, args, { cwd, stdio: 'inherit' })
  129. return r.status === 0
  130. }
  131. // Run a command async, quietly (so it doesn't clobber the animation), capturing
  132. // stderr for error reporting. Resolves { ok, err }.
  133. function runAsync(cmd, args, cwd) {
  134. return new Promise((res) => {
  135. const child = spawn(cmd, args, { cwd, stdio: ['ignore', 'ignore', 'pipe'] })
  136. let err = ''
  137. child.stderr.on('data', (d) => {
  138. err += d.toString()
  139. })
  140. child.on('error', (e) => res({ ok: false, err: `failed to start ${cmd}: ${e.message}` }))
  141. child.on('close', (code) => res({ ok: code === 0, err }))
  142. })
  143. }
  144. // A branded cyclone animation to watch while deps install. Renders in place over
  145. // a few lines, sky→teal gradient (the PortalJS palette), with a flowing funnel,
  146. // a spinner, rotating status, and an elapsed timer. No-ops (one plain line) when
  147. // stdout isn't a TTY, so CI logs stay clean.
  148. function createBuildAnimation() {
  149. if (!process.stdout.isTTY) {
  150. console.log('Installing dependencies…')
  151. return { stop() {} }
  152. }
  153. const SKY = [56, 189, 248]
  154. const TEAL = [94, 234, 212]
  155. const lerp = (a, b, t) => Math.round(a + (b - a) * t)
  156. const grad = (i, n, s) => {
  157. const t = n > 1 ? i / (n - 1) : 0
  158. return `\x1b[38;2;${lerp(SKY[0], TEAL[0], t)};${lerp(SKY[1], TEAL[1], t)};${lerp(SKY[2], TEAL[2], t)}m${s}\x1b[0m`
  159. }
  160. const WAVE = ['≋', 'β‰ˆ', '∿', '~'] // shifts per row β†’ flowing-funnel illusion
  161. const SPIN = ['◐', 'β—“', 'β—‘', 'β—’']
  162. const MSG = [
  163. 'spinning up your portal',
  164. 'wiring Home Β· Catalog Β· Showcase',
  165. 'loading sample datasets',
  166. 'almost ready',
  167. ]
  168. // Funnel rows: [indent, wave-count] narrowing to an apex.
  169. const FUNNEL = [
  170. [6, 5],
  171. [7, 3],
  172. [8, 1],
  173. ]
  174. const HEIGHT = 6 // 1 blank + 4 cyclone + 1 status
  175. const start = Date.now()
  176. let f = 0
  177. // Restore the cursor even if we never reach stop() (crash / SIGINT / SIGTERM),
  178. // otherwise the user's terminal is left with a hidden cursor.
  179. const showCursor = () => process.stdout.write('\x1b[?25h')
  180. const onExit = () => showCursor()
  181. const onSignal = () => {
  182. showCursor()
  183. process.exit(130)
  184. }
  185. process.once('exit', onExit)
  186. process.once('SIGINT', onSignal)
  187. process.once('SIGTERM', onSignal)
  188. process.stdout.write('\x1b[?25l') // hide cursor
  189. process.stdout.write('\n'.repeat(HEIGHT)) // reserve space
  190. const draw = () => {
  191. const secs = Math.floor((Date.now() - start) / 1000)
  192. const lines = ['']
  193. FUNNEL.forEach(([indent, width], ri) => {
  194. const w = WAVE[(f + ri) % WAVE.length]
  195. lines.push(' '.repeat(indent) + grad(ri, FUNNEL.length + 1, 'β•²' + w.repeat(width) + 'β•±'))
  196. })
  197. lines.push(' '.repeat(9) + grad(FUNNEL.length, FUNNEL.length + 1, 'β–Ώ'))
  198. const sp = `\x1b[38;2;${SKY[0]};${SKY[1]};${SKY[2]}m${SPIN[f % SPIN.length]}\x1b[0m`
  199. const msg = MSG[Math.floor(f / 14) % MSG.length]
  200. lines.push(` ${sp} \x1b[1mPortalJS\x1b[0m \x1b[2mΒ· ${msg}… (${secs}s)\x1b[0m`)
  201. process.stdout.write(`\x1b[${HEIGHT}A`) // up to the top of the reserved block
  202. for (const ln of lines.slice(0, HEIGHT)) process.stdout.write('\x1b[2K' + ln + '\n')
  203. f++
  204. }
  205. draw()
  206. const timer = setInterval(draw, 110)
  207. return {
  208. stop(ok) {
  209. clearInterval(timer)
  210. process.removeListener('exit', onExit)
  211. process.removeListener('SIGINT', onSignal)
  212. process.removeListener('SIGTERM', onSignal)
  213. process.stdout.write(`\x1b[${HEIGHT}A`)
  214. for (let i = 0; i < HEIGHT; i++) process.stdout.write('\x1b[2K\n')
  215. process.stdout.write(`\x1b[${HEIGHT}A\x1b[?25h`) // back to top, show cursor
  216. const secs = Math.floor((Date.now() - start) / 1000)
  217. console.log(
  218. ok
  219. ? `\x1b[38;2;94;234;212mβœ”\x1b[0m Dependencies installed \x1b[2m(${secs}s)\x1b[0m`
  220. : '\x1b[33m⚠\x1b[0m Dependency install did not complete'
  221. )
  222. },
  223. }
  224. }
  225. async function main() {
  226. const opts = parseArgs(process.argv.slice(2))
  227. if (opts.help) {
  228. console.log(HELP)
  229. return
  230. }
  231. // Lazy-load deps so --help works even before install resolves them.
  232. const prompts = (await import('prompts')).default
  233. const { downloadTemplate } = await import('giget')
  234. const onCancel = () => fail('Cancelled.')
  235. const ask = async (questions) =>
  236. opts.yes ? {} : prompts(questions, { onCancel })
  237. // 1. Resolve inputs (CLI flags win; otherwise prompt unless --yes).
  238. let dir = opts.dir
  239. if (!dir) {
  240. const a = await ask({
  241. type: 'text',
  242. name: 'dir',
  243. message: 'Project directory',
  244. initial: 'my-portal',
  245. })
  246. dir = a.dir || 'my-portal'
  247. }
  248. const slug = slugify(basename(dir))
  249. const name =
  250. opts.name ??
  251. (await ask({
  252. type: 'text',
  253. name: 'name',
  254. message: 'Project name',
  255. initial: slug.replace(/-/g, ' ').replace(/\b\w/g, (c) => c.toUpperCase()),
  256. })).name ??
  257. slug
  258. const description =
  259. opts.description ??
  260. (await ask({
  261. type: 'text',
  262. name: 'description',
  263. message: 'One-line description',
  264. initial: 'An open data portal.',
  265. })).description ??
  266. 'An open data portal.'
  267. const namespace =
  268. opts.namespace ??
  269. (await ask({
  270. type: 'select',
  271. name: 'namespace',
  272. message: 'Namespace mode',
  273. choices: [
  274. { title: 'theme β€” single publisher, group by subject', value: 'theme' },
  275. { title: 'owner β€” multiple publishers, group by who published', value: 'owner' },
  276. ],
  277. initial: 0,
  278. })).namespace ??
  279. 'theme'
  280. if (namespace !== 'theme' && namespace !== 'owner') fail(`Invalid --namespace: ${namespace}`)
  281. const doInstall = opts.install &&
  282. (opts.yes || (await ask({ type: 'confirm', name: 'v', message: 'Install dependencies?', initial: true })).v !== false)
  283. const doGit = opts.git &&
  284. (opts.yes || (await ask({ type: 'confirm', name: 'v', message: 'Initialize a git repo?', initial: true })).v !== false)
  285. // 2. Guard the target directory.
  286. const target = resolve(process.cwd(), dir)
  287. if (existsSync(target) && readdirSync(target).length > 0) {
  288. fail(`Directory "${dir}" already exists and is not empty. Choose another name.`)
  289. }
  290. // 3. Fetch the template (reliable subdir extraction; no degit fallback bug).
  291. console.log(`\nScaffolding ${name} β†’ ${dir} (template @ ${opts.ref})`)
  292. try {
  293. await downloadTemplate(`github:datopian/portaljs/${TEMPLATE}#${opts.ref}`, {
  294. dir: target,
  295. force: true,
  296. })
  297. } catch (e) {
  298. fail(`Failed to fetch the template: ${e?.message || e}`)
  299. }
  300. // 4. Substitute placeholder tokens.
  301. substitute(target, [
  302. ['__PROJECT_NAME__', name],
  303. ['__PROJECT_SLUG__', slug],
  304. ['__DESCRIPTION__', description],
  305. ])
  306. // 5. Set the namespace mode in lib/datasets.ts.
  307. const datasetsTs = join(target, 'lib', 'datasets.ts')
  308. if (existsSync(datasetsTs)) {
  309. const s = readFileSync(datasetsTs, 'utf8').replace(
  310. /export const NAMESPACE_TYPE: 'theme' \| 'owner' = '(theme|owner)'/,
  311. `export const NAMESPACE_TYPE: 'theme' | 'owner' = '${namespace}'`
  312. )
  313. writeFileSync(datasetsTs, s)
  314. }
  315. // 5b. Bundle the agentic skills into the project's .claude/commands/ so they
  316. // work the instant the user runs `claude` in the new portal β€” no global
  317. // install. Pinned to the same template ref, so skills match the scaffold.
  318. // The repo's .claude/commands holds only OSS skills today; we still prune to
  319. // the allowlist so a future internal command never leaks into a scaffold.
  320. // Non-fatal: a fetch failure just prints the manual install one-liner.
  321. const skillsDir = join(target, '.claude', 'commands')
  322. try {
  323. let skills = await fetchSkillAllowlist(downloadTemplate, opts.ref)
  324. if (!skills) {
  325. console.log('\x1b[2mCould not read .claude-plugin/plugin.json; using the built-in skill list.\x1b[0m')
  326. skills = FALLBACK_SKILLS
  327. }
  328. await downloadTemplate(`github:datopian/portaljs/${SKILLS_SRC}#${opts.ref}`, {
  329. dir: skillsDir,
  330. force: true,
  331. })
  332. const bundled = []
  333. for (const entry of readdirSync(skillsDir)) {
  334. const keep = entry.endsWith('.md') && skills.includes(entry.slice(0, -3))
  335. if (keep) bundled.push(entry.slice(0, -3))
  336. else rmSync(join(skillsDir, entry), { recursive: true, force: true })
  337. }
  338. console.log(`Bundled ${bundled.length} PortalJS skills into .claude/commands/`)
  339. // A listed skill missing from the download (e.g. an older --ref) isn't fatal, but
  340. // say which ones so the scaffold doesn't silently ship fewer than expected.
  341. const missing = skills.filter((s) => !bundled.includes(s))
  342. if (missing.length)
  343. console.log(`\x1b[33m⚠\x1b[0m Not found at ${opts.ref}: ${missing.map((s) => '/' + s).join(', ')}`)
  344. } catch (e) {
  345. console.log(
  346. `\x1b[33m⚠\x1b[0m Could not bundle skills (${e?.message || e}). Install them with:\n` +
  347. ' curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash'
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 372 on GitHub
  348. )
  349. }
  350. // 6. Optional git init + install.
  351. if (doGit && !existsSync(join(target, '.git'))) {
  352. if (run('git', ['init', '-q'], target)) {
  353. run('git', ['add', '-A'], target)
  354. run('git', ['commit', '-q', '-m', 'chore: scaffold PortalJS portal'], target)
  355. }
  356. }
  357. if (doInstall) {
  358. const anim = createBuildAnimation()
  359. const { ok, err } = await runAsync('npm', ['install'], target)
  360. anim.stop(ok)
  361. if (!ok) {
  362. console.log(' Run `npm install` yourself in the project to finish.')
  363. if (err) {
  364. console.log('\x1b[2m' + err.trim().split('\n').slice(-3).join('\n') + '\x1b[0m')
  365. }
  366. }
  367. }
  368. // 7. Next steps.
  369. console.log(`
  370. βœ” Created ${name} in ${dir}
  371. Next:
  372. cd ${dir}${doInstall ? '' : '\n npm install'}
  373. npm run dev # β†’ http://localhost:3000
  374. Then run \`claude\` in the project β€” the PortalJS skills are bundled in
  375. .claude/commands/, so these work right away:
  376. /portaljs-add-dataset ./data/your-file.csv add data (or /portaljs-add-resource for more files)
  377. /portaljs-connect-ckan <ckan-url> point at a CKAN backend instead
  378. /portaljs-deploy publish (Cloudflare Pages recommended)
  379. `)
  380. }
  381. main().catch((e) => fail(e?.message || String(e)))
scripts/install-portaljs-skills.sh Β· Bundled script Β· 83 lines Β· 1 finding marked
  1. #!/usr/bin/env bash
  2. # Install PortalJS agentic skills into your personal Claude Code commands dir.
  3. #
  4. # Usage:
  5. # # From a clone of the repo (copies local files):
  6. # ./scripts/install-portaljs-skills.sh
  7. #
  8. # # From anywhere (downloads from GitHub):
  9. # curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash
    • Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
    • Line 9 on GitHub
  10. #
  11. # Env overrides:
  12. # CLAUDE_COMMANDS_DIR target dir (default: $HOME/.claude/commands)
  13. # PORTALJS_SKILLS_REF git ref to download from when remote (default: main)
  14. #
  15. # Idempotent: re-running overwrites the installed skill files in place.
  16. set -euo pipefail
  17. REF="${PORTALJS_SKILLS_REF:-main}"
  18. DEST="${CLAUDE_COMMANDS_DIR:-$HOME/.claude/commands}"
  19. RAW_REPO="https://raw.githubusercontent.com/datopian/portaljs/${REF}"
  20. RAW_BASE="$RAW_REPO/.claude/commands"
  21. # OSS skills only β€” excludes Gas Town internal commands (done/handoff/review).
  22. # The canonical skill names are read from the `commands` array of
  23. # .claude-plugin/plugin.json, the single source of truth (create-portaljs reads
  24. # it too), so this installer can't drift from the plugin.
  25. # The bare old names below are thin alias stubs kept for one minor release
  26. # (back-compat) β€” installed too so existing `/deploy`-style invocations keep
  27. # working this release. (`login` was removed.)
  28. ALIASES="architect new-portal add-dataset add-resource add-chart add-map connect-ckan migrate define-schema deploy check-data-quality"
  29. # `"./.claude/commands/<name>.md"` entries of plugin.json -> `<name>`, one per line.
  30. # Plain grep/sed so the installer doesn't need jq.
  31. skill_names() {
  32. { grep -o '\.claude/commands/[a-z0-9-]*\.md' || true; } |
  33. sed -e 's#^\.claude/commands/##' -e 's#\.md$##'
  34. }
  35. # Detect a local checkout: this script lives in <repo>/scripts/, so the commands
  36. # dir is ../.claude/commands relative to the script.
  37. SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" >/dev/null 2>&1 && pwd)"
  38. LOCAL_COMMANDS="$SCRIPT_DIR/../.claude/commands"
  39. LOCAL_PLUGIN="$SCRIPT_DIR/../.claude-plugin/plugin.json"
  40. if [ -f "$LOCAL_COMMANDS/portaljs-new-portal.md" ] && [ -f "$LOCAL_PLUGIN" ]; then
  41. MODE=local
  42. CANONICAL="$(skill_names < "$LOCAL_PLUGIN")"
  43. else
  44. MODE=remote
  45. if ! MANIFEST="$(curl -fsSL "$RAW_REPO/.claude-plugin/plugin.json")"; then
  46. echo "error: could not download .claude-plugin/plugin.json from github.com/datopian/portaljs@$REF" >&2
  47. exit 1
  48. fi
  49. CANONICAL="$(printf '%s\n' "$MANIFEST" | skill_names)"
  50. fi
  51. if [ -z "$CANONICAL" ]; then
  52. echo "error: no skills found in the 'commands' list of .claude-plugin/plugin.json" >&2
  53. exit 1
  54. fi
  55. # Word-split on whitespace: names are [a-z0-9-] only.
  56. SKILLS="$(echo $CANONICAL $ALIASES)"
  57. mkdir -p "$DEST"
  58. if [ "$MODE" = local ]; then
  59. echo "Installing PortalJS skills from local checkout -> $DEST"
  60. for s in $SKILLS; do
  61. cp "$LOCAL_COMMANDS/$s.md" "$DEST/$s.md"
  62. echo " + $s.md"
  63. done
  64. else
  65. echo "Installing PortalJS skills from github.com/datopian/portaljs@$REF -> $DEST"
  66. for s in $SKILLS; do
  67. curl -fsSL "$RAW_BASE/$s.md" -o "$DEST/$s.md"
  68. echo " + $s.md"
  69. done
  70. fi
  71. echo ""
  72. echo "Done. Installed: $SKILLS"
  73. echo "Restart Claude Code (or open a new session) and run /portaljs-new-portal to start."

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.