portaljs
π AI-native framework for building data portals. Scaffold a full portal from a brief and load datasets in minutes with agentic skills β any backend (CKAN, GitHub, Frictionless).
Safety scan
A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.
Check it deeper on your machine
Its scripts were only partly scanned.
As a Claude Code plugin, install it and then ask Claude Code:
/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/datopian/portaljs with scanskillsafety
Or from a terminal, clone it and run it:
git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/datopian/portaljs'
It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.
Badge for your README
Markdown
[](/listings/datopian--portaljs)HTML
<a href="/listings/datopian--portaljs"><img src="/api/listings/datopian--portaljs/badge.svg" alt="findsafeskills safety rating" /></a>Install
Install instructions from the README
As written by the repo's author. findsafeskills has not verified these steps.
These steps run a script from the network that findsafeskills does not scan.
**Create a portal** β one command, nothing to install beyond Node 22+:
npm create portaljs@latest my-portal cd my-portal npm run dev # β http://localhost:3000
You get the three surfaces β Home, a Catalog (`/search`), and a dataset Showcase (`/@<namespace>/<slug>`) β over sample data. Plain, editable Next.js, no lock-in. Add your own CSV/JSON to `datasets.json` and it renders automatically.
**Build it with your AI assistant** β PortalJS ships [Claude Code](https://claude.com/claude-code) skills that do the assembly. Install them once (into `~/.claude/commands`):
curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash
Then, in a Claude Code session from any directory:
/portaljs-architect not sure what stack you need? start here /portaljs-new-portal "Auckland Council open data portal" /portaljs-add-dataset ./data/air-quality.csv
`/portaljs-new-portal` scaffolds the three surfaces; `/portaljs-add-dataset` (or `/portaljs-add-resource`) loads data; `/portaljs-connect-ckan` points it at a CKAN backend; `/portaljs-deploy` ships it. ([All skills + install β](.claude/INSTALL.md))
**Prefer the bare template** β plain Next.js, no AI, no lock-in:
npx tiged datopian/portaljs/examples/portaljs-catalog my-portal cd my-portal && npm install && npm run dev # β http://localhost:3000
You get Home, a Catalog (`/search`), and a dataset Showcase (`/@<namespace>/<slug>`) over sample data. Add your own CSV/JSON to `datasets.json` and it renders automatically.
β If it's useful, a star helps others find it.
What was scanned
The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.
.claude-plugin/marketplace.json Β· Manifest Β· 18 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- {
- "name": "datopian-portaljs",
- "owner": {
- "name": "Datopian",
- "url": "https://www.datopian.com"
- },
- "metadata": {
- "description": "PortalJS agentic skills β build data portals with AI.",
- "version": "0.1.0"
- },
- "plugins": [
- {
- "name": "portaljs",
- "source": ".",
- "description": "Scaffold and extend PortalJS data portals: /portaljs-new-portal, /portaljs-add-dataset, /portaljs-add-chart, /portaljs-add-map, /portaljs-deploy."
- }
- ]
- }
README.md Β· README Β· 263 lines
View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)
This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.
- <p align="center">
- <img src="assets/portaljs-logo-spin.svg" alt="PortalJS" width="96" height="96" />
- <h1 align="center">PortalJS</h1>
- <p align="center">
- <b>The AI-native framework for building data portals.</b>
- <br />
- Describe the portal you want β your agent helps you choose an architecture, scaffolds it, and loads your data.
- <br />
- <br />
- <a href="https://www.portaljs.com/docs">Docs</a>
- Β·
- <a href="https://github.com/datopian/portaljs/discussions">Discussions</a>
- Β·
- <a href="https://github.com/datopian/portaljs/issues/new">Report a bug</a>
- <br />
- <br />
- <a href="https://www.npmjs.com/package/create-portaljs"><img src="https://img.shields.io/npm/v/create-portaljs?logo=npm&logoColor=white&label=create-portaljs&color=cb3837" alt="npm version" /></a>
- <a href="https://github.com/datopian/portaljs/stargazers"><img src="https://img.shields.io/github/stars/datopian/portaljs?logo=github&logoColor=white&label=Stars&color=blue" alt="GitHub stars" /></a>
- <a href="https://discord.gg/krmj5HM6He"><img src="https://img.shields.io/badge/Discord-Join-5865F2?logo=discord&logoColor=white" alt="Join our Discord" /></a>
- <a href="license"><img src="https://img.shields.io/badge/License-MIT-blue" alt="MIT License" /></a>
- </p>
- </p>
- ---
- ## Quickstart
- **Create a portal** β one command, nothing to install beyond Node 22+:
- ```bash
- npm create portaljs@latest my-portal
- cd my-portal
- npm run dev # β http://localhost:3000
- ```
- You get the three surfaces β Home, a Catalog (`/search`), and a dataset Showcase
- (`/@<namespace>/<slug>`) β over sample data. Plain, editable Next.js, no lock-in. Add your
- own CSV/JSON to `datasets.json` and it renders automatically.
- **Build it with your AI assistant** β PortalJS ships [Claude Code](https://claude.com/claude-code)
- skills that do the assembly. Install them once (into `~/.claude/commands`):
- ```bash
- curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash
- ```
- Then, in a Claude Code session from any directory:
- ```text
- /portaljs-architect not sure what stack you need? start here
- /portaljs-new-portal "Auckland Council open data portal"
- /portaljs-add-dataset ./data/air-quality.csv
- ```
- `/portaljs-new-portal` scaffolds the three surfaces; `/portaljs-add-dataset` (or `/portaljs-add-resource`) loads data;
- `/portaljs-connect-ckan` points it at a CKAN backend; `/portaljs-deploy` ships it. ([All skills + install β](.claude/INSTALL.md))
- **Prefer the bare template** β plain Next.js, no AI, no lock-in:
- ```bash
- npx tiged datopian/portaljs/examples/portaljs-catalog my-portal
- cd my-portal && npm install && npm run dev # β http://localhost:3000
- ```
- You get Home, a Catalog (`/search`), and a dataset Showcase (`/@<namespace>/<slug>`) over
- sample data. Add your own CSV/JSON to `datasets.json` and it renders automatically.
- β If it's useful, a star helps others find it.
- ## Why PortalJS
- Building a data portal has always meant more than a website. You have to decide where
- the data lives, how it's versioned, how people search it, how it's served, and how it's
- governed β and then wire a frontend on top. Teams either over-build on a heavy data
- warehouse they don't need, or under-build on a pile of scripts that doesn't scale.
- PortalJS is an **open-source, agentic skills framework that helps data teams build,
- develop, and ship data portals β and the data infrastructure underneath them.** It isn't
- only a frontend. The skills do two jobs:
- - **Advise** β given _what you're building_, _what your data is_, and _what it's for_,
- they recommend an architecture: storage, compute, catalog, access, hosting, metadata.
- - **Build** β they scaffold that stack as plain, editable Next.js code with no lock-in.
- It is **opinionated but open**: the recommended modern path is git + object storage
- (Cloudflare R2) + Parquet, queried with [DuckDB](https://duckdb.org/) β an open lakehouse
- instead of a classic warehouse. For living, incremental tables you can layer on
- [DuckLake](https://ducklake.select/), and a traditional datastore (CKAN, a warehouse)
- stays a first-class option when you need it. You always own plain code.
- Built and maintained in the open by [Datopian](https://www.datopian.com/) and the
- PortalJS community.
- ## Architecture at a glance
- ```text
- π§ you describe what you want to build
- β
- βΌ
- ββ π€ AGENTIC SKILLS ββββββββββββββββββββββββββββββββββ decide + build
- β /portaljs-architect Β· /portaljs-new-portal Β· /portaljs-add-dataset Β· /portaljs-add-chart Β· /portaljs-add-map β¦
- β°β generates plain, editable Next.js code β no lock-in
- β
- βΌ
- ββ π₯οΈ SURFACES ββββββββββββββββββββββββββββββββββββββββ what users see
- β π Home / π Catalog /search π Showcase /@ns/slug
- β°β read data through one DataProvider contract
- β
- βΌ
- ββ π PROVIDERS βββββββββββββββββββββββββββββββββββββββ pluggable backends
- β π staticΒ·git π CKAN π OpenMetadata ποΈ git-LFS + R2
- β°β swap the source without touching a page
- β
- βΌ
- π¦ STORAGE + COMPUTE β choose your point on the spectrum:
- flat files ββΆ Git-LFS + R2 ββΆ Parquet on R2 + π¦ DuckDB ββΆ warehouse / CKAN
- simplest β open lakehouse (default) heaviest
- (+ DuckLake for living tables)
- βοΈ Substrate β Cloudflare R2 (storage) Β· Workers (runtime) Β· D1 (catalog) Β· Pages (static)
- object storage stays S3-compatible β R2 is the default, never a lock-in
- ```
- **Three surfaces.** Every data portal is built from three: a **Home** page that explains
- it and offers search, a **Catalog** (`/search`) to discover datasets, and a **Showcase**
- (`/@<namespace>/<slug>`) to explore one dataset β metadata, preview, download/API, and
- charts/maps. ([Core concepts β](https://www.portaljs.com/docs/core-concepts))
- **One seam.** The surfaces read data only through a `DataProvider`, so the source β static
- files today, a CKAN or lakehouse backend tomorrow β can change without touching a page.
- See [`ROADMAP.md`](ROADMAP.md) for the full model and the
- [architecture decision framework](https://www.portaljs.com/docs/architecture/decision-framework)
- for how `/portaljs-architect` turns your needs into a stack.
- ## Build a portal with your AI assistant
- PortalJS ships [Claude Code](https://claude.com/claude-code) skills that turn a brief into
- a working portal.
- ### Setup
- Install the skills once into your personal scope so they're available from **any**
- directory:
- ```bash
- curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash
- ```
- Restart Claude Code (or open a new session) and type `/` to see them. See
- [`.claude/INSTALL.md`](.claude/INSTALL.md) for other install options (versioned plugin, or
- running straight from a clone of this repo).
- ### Use
- If you're not sure how to set up your portal, start with the advisor, then build:
- ```text
- /portaljs-architect we have ~200 public CSVs, updated quarterly, and must publish DCAT-AP
- /portaljs-new-portal "Auckland Council open data portal"
- /portaljs-add-dataset ./data/air-quality.csv
- /portaljs-add-dataset https://example.com/parks.geojson
- ```
- The skills are **interactive** β if your brief is thin, they interview you in short rounds
- rather than erroring. `/portaljs-architect` recommends a stack and hands off; `/portaljs-new-portal`
- scaffolds the three surfaces; `/portaljs-add-dataset` appends to the `datasets.json` manifest and
- the showcase renders automatically at `/@<namespace>/<slug>`. Run `npm run dev` and you
- have a portal.
- **Prefer to build by hand?** The skills are a convenience, not a requirement β scaffold the
- template directly with the CLI:
- ```bash
- npm create portaljs@latest my-portal
- ```
- (Or grab the bare template with no prompts: `npx tiged datopian/portaljs/examples/portaljs-catalog my-portal`.)
- ### Available skills
- <!-- BEGIN:skills-table -->
- | Skill | What it does |
- | ----- | ------------ |
- | [`/portaljs-architect`](.claude/commands/portaljs-architect.md) | Advisory β turns your needs (data, scale, governance) into a recommended architecture before you build. Start here if you're unsure of the stack. |
- | [`/portaljs-new-portal`](.claude/commands/portaljs-new-portal.md) | Scaffold a new portal (Home + Catalog + Showcase) from a brief β copies the template, substitutes your project name and description, installs deps, verifies the build. |
- | [`/portaljs-add-dataset`](.claude/commands/portaljs-add-dataset.md) | Add a CSV, TSV, JSON, or GeoJSON dataset β registers it in the catalog and renders its showcase automatically; large local files are pushed to Cloudflare R2 via Git LFS for you. |
- | [`/portaljs-add-resource`](.claude/commands/portaljs-add-resource.md) | Attach another file (data dictionary, methodology, extra data) to an existing dataset β it becomes multi-resource and the showcase renders a section per file. |
- | [`/portaljs-add-chart`](.claude/commands/portaljs-add-chart.md) | Add a line, bar, area, pie, or scatter chart to a dataset's showcase. |
- | [`/portaljs-add-map`](.claude/commands/portaljs-add-map.md) | Render a GeoJSON dataset on an interactive map and register it on the home page. |
- | [`/portaljs-add-geo`](.claude/commands/portaljs-add-geo.md) | Auto-ingest a geospatial file (GeoJSON, Shapefile, GeoPackage, KML/KMZ, FlatGeobuf, CSV-with-geometry) on your own machine β no server: normalizes CRS to EPSG:4326, derives a PMTiles render tier and a GeoParquet query tier, pushes both plus the original to R2, and registers one dual-tier dataset the showcase maps and queries in place. |
- | [`/portaljs-define-schema`](.claude/commands/portaljs-define-schema.md) | Infer a Frictionless Table Schema from a dataset's data, add license/source/keyword metadata, and surface a typed field table on its showcase. |
- | [`/portaljs-add-dcat`](.claude/commands/portaljs-add-dcat.md) | Make the portal harvestable β emit standards-compliant DCAT feeds (DCAT 2/3, DCAT-AP, DCAT-US, national profiles) in JSON-LD, Turtle, and RDF/XML so national/EU/US open-data portals can harvest its datasets. |
- | [`/portaljs-connect-ckan`](.claude/commands/portaljs-connect-ckan.md) | Wire the portal to a CKAN backend over its API instead of static files. |
- | [`/portaljs-check-data-quality`](.claude/commands/portaljs-check-data-quality.md) | Validate a dataset against its schema and flag quality issues (type mismatches, missing values, constraint violations). |
- | [`/portaljs-migrate`](.claude/commands/portaljs-migrate.md) | Harvest or migrate a whole catalog into the portal from CKAN, Socrata, OpenDataSoft, ArcGIS, or DCAT-US, over a canonical Frictionless/DCAT model. |
- | [`/arcgis-to-portaljs`](.claude/commands/arcgis-to-portaljs.md) | Migrate a whole ArcGIS Hub site into the portal end-to-end β harvest its `/data.json`, export every FeatureService layer via the ArcGIS REST API, convert to the serverless dual tier (PMTiles + GeoParquet), push to R2, and write a source-vs-derived parity report. |
- | [`/portaljs-deploy`](.claude/commands/portaljs-deploy.md) | Build a static export and publish it to PortalJS Arc β Datopian-managed hosting on Cloudflare β with a live `<slug>.arc.portaljs.com` URL. |
- <!-- END:skills-table -->
- <!-- Generated from scripts/skills-manifest.mjs β edit there and run `npm run gen:skills`. -->
- Large-data scaling β big files pushed to Cloudflare R2 via Git LFS β already ships in
- `/portaljs-add-dataset`. More skill families β metadata schemas (Frictionless/DCAT), more
- backends (OpenMetadata), a browser DuckDB query layer, and access control β are on the
- [roadmap](ROADMAP.md). Write your own β see [`.claude/AUTHORING.md`](.claude/AUTHORING.md).
- ## What's in this repo
- ```text
- .claude/commands/ the agentic skills (slash commands)
- examples/ portaljs-catalog β the canonical template
- packages/
- ckan/ CKAN catalog UI + React (@portaljs/ckan)
- ckan-api-client-js/ pure CKAN API client (@portaljs/ckan-api-client-js)
- site/ portaljs.com β the marketing site + docs
- ROADMAP.md direction, the four contracts, sequencing
- ```
- The canonical template, [`examples/portaljs-catalog`](examples/portaljs-catalog/), is where
- the three surfaces and the `DataProvider` seam live β read it before building.
- ## What makes it different
- - π± **Open source, MIT, no lock-in** β every skill emits plain Next.js you can fork and own.
- - π§ **Advisory, not just generative** β `/portaljs-architect` helps you _decide_ the infrastructure, not only scaffold a UI.
- - π¦ **Open lakehouse by default** β git + R2 + Parquet queried with DuckDB, over a heavy warehouse; add DuckLake for living/incremental tables. A datastore/warehouse stays a supported choice.
- - βοΈ **Cloudflare-first, portable** β R2 / Workers / D1 / Pages as the default substrate, but object storage stays S3-compatible.
- - π§© **Decoupled, any backend** β one `DataProvider` contract in front of [CKAN](https://ckan.org/), [DKAN](https://getdkan.org/), [OpenMetadata](https://open-metadata.org/), [DataHub](https://datahubproject.io/), GitHub, [Frictionless](https://frictionlessdata.io/), plain files β or your own.
- - π¨ **Bring your own stack** β adopt the template or lift the skills and the three-surface model into an app you already have.
- ## Examples
- The canonical template lives in [`examples/`](examples/); OpenSpending, built on PortalJS, has its own repo:
- | Example | What it is |
- |---------|------------|
- | [`portaljs-catalog`](examples/portaljs-catalog/) | **Canonical template** β Home + Catalog + Showcase over a static manifest. Scaffold it with `npm create portaljs` or `/portaljs-new-portal`. |
- | [`openspending.org`](https://github.com/datopian/openspending.org) | Real-world portal behind [OpenSpending](https://www.openspending.org/) β now in its own repo |
- The earlier 2023-era examples (`portaljs-template`, `ckan`, `ckan-ssg`,
- `github-backed-catalog`, `dataset-frictionless`, `fivethirtyeight`, `turing`, `learn`)
- were retired; their source is preserved at
- [`59d816c`](https://github.com/datopian/portaljs/tree/59d816c73aa3569733ce26cf6c471d24b155d760/examples).
- ## Community & support
- - π¬ **Discord** β live chat and help: [join the server](https://discord.gg/krmj5HM6He)
- - π£οΈ **Discussions** β questions, ideas, show-and-tell: [github.com/datopian/portaljs/discussions](https://github.com/datopian/portaljs/discussions)
- - π **Issues** β bugs and feature requests: [open an issue](https://github.com/datopian/portaljs/issues/new)
- - π **Docs** β [portaljs.com/docs](https://www.portaljs.com/docs)
- ## Contributing
- PortalJS is built in the open and we welcome contributions of all sizes β new skills,
- examples, docs, and fixes. See [CONTRIBUTING.md](CONTRIBUTING.md) to get started, and read
- [ROADMAP.md](ROADMAP.md) and [VISION.md](VISION.md) for where the project is headed.
- ## License
- [MIT](license) Β© [Datopian](https://www.datopian.com/)
packages/create-portaljs/index.mjs Β· Bundled script Β· 411 lines Β· 1 finding marked
- #!/usr/bin/env node
- // create-portaljs β scaffold a PortalJS data portal.
- // npm create portaljs@latest my-portal
- // See SPEC.md. Plain ESM, Node >= 22, deps: giget + prompts.
- import { existsSync, readdirSync, readFileSync, writeFileSync, statSync, rmSync, mkdtempSync } from 'node:fs'
- import { join, basename, resolve } from 'node:path'
- import { tmpdir } from 'node:os'
- import { spawn, spawnSync } from 'node:child_process'
- const TEMPLATE = 'examples/portaljs-catalog'
- const TEXT_EXT = new Set(['.ts', '.tsx', '.js', '.mjs', '.json', '.css', '.md', '.html'])
- // Where the agentic skills live in the repo. We bundle them into the scaffold's
- // project-local .claude/commands/ (so `claude` in the new portal sees
- // /portaljs-add-dataset, /portaljs-connect-ckan, β¦ with no separate install step).
- const SKILLS_SRC = '.claude/commands'
- // The allowlist of skills to bundle comes from the `commands` array of
- // .claude-plugin/plugin.json, fetched at the same ref as the template β that file
- // is the single source of truth (scripts/install-portaljs-skills.sh reads it too).
- // Only canonical names are listed there, so the bare alias stubs (kept one release
- // for existing installs) are pruned and new projects start clean.
- const PLUGIN_MANIFEST_SRC = '.claude-plugin'
- // Used only if plugin.json can't be fetched. Must match plugin.json's `commands`
- // exactly β `npm run skills:check` (CI) fails if it drifts.
- const FALLBACK_SKILLS = [
- 'portaljs-architect',
- 'portaljs-new-portal',
- 'portaljs-add-dataset',
- 'portaljs-add-resource',
- 'portaljs-add-chart',
- 'portaljs-add-map',
- 'portaljs-add-geo',
- 'portaljs-connect-ckan',
- 'portaljs-migrate',
- 'arcgis-to-portaljs',
- 'portaljs-define-schema',
- 'portaljs-deploy',
- 'portaljs-check-data-quality',
- 'portaljs-add-dcat',
- ]
- // `./.claude/commands/<name>.md` entries of a plugin.json `commands` array β names.
- function skillNamesFromPlugin(manifest) {
- const names = []
- for (const entry of manifest?.commands ?? []) {
- const m = /^(?:\.\/)?\.claude\/commands\/([a-z0-9-]+)\.md$/.exec(entry)
- if (m) names.push(m[1])
- }
- return names
- }
- // Fetch plugin.json at `ref` and return its skill names, or null on any failure.
- async function fetchSkillAllowlist(downloadTemplate, ref) {
- const tmp = mkdtempSync(join(tmpdir(), 'create-portaljs-'))
- try {
- await downloadTemplate(`github:datopian/portaljs/${PLUGIN_MANIFEST_SRC}#${ref}`, {
- dir: tmp,
- force: true,
- })
- const names = skillNamesFromPlugin(JSON.parse(readFileSync(join(tmp, 'plugin.json'), 'utf8')))
- return names.length ? names : null
- } catch {
- return null
- } finally {
- rmSync(tmp, { recursive: true, force: true })
- }
- }
- function parseArgs(argv) {
- const o = { install: true, git: true, yes: false, ref: 'main' }
- for (let i = 0; i < argv.length; i++) {
- const a = argv[i]
- if (a === '-h' || a === '--help') o.help = true
- else if (a === '-y' || a === '--yes') o.yes = true
- else if (a === '--no-install') o.install = false
- else if (a === '--no-git') o.git = false
- else if (a === '--namespace') o.namespace = argv[++i]
- else if (a === '--name') o.name = argv[++i]
- else if (a === '--description') o.description = argv[++i]
- else if (a === '--ref') o.ref = argv[++i]
- else if (a.startsWith('--')) {
- const [k, v] = a.slice(2).split('=')
- if (v !== undefined) o[k] = v
- } else if (!o.dir) o.dir = a
- }
- return o
- }
- const HELP = `
- create-portaljs β scaffold a PortalJS data portal
- Usage:
- npm create portaljs@latest [directory] [options]
- Options:
- --namespace <theme|owner> Namespace mode (default: theme)
- --name <string> Human project name (default: from directory)
- --description <string> One-line description
- --ref <git-ref> Template ref to fetch (default: main)
- --no-install Skip npm install
- --no-git Skip git init
- -y, --yes Accept defaults, no prompts
- -h, --help Show this help
- `
- const slugify = (s) =>
- s
- .toLowerCase()
- .replace(/[^a-z0-9]+/g, '-')
- .replace(/^-+|-+$/g, '') || 'my-portal'
- function fail(msg) {
- console.error(`\nβ ${msg}\n`)
- process.exit(1)
- }
- // Recursively substitute the template's placeholder tokens in text files.
- function substitute(dir, replacements) {
- for (const entry of readdirSync(dir)) {
- if (entry === 'node_modules' || entry === '.git') continue
- const p = join(dir, entry)
- const st = statSync(p)
- if (st.isDirectory()) {
- substitute(p, replacements)
- } else if (TEXT_EXT.has(p.slice(p.lastIndexOf('.')))) {
- let s = readFileSync(p, 'utf8')
- let changed = false
- for (const [token, value] of replacements) {
- if (s.includes(token)) {
- s = s.split(token).join(value)
- changed = true
- }
- }
- if (changed) writeFileSync(p, s)
- }
- }
- }
- function run(cmd, args, cwd) {
- const r = spawnSync(cmd, args, { cwd, stdio: 'inherit' })
- return r.status === 0
- }
- // Run a command async, quietly (so it doesn't clobber the animation), capturing
- // stderr for error reporting. Resolves { ok, err }.
- function runAsync(cmd, args, cwd) {
- return new Promise((res) => {
- const child = spawn(cmd, args, { cwd, stdio: ['ignore', 'ignore', 'pipe'] })
- let err = ''
- child.stderr.on('data', (d) => {
- err += d.toString()
- })
- child.on('error', (e) => res({ ok: false, err: `failed to start ${cmd}: ${e.message}` }))
- child.on('close', (code) => res({ ok: code === 0, err }))
- })
- }
- // A branded cyclone animation to watch while deps install. Renders in place over
- // a few lines, skyβteal gradient (the PortalJS palette), with a flowing funnel,
- // a spinner, rotating status, and an elapsed timer. No-ops (one plain line) when
- // stdout isn't a TTY, so CI logs stay clean.
- function createBuildAnimation() {
- if (!process.stdout.isTTY) {
- console.log('Installing dependenciesβ¦')
- return { stop() {} }
- }
- const SKY = [56, 189, 248]
- const TEAL = [94, 234, 212]
- const lerp = (a, b, t) => Math.round(a + (b - a) * t)
- const grad = (i, n, s) => {
- const t = n > 1 ? i / (n - 1) : 0
- return `\x1b[38;2;${lerp(SKY[0], TEAL[0], t)};${lerp(SKY[1], TEAL[1], t)};${lerp(SKY[2], TEAL[2], t)}m${s}\x1b[0m`
- }
- const WAVE = ['β', 'β', 'βΏ', '~'] // shifts per row β flowing-funnel illusion
- const SPIN = ['β', 'β', 'β', 'β']
- const MSG = [
- 'spinning up your portal',
- 'wiring Home Β· Catalog Β· Showcase',
- 'loading sample datasets',
- 'almost ready',
- ]
- // Funnel rows: [indent, wave-count] narrowing to an apex.
- const FUNNEL = [
- [6, 5],
- [7, 3],
- [8, 1],
- ]
- const HEIGHT = 6 // 1 blank + 4 cyclone + 1 status
- const start = Date.now()
- let f = 0
- // Restore the cursor even if we never reach stop() (crash / SIGINT / SIGTERM),
- // otherwise the user's terminal is left with a hidden cursor.
- const showCursor = () => process.stdout.write('\x1b[?25h')
- const onExit = () => showCursor()
- const onSignal = () => {
- showCursor()
- process.exit(130)
- }
- process.once('exit', onExit)
- process.once('SIGINT', onSignal)
- process.once('SIGTERM', onSignal)
- process.stdout.write('\x1b[?25l') // hide cursor
- process.stdout.write('\n'.repeat(HEIGHT)) // reserve space
- const draw = () => {
- const secs = Math.floor((Date.now() - start) / 1000)
- const lines = ['']
- FUNNEL.forEach(([indent, width], ri) => {
- const w = WAVE[(f + ri) % WAVE.length]
- lines.push(' '.repeat(indent) + grad(ri, FUNNEL.length + 1, 'β²' + w.repeat(width) + 'β±'))
- })
- lines.push(' '.repeat(9) + grad(FUNNEL.length, FUNNEL.length + 1, 'βΏ'))
- const sp = `\x1b[38;2;${SKY[0]};${SKY[1]};${SKY[2]}m${SPIN[f % SPIN.length]}\x1b[0m`
- const msg = MSG[Math.floor(f / 14) % MSG.length]
- lines.push(` ${sp} \x1b[1mPortalJS\x1b[0m \x1b[2mΒ· ${msg}β¦ (${secs}s)\x1b[0m`)
- process.stdout.write(`\x1b[${HEIGHT}A`) // up to the top of the reserved block
- for (const ln of lines.slice(0, HEIGHT)) process.stdout.write('\x1b[2K' + ln + '\n')
- f++
- }
- draw()
- const timer = setInterval(draw, 110)
- return {
- stop(ok) {
- clearInterval(timer)
- process.removeListener('exit', onExit)
- process.removeListener('SIGINT', onSignal)
- process.removeListener('SIGTERM', onSignal)
- process.stdout.write(`\x1b[${HEIGHT}A`)
- for (let i = 0; i < HEIGHT; i++) process.stdout.write('\x1b[2K\n')
- process.stdout.write(`\x1b[${HEIGHT}A\x1b[?25h`) // back to top, show cursor
- const secs = Math.floor((Date.now() - start) / 1000)
- console.log(
- ok
- ? `\x1b[38;2;94;234;212mβ\x1b[0m Dependencies installed \x1b[2m(${secs}s)\x1b[0m`
- : '\x1b[33mβ \x1b[0m Dependency install did not complete'
- )
- },
- }
- }
- async function main() {
- const opts = parseArgs(process.argv.slice(2))
- if (opts.help) {
- console.log(HELP)
- return
- }
- // Lazy-load deps so --help works even before install resolves them.
- const prompts = (await import('prompts')).default
- const { downloadTemplate } = await import('giget')
- const onCancel = () => fail('Cancelled.')
- const ask = async (questions) =>
- opts.yes ? {} : prompts(questions, { onCancel })
- // 1. Resolve inputs (CLI flags win; otherwise prompt unless --yes).
- let dir = opts.dir
- if (!dir) {
- const a = await ask({
- type: 'text',
- name: 'dir',
- message: 'Project directory',
- initial: 'my-portal',
- })
- dir = a.dir || 'my-portal'
- }
- const slug = slugify(basename(dir))
- const name =
- opts.name ??
- (await ask({
- type: 'text',
- name: 'name',
- message: 'Project name',
- initial: slug.replace(/-/g, ' ').replace(/\b\w/g, (c) => c.toUpperCase()),
- })).name ??
- slug
- const description =
- opts.description ??
- (await ask({
- type: 'text',
- name: 'description',
- message: 'One-line description',
- initial: 'An open data portal.',
- })).description ??
- 'An open data portal.'
- const namespace =
- opts.namespace ??
- (await ask({
- type: 'select',
- name: 'namespace',
- message: 'Namespace mode',
- choices: [
- { title: 'theme β single publisher, group by subject', value: 'theme' },
- { title: 'owner β multiple publishers, group by who published', value: 'owner' },
- ],
- initial: 0,
- })).namespace ??
- 'theme'
- if (namespace !== 'theme' && namespace !== 'owner') fail(`Invalid --namespace: ${namespace}`)
- const doInstall = opts.install &&
- (opts.yes || (await ask({ type: 'confirm', name: 'v', message: 'Install dependencies?', initial: true })).v !== false)
- const doGit = opts.git &&
- (opts.yes || (await ask({ type: 'confirm', name: 'v', message: 'Initialize a git repo?', initial: true })).v !== false)
- // 2. Guard the target directory.
- const target = resolve(process.cwd(), dir)
- if (existsSync(target) && readdirSync(target).length > 0) {
- fail(`Directory "${dir}" already exists and is not empty. Choose another name.`)
- }
- // 3. Fetch the template (reliable subdir extraction; no degit fallback bug).
- console.log(`\nScaffolding ${name} β ${dir} (template @ ${opts.ref})`)
- try {
- await downloadTemplate(`github:datopian/portaljs/${TEMPLATE}#${opts.ref}`, {
- dir: target,
- force: true,
- })
- } catch (e) {
- fail(`Failed to fetch the template: ${e?.message || e}`)
- }
- // 4. Substitute placeholder tokens.
- substitute(target, [
- ['__PROJECT_NAME__', name],
- ['__PROJECT_SLUG__', slug],
- ['__DESCRIPTION__', description],
- ])
- // 5. Set the namespace mode in lib/datasets.ts.
- const datasetsTs = join(target, 'lib', 'datasets.ts')
- if (existsSync(datasetsTs)) {
- const s = readFileSync(datasetsTs, 'utf8').replace(
- /export const NAMESPACE_TYPE: 'theme' \| 'owner' = '(theme|owner)'/,
- `export const NAMESPACE_TYPE: 'theme' | 'owner' = '${namespace}'`
- )
- writeFileSync(datasetsTs, s)
- }
- // 5b. Bundle the agentic skills into the project's .claude/commands/ so they
- // work the instant the user runs `claude` in the new portal β no global
- // install. Pinned to the same template ref, so skills match the scaffold.
- // The repo's .claude/commands holds only OSS skills today; we still prune to
- // the allowlist so a future internal command never leaks into a scaffold.
- // Non-fatal: a fetch failure just prints the manual install one-liner.
- const skillsDir = join(target, '.claude', 'commands')
- try {
- let skills = await fetchSkillAllowlist(downloadTemplate, opts.ref)
- if (!skills) {
- console.log('\x1b[2mCould not read .claude-plugin/plugin.json; using the built-in skill list.\x1b[0m')
- skills = FALLBACK_SKILLS
- }
- await downloadTemplate(`github:datopian/portaljs/${SKILLS_SRC}#${opts.ref}`, {
- dir: skillsDir,
- force: true,
- })
- const bundled = []
- for (const entry of readdirSync(skillsDir)) {
- const keep = entry.endsWith('.md') && skills.includes(entry.slice(0, -3))
- if (keep) bundled.push(entry.slice(0, -3))
- else rmSync(join(skillsDir, entry), { recursive: true, force: true })
- }
- console.log(`Bundled ${bundled.length} PortalJS skills into .claude/commands/`)
- // A listed skill missing from the download (e.g. an older --ref) isn't fatal, but
- // say which ones so the scaffold doesn't silently ship fewer than expected.
- const missing = skills.filter((s) => !bundled.includes(s))
- if (missing.length)
- console.log(`\x1b[33mβ \x1b[0m Not found at ${opts.ref}: ${missing.map((s) => '/' + s).join(', ')}`)
- } catch (e) {
- console.log(
- `\x1b[33mβ \x1b[0m Could not bundle skills (${e?.message || e}). Install them with:\n` +
- ' curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash'
- Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
- Line 372 on GitHub
- )
- }
- // 6. Optional git init + install.
- if (doGit && !existsSync(join(target, '.git'))) {
- if (run('git', ['init', '-q'], target)) {
- run('git', ['add', '-A'], target)
- run('git', ['commit', '-q', '-m', 'chore: scaffold PortalJS portal'], target)
- }
- }
- if (doInstall) {
- const anim = createBuildAnimation()
- const { ok, err } = await runAsync('npm', ['install'], target)
- anim.stop(ok)
- if (!ok) {
- console.log(' Run `npm install` yourself in the project to finish.')
- if (err) {
- console.log('\x1b[2m' + err.trim().split('\n').slice(-3).join('\n') + '\x1b[0m')
- }
- }
- }
- // 7. Next steps.
- console.log(`
- β Created ${name} in ${dir}
- Next:
- cd ${dir}${doInstall ? '' : '\n npm install'}
- npm run dev # β http://localhost:3000
- Then run \`claude\` in the project β the PortalJS skills are bundled in
- .claude/commands/, so these work right away:
- /portaljs-add-dataset ./data/your-file.csv add data (or /portaljs-add-resource for more files)
- /portaljs-connect-ckan <ckan-url> point at a CKAN backend instead
- /portaljs-deploy publish (Cloudflare Pages recommended)
- `)
- }
- main().catch((e) => fail(e?.message || String(e)))
scripts/install-portaljs-skills.sh Β· Bundled script Β· 83 lines Β· 1 finding marked
- #!/usr/bin/env bash
- # Install PortalJS agentic skills into your personal Claude Code commands dir.
- #
- # Usage:
- # # From a clone of the repo (copies local files):
- # ./scripts/install-portaljs-skills.sh
- #
- # # From anywhere (downloads from GitHub):
- # curl -fsSL https://raw.githubusercontent.com/datopian/portaljs/main/scripts/install-portaljs-skills.sh | bash
- Note: Pipes a downloaded script directly into a shell (curl | sh / wget | bash)
- Line 9 on GitHub
- #
- # Env overrides:
- # CLAUDE_COMMANDS_DIR target dir (default: $HOME/.claude/commands)
- # PORTALJS_SKILLS_REF git ref to download from when remote (default: main)
- #
- # Idempotent: re-running overwrites the installed skill files in place.
- set -euo pipefail
- REF="${PORTALJS_SKILLS_REF:-main}"
- DEST="${CLAUDE_COMMANDS_DIR:-$HOME/.claude/commands}"
- RAW_REPO="https://raw.githubusercontent.com/datopian/portaljs/${REF}"
- RAW_BASE="$RAW_REPO/.claude/commands"
- # OSS skills only β excludes Gas Town internal commands (done/handoff/review).
- # The canonical skill names are read from the `commands` array of
- # .claude-plugin/plugin.json, the single source of truth (create-portaljs reads
- # it too), so this installer can't drift from the plugin.
- # The bare old names below are thin alias stubs kept for one minor release
- # (back-compat) β installed too so existing `/deploy`-style invocations keep
- # working this release. (`login` was removed.)
- ALIASES="architect new-portal add-dataset add-resource add-chart add-map connect-ckan migrate define-schema deploy check-data-quality"
- # `"./.claude/commands/<name>.md"` entries of plugin.json -> `<name>`, one per line.
- # Plain grep/sed so the installer doesn't need jq.
- skill_names() {
- { grep -o '\.claude/commands/[a-z0-9-]*\.md' || true; } |
- sed -e 's#^\.claude/commands/##' -e 's#\.md$##'
- }
- # Detect a local checkout: this script lives in <repo>/scripts/, so the commands
- # dir is ../.claude/commands relative to the script.
- SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" >/dev/null 2>&1 && pwd)"
- LOCAL_COMMANDS="$SCRIPT_DIR/../.claude/commands"
- LOCAL_PLUGIN="$SCRIPT_DIR/../.claude-plugin/plugin.json"
- if [ -f "$LOCAL_COMMANDS/portaljs-new-portal.md" ] && [ -f "$LOCAL_PLUGIN" ]; then
- MODE=local
- CANONICAL="$(skill_names < "$LOCAL_PLUGIN")"
- else
- MODE=remote
- if ! MANIFEST="$(curl -fsSL "$RAW_REPO/.claude-plugin/plugin.json")"; then
- echo "error: could not download .claude-plugin/plugin.json from github.com/datopian/portaljs@$REF" >&2
- exit 1
- fi
- CANONICAL="$(printf '%s\n' "$MANIFEST" | skill_names)"
- fi
- if [ -z "$CANONICAL" ]; then
- echo "error: no skills found in the 'commands' list of .claude-plugin/plugin.json" >&2
- exit 1
- fi
- # Word-split on whitespace: names are [a-z0-9-] only.
- SKILLS="$(echo $CANONICAL $ALIASES)"
- mkdir -p "$DEST"
- if [ "$MODE" = local ]; then
- echo "Installing PortalJS skills from local checkout -> $DEST"
- for s in $SKILLS; do
- cp "$LOCAL_COMMANDS/$s.md" "$DEST/$s.md"
- echo " + $s.md"
- done
- else
- echo "Installing PortalJS skills from github.com/datopian/portaljs@$REF -> $DEST"
- for s in $SKILLS; do
- curl -fsSL "$RAW_BASE/$s.md" -o "$DEST/$s.md"
- echo " + $s.md"
- done
- fi
- echo ""
- echo "Done. Installed: $SKILLS"
- echo "Restart Claude Code (or open a new session) and run /portaljs-new-portal to start."
Ratings and comments
Sign in to rate this skill.
Sign inFree to use. If itβs helped, pass it on.