PortalJS — Add Dataset
Overview
Register a dataset in a PortalJS (portaljs-catalog) portal. The skill appends one entry to
datasets.json — the single source of truth for the catalog — and routes the underlying
bytes by source first, then size: a local file defaults to R2 via Git LFS, a remote URL
defaults to passthrough (no copy). No per-dataset page is created; the catalog at /search
lists the new entry and the dynamic showcase route pages/[owner]/[slug].tsx renders it
automatically at /@<namespace>/<slug>. Supported formats for the showcase preview: CSV,
TSV, JSON (array), and GeoJSON.
Prerequisites
- A scaffolded PortalJS portal (see
portaljs-new-portal) with datasets.json,
package.json, and pages/[owner]/[slug].tsx present.
- The source data: a local file path or a publicly reachable URL.
- For the R2/Git LFS default route:
git and git-lfs installed, and an Arc account token
(or an OSS Giftless key) to mint a push-scoped LFS credential.
- Node 18+ and npm available in the portal directory.
Instructions
The canonical, full step-by-step workflow is
.claude/commands/portaljs-add-dataset.md —
the single source of truth. Read and follow it when executing. Summary:
- Gather input from
$ARGUMENTS — source (file path or URL), portal directory (default
.), dataset name/slug, description, namespace. If the source is missing, interview the
user; never dead-end.
- Validate the portal directory: confirm
datasets.json, package.json, and
pages/[owner]/[slug].tsx exist.
- Detect the format from the file extension, URL extension, or
Content-Type header
(CSV, TSV, JSON array, or GeoJSON); reject anything else and ask for a conversion.
- Route the data by source: remote URL → passthrough (default) or adopt into R2 (opt-in);
local file → R2 via Git LFS (default) or inline into
public/data/ (fenced exception for
bundled samples or an OSS-no-R2 fallback).
- Append one entry to
datasets.json — slug, namespace, name, description, file
(the routed path/URL), format — keeping (namespace, slug) unique.
- Verify the build with
npx next build; fix errors (commonly malformed JSON) before
reporting success.
- Report the route taken, the manifest change, and the showcase URL.
Output
- Modified:
datasets.json (one entry appended).
- Created (route-dependent):
data/<slug>.<ext> tracked via Git LFS (R2 default), or
public/data/<slug>.<ext> (inline exception). Nothing is created for remote passthrough.
- Verified:
npx next build passes.
- Result: the dataset appears in
/search and renders at /@<namespace>/<slug>.
Error Handling
| Symptom |
Cause |
Fix |
| Fetch fails for a URL source |
Non-200 status or unreachable host |
Report the HTTP status and ask the user to confirm the URL is publicly accessible. |
| "Not a portaljs-catalog portal" |
datasets.json missing |
This is an older single-page template; ask the user how to proceed rather than failing silently. |
| Unsupported format |
Extension/content-type isn't csv/tsv/json/geojson |
Ask the user to convert the source before continuing. |
git lfs push has nothing to stream |
git lfs install --local never ran, so raw bytes were committed instead of a pointer |
Run git lfs install --local before git lfs track, re-add and re-commit the file. |
| R2 PUT returns 400 |
A broad http.extraHeader was set and replayed onto the presigned URL |
Use the _jwt Basic-auth piggyback in lfs.url only — never a global http.extraHeader. |
(namespace, slug) clash |
Another entry already uses that pair |
Ask the user for a different slug or namespace. |
next build fails |
Malformed JSON in datasets.json |
Print the build log, fix the JSON, rebuild before reporting success. |
Examples
Example 1 — Local CSV, default R2 route
/portaljs-add-dataset ./data/co2-emissions.csv namespace=climate
Moves the file into data/, tracks it with Git LFS, pushes it to R2, and appends a manifest
entry whose file is the resulting https://data.portaljs.com/... URL.
Example 2 — Remote URL, passthrough (no download)
/portaljs-add-dataset https://example.org/open-data/trade.csv namespace=trade
Detects the format from the response headers and records the URL as-is in datasets.json —
no bytes are copied.
Example 3 — GeoJSON adopted into R2
/portaljs-add-dataset https://example.org/boundaries.geojson namespace=reference adopt=true
Downloads the file, then routes it as a local file through the Git LFS → R2 path so it is
hosted and versioned under the portal (useful when in-browser range queries are needed).
Example 4 — Bundled sample data, inline exception
/portaljs-add-dataset ./samples/demo.csv namespace=reference
When the portal has no R2 credentials (OSS self-host) or the file is bundled sample data,
the skill copies it into public/data/ instead, per the .gitattributes inline fence.
Resources
Source: jeremylongshore/claude-code-plugins-plus-skills → plugins/community/portaljs/skills/portaljs-add-dataset/SKILL.md
1---2name: portaljs-add-dataset3description: Add a dataset (CSV, TSV, JSON, or GeoJSON) to an existing PortalJS portal. Appends an entry to datasets.json so the catalog and showcase render it automatically; routes the data by source (local file vs remote URL) — R2 via Git LFS by default, remote URLs by passthrough. Use when registering a new dataset in a scaffolded portal.4---567# PortalJS — Add Dataset89## Overview1011Register a dataset in a PortalJS (`portaljs-catalog`) portal. The skill appends one entry to12`datasets.json` — the single source of truth for the catalog — and routes the underlying13bytes by **source first, then size**: a local file defaults to R2 via Git LFS, a remote URL14defaults to passthrough (no copy). No per-dataset page is created; the catalog at `/search`15lists the new entry and the dynamic showcase route `pages/[owner]/[slug].tsx` renders it16automatically at `/@<namespace>/<slug>`. Supported formats for the showcase preview: CSV,17TSV, JSON (array), and GeoJSON.1819## Prerequisites2021- A scaffolded PortalJS portal (see `portaljs-new-portal`) with `datasets.json`,22 `package.json`, and `pages/[owner]/[slug].tsx` present.23- The source data: a local file path or a publicly reachable URL.24- For the R2/Git LFS default route: `git` and `git-lfs` installed, and an Arc account token25 (or an OSS Giftless key) to mint a push-scoped LFS credential.26- Node 18+ and npm available in the portal directory.2728## Instructions2930The canonical, full step-by-step workflow is31[`.claude/commands/portaljs-add-dataset.md`](https://github.com/datopian/portaljs/blob/main/.claude/commands/portaljs-add-dataset.md) —32the single source of truth. Read and follow it when executing. Summary:33341. Gather input from `$ARGUMENTS` — source (file path or URL), portal directory (default35 `.`), dataset name/slug, description, namespace. If the source is missing, interview the36 user; never dead-end.372. Validate the portal directory: confirm `datasets.json`, `package.json`, and38 `pages/[owner]/[slug].tsx` exist.393. Detect the format from the file extension, URL extension, or `Content-Type` header40 (CSV, TSV, JSON array, or GeoJSON); reject anything else and ask for a conversion.414. Route the data by source: remote URL → passthrough (default) or adopt into R2 (opt-in);42 local file → R2 via Git LFS (default) or inline into `public/data/` (fenced exception for43 bundled samples or an OSS-no-R2 fallback).445. Append one entry to `datasets.json` — `slug`, `namespace`, `name`, `description`, `file`45 (the routed path/URL), `format` — keeping `(namespace, slug)` unique.466. Verify the build with `npx next build`; fix errors (commonly malformed JSON) before47 reporting success.487. Report the route taken, the manifest change, and the showcase URL.4950## Output5152- **Modified:** `datasets.json` (one entry appended).53- **Created (route-dependent):** `data/<slug>.<ext>` tracked via Git LFS (R2 default), or54 `public/data/<slug>.<ext>` (inline exception). Nothing is created for remote passthrough.55- **Verified:** `npx next build` passes.56- **Result:** the dataset appears in `/search` and renders at `/@<namespace>/<slug>`.5758## Error Handling5960| Symptom | Cause | Fix |61| --- | --- | --- |62| Fetch fails for a URL source | Non-200 status or unreachable host | Report the HTTP status and ask the user to confirm the URL is publicly accessible. |63| "Not a portaljs-catalog portal" | `datasets.json` missing | This is an older single-page template; ask the user how to proceed rather than failing silently. |64| Unsupported format | Extension/content-type isn't csv/tsv/json/geojson | Ask the user to convert the source before continuing. |65| `git lfs push` has nothing to stream | `git lfs install --local` never ran, so raw bytes were committed instead of a pointer | Run `git lfs install --local` before `git lfs track`, re-add and re-commit the file. |66| R2 PUT returns 400 | A broad `http.extraHeader` was set and replayed onto the presigned URL | Use the `_jwt` Basic-auth piggyback in `lfs.url` only — never a global `http.extraHeader`. |67| `(namespace, slug)` clash | Another entry already uses that pair | Ask the user for a different slug or namespace. |68| `next build` fails | Malformed JSON in `datasets.json` | Print the build log, fix the JSON, rebuild before reporting success. |6970## Examples7172### Example 1 — Local CSV, default R2 route7374```75/portaljs-add-dataset ./data/co2-emissions.csv namespace=climate76```77Moves the file into `data/`, tracks it with Git LFS, pushes it to R2, and appends a manifest78entry whose `file` is the resulting `https://data.portaljs.com/...` URL.7980### Example 2 — Remote URL, passthrough (no download)8182```83/portaljs-add-dataset https://example.org/open-data/trade.csv namespace=trade84```85Detects the format from the response headers and records the URL as-is in `datasets.json` —86no bytes are copied.8788### Example 3 — GeoJSON adopted into R28990```91/portaljs-add-dataset https://example.org/boundaries.geojson namespace=reference adopt=true92```93Downloads the file, then routes it as a local file through the Git LFS → R2 path so it is94hosted and versioned under the portal (useful when in-browser range queries are needed).9596### Example 4 — Bundled sample data, inline exception9798```99/portaljs-add-dataset ./samples/demo.csv namespace=reference100```101When the portal has no R2 credentials (OSS self-host) or the file is bundled sample data,102the skill copies it into `public/data/` instead, per the `.gitattributes` inline fence.103104## Resources105106- Full workflow: [`.claude/commands/portaljs-add-dataset.md`](https://github.com/datopian/portaljs/blob/main/.claude/commands/portaljs-add-dataset.md)107- Manifest fields and routing details: [`references/reference.md`](references/reference.md)108- Related skills: `portaljs-new-portal`, `portaljs-add-chart`, `portaljs-add-map`, `portaljs-define-schema`109- Git LFS documentation: <https://git-lfs.com/>110111---112113**Source:** [`jeremylongshore/claude-code-plugins-plus-skills`](https://github.com/jeremylongshore/claude-code-plugins-plus-skills) → `plugins/community/portaljs/skills/portaljs-add-dataset/SKILL.md`
thedixitjain