PortalJS — Add Geo
Overview
Turn one geospatial upload into a dual-tier PortalJS dataset entirely on the user's
machine — no server-side container compute, keeping the zero-backend bet end-to-end.
From a single source the skill derives two serverless tiers plus the preserved original:
- PMTiles (render tier) — a vector-tile archive
<MapPreview> renders with MapLibre GL
over HTTP range requests; any dataset size pans and zooms with no tile server.
- GeoParquet (query tier) — a GeoParquet 1.1 file (covering
bbox column, Hilbert-sorted)
<GeoQuery> runs spatial SQL over in place via DuckDB-Wasm.
- Original — the untouched upload in its native CRS, kept downloadable.
All three land on Cloudflare R2 (Git LFS → Giftless), and the skill appends one
datasets.json entry whose resources[] match the shape the showcase auto-renders (the
@reference/world-boundaries demo, PR #1647): pmtiles → <MapPreview>, geoparquet →
<GeoQuery>, original → download. No page edits are needed. It automates the manual
tippecanoe/duckdb recipes in the template README.
Prerequisites
- A scaffolded
portaljs-catalog portal whose template ships components/MapPreview.tsx and
components/GeoQuery.tsx (PR #1647 or later).
- Native CLIs on the user's machine: GDAL (
ogr2ogr, ogrinfo), tippecanoe, and
duckdb (with the spatial extension). macOS: brew install gdal tippecanoe duckdb;
Debian/Ubuntu: apt-get install gdal-bin duckdb + tippecanoe (apt or build from source);
Windows: use WSL. The skill hard-stops with the install hint if any is missing.
- Arc credentials for the Git-LFS → R2 push (the same token
portaljs-deploy resolves), or
an OSS self-hosted Giftless.
Instructions
The canonical, full step-by-step workflow is
.claude/commands/portaljs-add-geo.md
— the single source of truth. Read and follow it when executing. Summary:
- Gather input — source (geo file path or URL), portal directory, namespace. Interview if
missing; never dead-end.
- Detect the native tools (
ogr2ogr, tippecanoe, duckdb + spatial). Any missing →
print the per-OS install and stop.
- Validate the portal directory and confirm the geo showcase components exist.
- Fetch (if URL) and detect the input format; reject raster and bare
.shp.
- Preserve the original, then normalize to EPSG:4326 GeoJSON with
ogr2ogr -t_srs EPSG:4326.
- Apply the size escape hatch (measure normalized-GeoJSON bytes): warn ≥ 500 MB; skip PMTiles
and emit GeoParquet-only > 2 GB (documented, never silent).
- Derive PMTiles with
tippecanoe -zg --drop-densest-as-needed.
- Derive GeoParquet with
duckdb — covering bbox STRUCT + ST_Hilbert order; verify a
range read.
- Track all three files with Git LFS, mint an Arc JWT, push to R2, prune, and build the
absolute
data.portaljs.com URLs.
- Append one dual-tier
datasets.json entry (pmtiles + geoparquet + original resources).
- Verify:
npx tsc --noEmit and a Range: 206 check on the R2 URL.
- Report the tiers, manifest entry, and showcase route.
Output
- Created:
data/<slug>.pmtiles, data/<slug>.parquet, data/<slug>.<origext> (all
LFS-tracked → R2); data/<slug>.4326.geojson (intermediate; can be pruned).
- Modified:
datasets.json (one dual-tier entry appended); .gitattributes (LFS tracking).
- Verified:
npx tsc --noEmit passes; the R2 URLs answer HTTP 206 range reads.
- Result:
/@<namespace>/<slug> renders the PMTiles map and the GeoParquet spatial-query
panel together, with no page edits.
Error Handling
| Symptom |
Cause |
Fix |
MISSING_TOOLS |
ogr2ogr/tippecanoe/duckdb (or duckdb spatial) absent |
Print the per-OS install line and stop; re-run after installing. |
UNSUPPORTED_FORMAT |
Raster .tif/.tiff (COG tier, later phase) |
Use portaljs-add-dataset for tabular; await the COG phase. |
SHAPEFILE_NOT_ZIPPED |
Bare .shp with no siblings |
Zip the whole .shp/.dbf/.shx/.prj set and pass the .zip. |
MULTI_LAYER |
GPKG/KML with more than one layer |
List layers and ask which to ingest; pass the layer name. |
REPROJECTION_FAILED |
ogr2ogr error, often a missing .prj |
Show stderr; re-run with -s_srs EPSG:<code>. |
EMPTY_OUTPUT |
Normalized GeoJSON has 0 features |
Check the geometry column / CRS of the source. |
TILING_FAILED / GEOPARQUET_FAILED |
tippecanoe or duckdb non-zero |
Show stderr; for very large inputs see the size escape hatch. |
LFS_PUSH_FAILED |
Missing/expired Arc token or unset lfs.url |
Re-mint the JWT (see portaljs-deploy); confirm git config lfs.url. |
Examples
Example 1 — Ingest a GeoJSON into both tiers
/portaljs-add-geo source=./data/rivers.geojson name="World Rivers" namespace=reference
Example 2 — Zipped Shapefile from a URL
/portaljs-add-geo source=https://example.com/admin-boundaries.zip slug=admin-boundaries
Example 3 — GeoPackage with an explicit layer and namespace
/portaljs-add-geo source=./data/census.gpkg slug=census-tracts namespace=statistics
Resources
Source: jeremylongshore/claude-code-plugins-plus-skills → plugins/community/portaljs/skills/portaljs-add-geo/SKILL.md
1---
2name: portaljs-add-geo
3description: "Auto-ingest a geospatial file (GeoJSON, Shapefile, GeoPackage, KML/KMZ, FlatGeobuf, CSV-with-geometry) into a PortalJS portal on the user's own machine, with no server. Normalizes CRS to EPSG:4326, derives a PMTiles render tier and a GeoParquet query tier, pushes all three artifacts to Cloudflare R2 via Git LFS, and appends one dual-tier datasets.json entry the showcase auto-renders. Use when the source is a vector geo format that needs a map or spatial-query view."
4allowed-tools: "Read, Write, Edit, Bash(ogr2ogr:*), Bash(ogrinfo:*), Bash(tippecanoe:*), Bash(duckdb:*), Bash(git:*), Bash(curl:*), Bash(npx:*), Bash(node:*), Bash(mkdir:*), Bash(cp:*), Bash(wc:*), Bash(command:*), WebFetch"
5category: devops-and-infra
6source_repo: jeremylongshore/claude-code-plugins-plus-skills
7source_path: "plugins/community/portaljs/skills/portaljs-add-geo/SKILL.md"
8source_url: https://github.com/jeremylongshore/claude-code-plugins-plus-skills/blob/HEAD/plugins/community/portaljs/skills/portaljs-add-geo/SKILL.md
9---
10
11
12# PortalJS — Add Geo
13
14## Overview
15
16Turn one geospatial upload into a **dual-tier** PortalJS dataset entirely on the user's
17machine — **no server-side container compute**, keeping the zero-backend bet end-to-end.
18From a single source the skill derives two serverless tiers plus the preserved original:
19
20- **PMTiles** (render tier) — a vector-tile archive `<MapPreview>` renders with MapLibre GL
21 over HTTP range requests; any dataset size pans and zooms with no tile server.
22- **GeoParquet** (query tier) — a GeoParquet 1.1 file (covering `bbox` column, Hilbert-sorted)
23 `<GeoQuery>` runs spatial SQL over in place via DuckDB-Wasm.
24- **Original** — the untouched upload in its native CRS, kept downloadable.
25
26All three land on Cloudflare R2 (Git LFS → Giftless), and the skill appends **one**
27`datasets.json` entry whose `resources[]` match the shape the showcase auto-renders (the
28`@reference/world-boundaries` demo, PR #1647): `pmtiles` → `<MapPreview>`, `geoparquet` →
29`<GeoQuery>`, original → download. No page edits are needed. It automates the manual
30`tippecanoe`/`duckdb` recipes in the template README.
31
32## Prerequisites
33
34- A scaffolded `portaljs-catalog` portal whose template ships `components/MapPreview.tsx` and
35 `components/GeoQuery.tsx` (PR #1647 or later).
36- Native CLIs on the user's machine: **GDAL** (`ogr2ogr`, `ogrinfo`), **tippecanoe**, and
37 **duckdb** (with the `spatial` extension). macOS: `brew install gdal tippecanoe duckdb`;
38 Debian/Ubuntu: `apt-get install gdal-bin duckdb` + tippecanoe (apt or build from source);
39 Windows: use WSL. The skill hard-stops with the install hint if any is missing.
40- Arc credentials for the Git-LFS → R2 push (the same token `portaljs-deploy` resolves), or
41 an OSS self-hosted Giftless.
42
43## Instructions
44
45The canonical, full step-by-step workflow is
46[`.claude/commands/portaljs-add-geo.md`](https://github.com/datopian/portaljs/blob/main/.claude/commands/portaljs-add-geo.md)
47— the single source of truth. Read and follow it when executing. Summary:
48
491. Gather input — source (geo file path or URL), portal directory, namespace. Interview if
50 missing; never dead-end.
512. Detect the native tools (`ogr2ogr`, `tippecanoe`, `duckdb` + `spatial`). Any missing →
52 print the per-OS install and stop.
533. Validate the portal directory and confirm the geo showcase components exist.
544. Fetch (if URL) and detect the input format; reject raster and bare `.shp`.
555. Preserve the original, then normalize to EPSG:4326 GeoJSON with `ogr2ogr -t_srs EPSG:4326`.
566. Apply the size escape hatch (measure normalized-GeoJSON bytes): warn ≥ 500 MB; skip PMTiles
57 and emit GeoParquet-only > 2 GB (documented, never silent).
587. Derive PMTiles with `tippecanoe -zg --drop-densest-as-needed`.
598. Derive GeoParquet with `duckdb` — covering `bbox` STRUCT + `ST_Hilbert` order; verify a
60 range read.
619. Track all three files with Git LFS, mint an Arc JWT, push to R2, prune, and build the
62 absolute `data.portaljs.com` URLs.
6310. Append one dual-tier `datasets.json` entry (`pmtiles` + `geoparquet` + original resources).
6411. Verify: `npx tsc --noEmit` and a `Range:` 206 check on the R2 URL.
6512. Report the tiers, manifest entry, and showcase route.
66
67## Output
68
69- **Created:** `data/<slug>.pmtiles`, `data/<slug>.parquet`, `data/<slug>.<origext>` (all
70 LFS-tracked → R2); `data/<slug>.4326.geojson` (intermediate; can be pruned).
71- **Modified:** `datasets.json` (one dual-tier entry appended); `.gitattributes` (LFS tracking).
72- **Verified:** `npx tsc --noEmit` passes; the R2 URLs answer HTTP 206 range reads.
73- **Result:** `/@<namespace>/<slug>` renders the PMTiles map and the GeoParquet spatial-query
74 panel together, with no page edits.
75
76## Error Handling
77
78| Symptom | Cause | Fix |
79| --- | --- | --- |
80| `MISSING_TOOLS` | `ogr2ogr`/`tippecanoe`/`duckdb` (or duckdb `spatial`) absent | Print the per-OS install line and stop; re-run after installing. |
81| `UNSUPPORTED_FORMAT` | Raster `.tif`/`.tiff` (COG tier, later phase) | Use `portaljs-add-dataset` for tabular; await the COG phase. |
82| `SHAPEFILE_NOT_ZIPPED` | Bare `.shp` with no siblings | Zip the whole `.shp/.dbf/.shx/.prj` set and pass the `.zip`. |
83| `MULTI_LAYER` | GPKG/KML with more than one layer | List layers and ask which to ingest; pass the layer name. |
84| `REPROJECTION_FAILED` | `ogr2ogr` error, often a missing `.prj` | Show stderr; re-run with `-s_srs EPSG:<code>`. |
85| `EMPTY_OUTPUT` | Normalized GeoJSON has 0 features | Check the geometry column / CRS of the source. |
86| `TILING_FAILED` / `GEOPARQUET_FAILED` | tippecanoe or duckdb non-zero | Show stderr; for very large inputs see the size escape hatch. |
87| `LFS_PUSH_FAILED` | Missing/expired Arc token or unset `lfs.url` | Re-mint the JWT (see `portaljs-deploy`); confirm `git config lfs.url`. |
88
89## Examples
90
91### Example 1 — Ingest a GeoJSON into both tiers
92
93```
94/portaljs-add-geo source=./data/rivers.geojson name="World Rivers" namespace=reference
95```
96
97### Example 2 — Zipped Shapefile from a URL
98
99```
100/portaljs-add-geo source=https://example.com/admin-boundaries.zip slug=admin-boundaries
101```
102
103### Example 3 — GeoPackage with an explicit layer and namespace
104
105```
106/portaljs-add-geo source=./data/census.gpkg slug=census-tracts namespace=statistics
107```
108
109## Resources
110
111- Full workflow: [`.claude/commands/portaljs-add-geo.md`](https://github.com/datopian/portaljs/blob/main/.claude/commands/portaljs-add-geo.md)
112- Flags, thresholds, and the dual-tier manifest shape: [`references/reference.md`](references/reference.md)
113- Related skills: `portaljs-add-dataset`, `portaljs-add-map`, `portaljs-add-resource`, `portaljs-deploy`
114- tippecanoe: <https://github.com/felt/tippecanoe> · DuckDB spatial: <https://duckdb.org/docs/extensions/spatial>
115
116---
117
118**Source:** [`jeremylongshore/claude-code-plugins-plus-skills`](https://github.com/jeremylongshore/claude-code-plugins-plus-skills) → `plugins/community/portaljs/skills/portaljs-add-geo/SKILL.md`
jeremylongshore