PortalJS — Define Schema
Overview
Define a dataset's metadata profile — the authoring skill for the metadata-profile
contract (lib/metadata). Where portaljs-add-dataset registers that a dataset exists,
this skill describes what its data means: infer a Frictionless Table Schema (fields,
types, constraints) from sampled data, add the Data Package fields a catalog surfaces
(title, licenses, sources, keywords), and write them onto the dataset's entry in
datasets.json. The showcase at /@<namespace>/<slug> then renders a typed field table
instead of a bare preview. The model is Frictionless-native; DCAT is a serialization layer
built on top later, not authored here.
The skill runs on a profile ladder — reach for higher levels only when needed:
| Level |
What it is |
When |
| L0 |
Default frictionless-tabular profile; declare schema + metadata. |
Default. Standard tabular CSV/TSV. |
| L1 |
L0 plus extra descriptive package fields. |
Extra metadata, standard validation is fine. |
| L2 |
Fully custom profile (own schema template + validate()). |
A dataset type needing custom validation rules. |
| L3 |
Multiple registered profiles, resolved per dataset. |
A portal mixing dataset types. |
The skill is interactive and never dead-ends: if input is thin it interviews in short
rounds, infers defaults from the data, echoes the schema for confirmation, and accepts
"use defaults" to proceed with the inferred schema as-is.
Prerequisites
- A scaffolded PortalJS portal with the metadata contract (
lib/metadata/types.ts,
pages/[owner]/[slug].tsx); see portaljs-new-portal.
- The target dataset already registered in
datasets.json (see portaljs-add-dataset).
- For tabular schema inference, the dataset's CSV/TSV file present under
PORTAL_DIR/public/data/. JSON/GeoJSON datasets get package metadata only — no fields.
- Node 18+;
tsx optional, used for the schema-validation check.
Instructions
The canonical, full step-by-step workflow is
.claude/commands/portaljs-define-schema.md —
the single source of truth. Read and follow it when executing. Summary:
- Gather
PORTAL_DIR, DATASET (slug or namespace/slug), and LEVEL (default L0)
from input; if DATASET is missing, list the portal's slugs and ask.
- Validate the portal has the metadata contract (
datasets.json, lib/metadata/types.ts,
the showcase route); proceed anyway if lib/metadata/ predates the contract.
- For tabular datasets, sample the header and ~50 rows from
public/data/<file> and infer
each field's type, constraints (required, unique, pattern), and a primary key.
- Echo the inferred schema as a table for confirmation; offer to go beyond L0 only if
warranted.
- Ask for optional Data Package metadata: license, source(s), keywords, version.
- Write the schema and metadata onto the dataset's entry in
datasets.json in place,
preserving all other fields; for L2/L3, scaffold and register a custom profile module.
- Optionally validate the schema against the data's rows via the profile's
validate().
- Verify with
npx next build; fix malformed JSON or an invalid FieldType before
reporting success.
- Report the profile, fields, metadata set, and the showcase URL.
Output
- Modified:
datasets.json (target entry gains profile, schema, licenses,
sources, keywords, version — unset fields omitted).
- Created (L2/L3 only):
lib/metadata/<profile-id>.ts; lib/metadata/registry.ts
updated with a registerProfile(...) call.
- Verified:
npx next build succeeds.
- Result:
/@<namespace>/<slug> renders a typed field table in place of a bare preview.
Error Handling
| Symptom |
Cause |
Fix |
Dataset not found in datasets.json |
Wrong slug or missing namespace/ prefix |
List available slugs and re-prompt. |
lib/metadata/ missing |
Portal predates the metadata-profile contract |
Proceed anyway — schema fields are optional and ignored by older showcases. |
No fields schema produced |
Dataset is JSON/GeoJSON, not tabular |
Expected — capture Data Package metadata only. |
| Validation reports type errors |
Sampled values don't coerce to the inferred type |
Relax the type or drop the offending required/pattern constraint. |
next build fails on datasets.json |
Stray comma or a type outside FieldType |
Fix the JSON/type and rebuild before reporting success. |
Examples
Example 1 — Default L0 schema for a CSV dataset
/portaljs-define-schema population-2022
Infers fields (e.g. country: string, population: integer), drafts titles, asks for a
license and source, and writes the schema under the default frictionless-tabular profile.
Example 2 — Metadata only for a GeoJSON dataset
/portaljs-define-schema neighborhoods-geo
GeoJSON has no tabular fields; the skill captures license, sources, and keywords onto the
entry and skips schema inference.
Example 3 — Custom L2 profile with its own validation
/portaljs-define-schema co2-emissions level=L2
Scaffolds lib/metadata/co2-emissions-profile.ts with a custom validate(), registers it
in lib/metadata/registry.ts, and sets "profile": "co2-emissions-profile" on the entry.
Resources
1---2name: portaljs-define-schema3description: Define a dataset's metadata profile — infer a Frictionless Table Schema from its data, add Data Package metadata (license, sources, keywords), and write it into datasets.json so the showcase renders a typed field table. Extend or customize via the L0-L3 profile ladder. Use when a registered dataset needs field types, constraints, or catalog metadata before publishing.4license: MIT5---6
7# PortalJS — Define Schema
8
9## Overview
10
11Define a dataset's metadata profile — the **authoring** skill for the metadata-profile
12contract (`lib/metadata`). Where `portaljs-add-dataset` registers *that* a dataset exists,
13this skill describes *what its data means*: infer a Frictionless **Table Schema** (fields,
14types, constraints) from sampled data, add the **Data Package** fields a catalog surfaces
15(title, licenses, sources, keywords), and write them onto the dataset's entry in
16`datasets.json`. The showcase at `/@<namespace>/<slug>` then renders a typed field table
17instead of a bare preview. The model is Frictionless-native; DCAT is a serialization layer
18built on top later, not authored here.
19
20The skill runs on a profile ladder — reach for higher levels only when needed:
21
22| Level | What it is | When |
23| --- | --- | --- |
24| L0 | Default `frictionless-tabular` profile; declare schema + metadata. | Default. Standard tabular CSV/TSV. |
25| L1 | L0 plus extra descriptive package fields. | Extra metadata, standard validation is fine. |
26| L2 | Fully custom profile (own schema template + `validate()`). | A dataset type needing custom validation rules. |
27| L3 | Multiple registered profiles, resolved per dataset. | A portal mixing dataset types. |
28
29The skill is interactive and never dead-ends: if input is thin it interviews in short
30rounds, infers defaults from the data, echoes the schema for confirmation, and accepts
31"use defaults" to proceed with the inferred schema as-is.
32
33## Prerequisites
34
35- A scaffolded PortalJS portal with the metadata contract (`lib/metadata/types.ts`,
36 `pages/[owner]/[slug].tsx`); see `portaljs-new-portal`.
37- The target dataset already registered in `datasets.json` (see `portaljs-add-dataset`).
38- For tabular schema inference, the dataset's CSV/TSV file present under
39 `PORTAL_DIR/public/data/`. JSON/GeoJSON datasets get package metadata only — no `fields`.
40- Node 18+; `tsx` optional, used for the schema-validation check.
41
42## Instructions
43
44The canonical, full step-by-step workflow is
45[`.claude/commands/portaljs-define-schema.md`](https://github.com/datopian/portaljs/blob/main/.claude/commands/portaljs-define-schema.md) —
46the single source of truth. Read and follow it when executing. Summary:
47
481. Gather `PORTAL_DIR`, `DATASET` (slug or `namespace/slug`), and `LEVEL` (default `L0`)
49 from input; if `DATASET` is missing, list the portal's slugs and ask.
502. Validate the portal has the metadata contract (`datasets.json`, `lib/metadata/types.ts`,
51 the showcase route); proceed anyway if `lib/metadata/` predates the contract.
523. For tabular datasets, sample the header and ~50 rows from `public/data/<file>` and infer
53 each field's type, constraints (`required`, `unique`, `pattern`), and a primary key.
544. Echo the inferred schema as a table for confirmation; offer to go beyond L0 only if
55 warranted.
565. Ask for optional Data Package metadata: license, source(s), keywords, version.
576. Write the schema and metadata onto the dataset's entry in `datasets.json` in place,
58 preserving all other fields; for L2/L3, scaffold and register a custom profile module.
597. Optionally validate the schema against the data's rows via the profile's `validate()`.
608. Verify with `npx next build`; fix malformed JSON or an invalid `FieldType` before
61 reporting success.
629. Report the profile, fields, metadata set, and the showcase URL.
63
64## Output
65
66- **Modified:** `datasets.json` (target entry gains `profile`, `schema`, `licenses`,
67 `sources`, `keywords`, `version` — unset fields omitted).
68- **Created (L2/L3 only):** `lib/metadata/<profile-id>.ts`; `lib/metadata/registry.ts`
69 updated with a `registerProfile(...)` call.
70- **Verified:** `npx next build` succeeds.
71- **Result:** `/@<namespace>/<slug>` renders a typed field table in place of a bare preview.
72
73## Error Handling
74
75| Symptom | Cause | Fix |
76| --- | --- | --- |
77| Dataset not found in `datasets.json` | Wrong slug or missing `namespace/` prefix | List available slugs and re-prompt. |
78| `lib/metadata/` missing | Portal predates the metadata-profile contract | Proceed anyway — schema fields are optional and ignored by older showcases. |
79| No `fields` schema produced | Dataset is JSON/GeoJSON, not tabular | Expected — capture Data Package metadata only. |
80| Validation reports type errors | Sampled values don't coerce to the inferred type | Relax the `type` or drop the offending `required`/`pattern` constraint. |
81| `next build` fails on `datasets.json` | Stray comma or a type outside `FieldType` | Fix the JSON/type and rebuild before reporting success. |
82
83## Examples
84
85### Example 1 — Default L0 schema for a CSV dataset
86
87```
88/portaljs-define-schema population-2022
89```
90Infers fields (e.g. `country: string`, `population: integer`), drafts titles, asks for a
91license and source, and writes the schema under the default `frictionless-tabular` profile.
92
93### Example 2 — Metadata only for a GeoJSON dataset
94
95```
96/portaljs-define-schema neighborhoods-geo
97```
98GeoJSON has no tabular `fields`; the skill captures license, sources, and keywords onto the
99entry and skips schema inference.
100
101### Example 3 — Custom L2 profile with its own validation
102
103```
104/portaljs-define-schema co2-emissions level=L2
105```
106Scaffolds `lib/metadata/co2-emissions-profile.ts` with a custom `validate()`, registers it
107in `lib/metadata/registry.ts`, and sets `"profile": "co2-emissions-profile"` on the entry.
108
109## Resources
110
111- Full workflow: [`.claude/commands/portaljs-define-schema.md`](https://github.com/datopian/portaljs/blob/main/.claude/commands/portaljs-define-schema.md)
112- Field-type and troubleshooting reference: [`references/reference.md`](references/reference.md)
113- Related skills: `portaljs-add-dataset`, `portaljs-add-dcat`, `portaljs-check-data-quality`
114- Frictionless Table Schema specification: <https://datapackage.org/standard/table-schema/>
datopian