# Dataset Register

> Use when: after /data-acquisition, formalize staged dataset in topic catalog (provenance, variables, quirks). NOT downloading (/data-acquisition).

- Skill: `markusstrasser/dataset-register` (Agent Skill)
- Install (CLI): `npx skillmds@latest add markusstrasser/dataset-register`
- Raw SKILL.md: https://api.skillmd.com/api/skills/markusstrasser/dataset-register/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: markusstrasser (https://skillmd.com/u/markusstrasser)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/markusstrasser/dataset-register

---


# Dataset Register

Normalize per-topic dataset catalogs. Existing examples (the `research` repo split 2026-06-23 into
per-topic repos — paths resolve inside each topic's own repo):
- `~/Projects/immigration-research/research/immigration-dataset-register.md` (register)
- `~/Projects/immigration-research/research/immigration-dataset-roadmap.md` (roadmap: verified-URL cards grouped by the gap they fill)
- `~/Projects/iq-sex-differences/research/iq-sex-differences-dataset-cards.md` (long-form cards)
- `~/Projects/iq-sex-differences/research/iq-sex-differences-dataset-roadmap.md` (roadmap)

Goal: one card per dataset, queryable by grep, surviving across sessions.

---

## Card format

Append to `research/<topic>-dataset-register.md`:

```markdown
### <DATASET_ID> — <short name>

**Source:** <agency / provider>
**Acquired:** YYYY-MM-DD  [or Access-gated / In-progress / Blocked]
**Local path:** `sources/<topic>/data/external/stage3/<source>/<dataset>/`
**Official:** <landing URL>
**Codebook:** `<local codebook filename>` | <codebook URL>
**Size:** <N> files, <total MB>
**License:** <public-use / restricted / registration required>

**Key variables:**
- `<VAR>` — <meaning>
- `<VAR>` — <meaning>

**Known quirks:**
- <gotcha 1>
- <gotcha 2>

**Used in:**
- `research/<topic>-<memo>.md` (if already analyzed)
- `sources/<topic>/scripts/<script>.py` (if already coded)
```

---

## Minimal card (when you only have URL + path)

```markdown
### <DATASET_ID> — <short name>
**Source:** <agency>  **Acquired:** YYYY-MM-DD
**Local:** `sources/<topic>/data/external/stage3/<source>/<dataset>/`
**Official:** <url>
```

Expand when you actually use it.

---

## Register vs cards vs roadmap

| File | Purpose | When to use |
|---|---|---|
| `<topic>-dataset-register.md` | Flat list of what we have | Default. Update on every acquisition. |
| `<topic>-dataset-cards.md` | Long-form per-dataset notes | When a single dataset warrants >1 page of documentation. |
| `<topic>-dataset-roadmap.md` | What we want next + status | When planning acquisitions. Update `✅ acquired` / `🚧 gated` / `❌ blocked`. |

Three files can coexist for large topics (e.g., iq-sex-differences has all three).

---

## Deduplication rule

Before writing a new card, grep for the dataset ID:
```bash
grep -i "<DATASET_ID>" research/*-register.md research/*-cards.md research/*-roadmap.md
```

If found, **edit the existing card** — don't create a duplicate. Update `Acquired:`
date and add any new variables used.

---

## After registering

Commit with message like:
```
[research] Register <DATASET_ID> — <brief why>
```

Body (per commit-conventions.md for `research/` paths):
> Names the concept (dataset) and what changed directionally (added / updated / moved to acquired).

## Evidence

- Parallel formats across `research/immigration-dataset-register.md`,
  `research/iq-sex-differences-dataset-cards.md`, `research/iq-sex-differences-dataset-roadmap.md`
  → inconsistent card shape. This skill picks one.