Data Catalog Lookup
Search across the workspace's data documentation to find datasets, variables, and cross-project data linkages.
Finding the workspace root
The workspace root contains CLAUDE.md and subdirectories projects/, pipelines/, data_catalog/, research/. If the current working directory is inside a project or pipeline, search upward to find the root. For example, if you're in projects/deterrence/, the root is two levels up.
Use git rev-parse --show-toplevel or search upward for CLAUDE.md with research/rules/workspace.md alongside it. Store the resolved root as $ROOT for the paths below.
Where to look
Search these files in order of relevance:
$ROOT/data_catalog/DATA_CATALOG.md (symlink to sibling repo) — master registry of all raw datasets with provenance, structure, and restrictions$ROOT/data_catalog/docs/codebooks/ (symlink) — detailed codebooks per dataset$ROOT/research/meta/variable_dictionary.md — shared variable definitions used across projects$ROOT/research/meta/data_linkages.md — how datasets link across projects (join keys, shared identifiers)- Project-level
docs/data.md files — in $ROOT/projects/*/docs/data.md
- Pipeline
docs/data.md and docs/summary.md — in $ROOT/pipelines/*/docs/
How to search
Based on the user's query:
- "What data do we have on X?" — Search DATA_CATALOG.md and codebooks for topic matches. Report: dataset name, provider, coverage, key variables, which projects use it.
- "What is variable X?" — Search variable_dictionary.md first, then project data.md files. Report: definition, construction, which datasets contain it, which projects use it.
- "How do datasets X and Y connect?" — Search data_linkages.md. Report: join keys, shared identifiers, known issues with linkage.
- "Which projects use dataset X?" — Search all project
docs/data.md files. Report: project name, how they use it, any sample restrictions.
- "What does pipeline X produce?" — Read the pipeline's
docs/summary.md and docs/data.md. Report: outputs, coverage, key variables.
Output format
## <Dataset or Variable Name>
**Source:** <provider, access method>
**Coverage:** <time period, geographic scope>
**Key variables:** <list>
**Used by:** <project1, project2, ...>
**Linked via:** <join keys to other datasets>
**Notes:** <restrictions, quality issues, gotchas>
If multiple results match, list them all with brief descriptions so the user can drill down.
Gotchas
- The
data_catalog/ directory contains symlinks to a sibling repository. If the symlinks don't resolve, tell the user they need to clone the data_catalog repo as a sibling.
- Raw data is in
data/ which is in .claudeignore — never try to read raw data files directly. Only use metadata from the catalog.
- Pipeline outputs in
build/ are typically gitignored and may not exist on disk. Describe what they produce based on docs, don't try to read the files.
- Some projects have data that contains PII (party names, case details). Flag this if relevant to the query.
1---2name: data3description: Search the data catalog, variable dictionary, and data linkages for datasets, variables, or data sources. Use when the user asks about available data, what variables exist, or how datasets connect across projects.4---56# Data Catalog Lookup78Search across the workspace's data documentation to find datasets, variables, and cross-project data linkages.910## Finding the workspace root1112The workspace root contains `CLAUDE.md` and subdirectories `projects/`, `pipelines/`, `data_catalog/`, `research/`. If the current working directory is inside a project or pipeline, search upward to find the root. For example, if you're in `projects/deterrence/`, the root is two levels up.1314Use `git rev-parse --show-toplevel` or search upward for `CLAUDE.md` with `research/rules/workspace.md` alongside it. Store the resolved root as `$ROOT` for the paths below.1516## Where to look1718Search these files in order of relevance:19201. **`$ROOT/data_catalog/DATA_CATALOG.md`** (symlink to sibling repo) — master registry of all raw datasets with provenance, structure, and restrictions212. **`$ROOT/data_catalog/docs/codebooks/`** (symlink) — detailed codebooks per dataset223. **`$ROOT/research/meta/variable_dictionary.md`** — shared variable definitions used across projects234. **`$ROOT/research/meta/data_linkages.md`** — how datasets link across projects (join keys, shared identifiers)245. **Project-level `docs/data.md`** files — in `$ROOT/projects/*/docs/data.md`256. **Pipeline `docs/data.md`** and `docs/summary.md` — in `$ROOT/pipelines/*/docs/`2627## How to search2829Based on the user's query:3031- **"What data do we have on X?"** — Search DATA_CATALOG.md and codebooks for topic matches. Report: dataset name, provider, coverage, key variables, which projects use it.32- **"What is variable X?"** — Search variable_dictionary.md first, then project data.md files. Report: definition, construction, which datasets contain it, which projects use it.33- **"How do datasets X and Y connect?"** — Search data_linkages.md. Report: join keys, shared identifiers, known issues with linkage.34- **"Which projects use dataset X?"** — Search all project `docs/data.md` files. Report: project name, how they use it, any sample restrictions.35- **"What does pipeline X produce?"** — Read the pipeline's `docs/summary.md` and `docs/data.md`. Report: outputs, coverage, key variables.3637## Output format3839```40## <Dataset or Variable Name>4142**Source:** <provider, access method>43**Coverage:** <time period, geographic scope>44**Key variables:** <list>45**Used by:** <project1, project2, ...>46**Linked via:** <join keys to other datasets>47**Notes:** <restrictions, quality issues, gotchas>48```4950If multiple results match, list them all with brief descriptions so the user can drill down.5152## Gotchas5354- The `data_catalog/` directory contains **symlinks** to a sibling repository. If the symlinks don't resolve, tell the user they need to clone the `data_catalog` repo as a sibling.55- Raw data is in `data/` which is in `.claudeignore` — never try to read raw data files directly. Only use metadata from the catalog.56- Pipeline outputs in `build/` are typically gitignored and may not exist on disk. Describe what they produce based on docs, don't try to read the files.57- Some projects have data that contains PII (party names, case details). Flag this if relevant to the query.
hsigstad