Source-Type Registry
A research corpus outgrows "papers" — it catalogs preprints, code repos, blog
posts, lab announcements, vendor docs, books, RFCs, discussions, and more. This
skill is the canonical, extensible source-type registry that makes source
type a first-class, config-driven dimension.
The problem it solves
Corpora accumulate three drifting type vocabularies:
| Surface |
Example values |
Drift |
frontmatter type: |
book, reference, gap-note, internal-research |
mixes source type with doc role |
frontmatter source_type: |
conference-paper / conference_paper, book_chapter |
hyphen vs underscore |
| body "Source Type" |
paper, maintainer-doc, discussion |
a third overlapping enum |
The registry folds all three (plus a venue-classification fallback) into one
canonical source type per artifact.
How to use
# List the registry — canonical types + per-type rules
aiwg corpus source-types
aiwg corpus source-types --json
# The by-source-type index view groups the corpus by normalized type
aiwg index build --graph by-source-type # → indices/by-source-type.md
What each type declares
Per canonical type: template, required-sections, citation-format,
acquisition, storage, quality-rules, default-radar-cadence. Example:
| type |
template |
citation |
acquisition |
storage |
cadence |
| paper |
reference-academic |
doi-bibtex |
pdf-download |
sources/pdfs/full |
quarterly |
| preprint |
reference-academic |
arxiv-id |
pdf-download |
sources/pdfs/full |
quarterly |
| blog |
reference-web |
url-venue-retrieved |
web-snapshot |
sources/web |
biannual |
| repo |
reference-repo |
repo-url-commit |
git-clone |
sources/repos |
on-demand |
| standard |
reference-web |
standard-id |
web-snapshot |
sources/web |
annual |
| video |
reference-media |
timestamp-transcript |
media-curator |
media/video |
on-demand |
| podcast |
reference-media |
timestamp-transcript |
media-curator |
media/audio |
on-demand |
(17 canonical types ship by default; aiwg corpus source-types lists them all.)
Adding a new source type is config, not code
Create documentation/source-types.yaml in your corpus to replace the
default registry (include the defaults you keep). Adding a podcast, dataset, or
talk type is a registry entry:
version: 1
types:
podcast:
description: Podcast episode.
aliases: [podcast, episode]
template: reference-media
required_sections: [Citation, Media Profile, Summary, Key Timestamps]
citation_format: timestamp-transcript
acquisition: media-curator
storage: media/audio
quality_rules: interview-hedged-grade
default_radar_cadence: on-demand
venue_fallback: { … }
meta_roles: [redirect, stub, gap-note]
Normalization rules
source_type: → type: → body "Source Type" are checked in that order; the
first that matches a canonical type or alias wins.- Doc-role values (
redirect, stub, gap-note, merged, index) map to
the meta pseudo-type (excluded from source-type analytics).
- If no explicit type matches, the classified venue falls back to a type
(academic venues →
paper, arXiv → preprint, GitHub → repo, RFC →
standard, lab/vendor research posts → blog, Wikipedia → encyclopedia).
- Otherwise →
other.
Validated on a real 1,273-ref corpus: the registry normalizes ~93% (paper,
preprint, blog, repo, standard, book, chapter, encyclopedia, internal-review),
with meta excluding redirects/stubs and other capturing genuinely untyped refs.
Consumers
The registry is the foundation other subsystems read:
- by-source-type index view (this framework) — groups refs by normalized type.
- Per-type induction audit (
induction-audit / #1504) — required-section + depth checks vary by type (don't flag a blog for missing Ablation Studies).
- Acquisition dispatch (
research-acquire / #1507) — PDF download vs web snapshot vs git clone by type.
- Templates (#1497) — per-type reference templates selected by source type.
- Quality/GRADE — non-peer-reviewed types carry different hedging expectations.
Triggers
- "source type registry"
- "normalize source types"
- "what source types does the corpus have"
- "add a new source type"
- "by source type"
Notes
- Authoritative runtime default:
src/artifacts/corpus-tools/source-types.ts
(DEFAULT_SOURCE_TYPES); human-readable + override form:
agentic/code/frameworks/research-complete/config/source-types.yaml. A drift
test keeps them in sync.
- The venue fallback reuses the existing
VENUE_PATTERNS classifier
(src/artifacts/corpus-views/taxonomies.ts).
1---2namespace: aiwg3platforms: [all]4name: source-types5description: The canonical, extensible source-type registry for a research corpus — paper, preprint, blog, repo, book, chapter, standard, doc, discussion, encyclopedia, expert-material, video, audio, podcast, lecture, internal-review. Replaces the drifting type / source_type / "Source Type" vocabularies with one registry that declares per-type template, required sections, citation format, acquisition method, storage, quality rules, and radar cadence. Surfaced via `aiwg corpus source-types`; consumed by the by-source-type index view, per-type induction audit, and acquisition dispatch.6commandHint:7 argumentHint: "source-types [--json]"8 allowedTools: Read, Bash, Write9 model: haiku10 category: research-foundation11 modelRole: efficiency12 modelTier: economy13---1415# Source-Type Registry1617A research corpus outgrows "papers" — it catalogs preprints, code repos, blog18posts, lab announcements, vendor docs, books, RFCs, discussions, and more. This19skill is the **canonical, extensible source-type registry** that makes source20type a first-class, config-driven dimension.2122## The problem it solves2324Corpora accumulate **three drifting type vocabularies**:2526| Surface | Example values | Drift |27|---------|---------------|-------|28| frontmatter `type:` | `book`, `reference`, `gap-note`, `internal-research` | mixes source type with doc role |29| frontmatter `source_type:` | `conference-paper` / `conference_paper`, `book_chapter` | hyphen vs underscore |30| body "Source Type" | `paper`, `maintainer-doc`, `discussion` | a third overlapping enum |3132The registry folds all three (plus a venue-classification fallback) into **one33canonical source type** per artifact.3435## How to use3637```bash38# List the registry — canonical types + per-type rules39aiwg corpus source-types40aiwg corpus source-types --json4142# The by-source-type index view groups the corpus by normalized type43aiwg index build --graph by-source-type # → indices/by-source-type.md44```4546## What each type declares4748Per canonical type: `template`, `required-sections`, `citation-format`,49`acquisition`, `storage`, `quality-rules`, `default-radar-cadence`. Example:5051| type | template | citation | acquisition | storage | cadence |52|------|----------|----------|-------------|---------|---------|53| paper | reference-academic | doi-bibtex | pdf-download | sources/pdfs/full | quarterly |54| preprint | reference-academic | arxiv-id | pdf-download | sources/pdfs/full | quarterly |55| blog | reference-web | url-venue-retrieved | web-snapshot | sources/web | biannual |56| repo | reference-repo | repo-url-commit | git-clone | sources/repos | on-demand |57| standard | reference-web | standard-id | web-snapshot | sources/web | annual |58| video | reference-media | timestamp-transcript | media-curator | media/video | on-demand |59| podcast | reference-media | timestamp-transcript | media-curator | media/audio | on-demand |6061(17 canonical types ship by default; `aiwg corpus source-types` lists them all.)6263## Adding a new source type is config, not code6465Create `documentation/source-types.yaml` in your corpus to **replace** the66default registry (include the defaults you keep). Adding a podcast, dataset, or67talk type is a registry entry:6869```yaml70version: 171types:72 podcast:73 description: Podcast episode.74 aliases: [podcast, episode]75 template: reference-media76 required_sections: [Citation, Media Profile, Summary, Key Timestamps]77 citation_format: timestamp-transcript78 acquisition: media-curator79 storage: media/audio80 quality_rules: interview-hedged-grade81 default_radar_cadence: on-demand82venue_fallback: { … }83meta_roles: [redirect, stub, gap-note]84```8586## Normalization rules87881. `source_type:` → `type:` → body "Source Type" are checked in that order; the89 first that matches a canonical type or alias wins.902. Doc-role values (`redirect`, `stub`, `gap-note`, `merged`, `index`) map to91 the **`meta`** pseudo-type (excluded from source-type analytics).923. If no explicit type matches, the classified **venue** falls back to a type93 (academic venues → `paper`, arXiv → `preprint`, GitHub → `repo`, RFC →94 `standard`, lab/vendor research posts → `blog`, Wikipedia → `encyclopedia`).954. Otherwise → `other`.9697Validated on a real 1,273-ref corpus: the registry normalizes ~93% (paper,98preprint, blog, repo, standard, book, chapter, encyclopedia, internal-review),99with `meta` excluding redirects/stubs and `other` capturing genuinely untyped refs.100101## Consumers102103The registry is the foundation other subsystems read:104105- **by-source-type index view** (this framework) — groups refs by normalized type.106- **Per-type induction audit** (`induction-audit` / #1504) — required-section + depth checks vary by type (don't flag a blog for missing Ablation Studies).107- **Acquisition dispatch** (`research-acquire` / #1507) — PDF download vs web snapshot vs git clone by type.108- **Templates** (#1497) — per-type reference templates selected by source type.109- **Quality/GRADE** — non-peer-reviewed types carry different hedging expectations.110111## Triggers112113- "source type registry"114- "normalize source types"115- "what source types does the corpus have"116- "add a new source type"117- "by source type"118119## Notes120121- Authoritative runtime default: `src/artifacts/corpus-tools/source-types.ts`122 (`DEFAULT_SOURCE_TYPES`); human-readable + override form:123 `agentic/code/frameworks/research-complete/config/source-types.yaml`. A drift124 test keeps them in sync.125- The venue fallback reuses the existing `VENUE_PATTERNS` classifier126 (`src/artifacts/corpus-views/taxonomies.ts`).
jmagly 