Knowledge Source Reconnaissance
When to Use
- Preparing raw source inventories for LLM Wiki implementation
- Planning knowledge base consolidation across workspace-hub
- Auditing what knowledge exists before building new pipelines
- Creating GitHub issues for knowledge infrastructure work
- Any task requiring a "what do we know and where is it" answer
Core Principle
DO NOT re-scan directories. The workspace-hub ecosystem already has comprehensive intelligence infrastructure — read the registries, catalogs, and reports that already exist. Re-scanning is wasteful and misses the registry metadata (classification, status, relationships).
The Three Intelligence Systems
The workspace ecosystem tracks knowledge across three layers:
- Document/Resource Intelligence — Indexed local files (standards, conference papers, research literature, engineering refs)
- Online Intelligence — Remote resources cataloged for future download (papers, tools, APIs, data portals)
- Repo Intelligence — Engineering code, functions, standards implementations in digitalmodel repo
Scan Procedure
Read these specific files — they are the authoritative sources:
Phase 1: Document/Resource Intelligence
| File |
What It Contains |
Command |
data/document-index/index.jsonl |
Master document index (647K+ lines) |
wc -l for line count |
data/document-index/enhancement-plan.yaml |
Classified files by domain (1M+ files) |
Parse YAML, read by_domain section |
data/document-index/standards-transfer-ledger.yaml |
Standards tracking (status, impl, domains) |
Read summary section |
data/document-index/conference-index.jsonl |
Conference paper catalog (27K+ papers) |
wc -l for count |
data/document-index/conference-index-stats.yaml |
Conference stats per collection |
cat for full stats |
data/document-index/research-literature-report.md |
Domain-organized research PDFs |
cat for full breakdown |
data/document-index/engineering-refs-catalog.md |
Engineering reference files |
cat for catalog |
Phase 2: Online Intelligence
| File |
What It Contains |
Command |
data/document-index/online-resource-registry.yaml |
247 remote resources (tools, repos, papers, APIs) |
Read summary section for breakdown |
data/document-index/public-og-data-sources.yaml |
38 data API/portal sources (ingested + pending) |
Read already_ingested, known_not_ingested, newly_discovered |
data/document-index/conference-paper-catalog.yaml |
Conference paper metadata catalog |
wc -l for scope |
data/document-index/intelligence-accessibility-registry.yaml |
Discoverability/accessibility map for intelligence assets; flags hard-to-discover registries/wikis |
Read assets entries with discoverability/gaps |
data/document-index/resource-intelligence-maturity.yaml |
Canonical progress/coverage ledger for resource-intelligence parsing and review |
Read status section |
data/document-index/resource-intelligence-maturity.md |
Human summary only; may be stale relative to YAML |
Cross-check against YAML, do not treat as authoritative |
Phase 3: Repo Intelligence
| Path |
What It Contains |
Command |
knowledge/seeds/*.yaml |
Structured knowledge (career learnings, law cases, mooring failures, naval arch resources) |
ls -la + count entries per file |
knowledge-base/wrk-completions.jsonl |
Session work summaries (420 records) |
wc -l |
knowledge/dark-intelligence/ |
Excel-to-YAML extraction outputs |
`find -name "*.yaml" |
digitalmodel/specs/module-registry.yaml |
Engineering function registry |
wc -l for scope |
digitalmodel/ repo stats |
7,355 public functions, 42 standards impl |
Read README.md or capability report |
Phase 4: Mounted Filesystem Sources
| File |
What It Contains |
data/document-index/mounted-source-registry.yaml |
11 source roots with mount paths, dedup rules, availability checks |
Read the source_roots list — each entry has source_id, mount_root, local_or_remote, and canonical_storage_policy.
Output Format
Produce a markdown table organized by intelligence system with columns: Source Name, Location, Scale/Count, Status, Notes. Always include a summary table with totals.
## SCALE SUMMARY
| Category | Count | Notes |
|---|---|---|
| Classified documents | 1,033,933 | 12 domains via enhancement-plan.yaml |
| Conference papers | 27,735 | 30 collections |
| Research literature | 174 | 12 domain folders |
| Online resources | 247 | 221 pending download |
| Data API sources | 38 | 20 ingestable |
| Knowledge seeds | ~100 | YAML entries across 5 files |
| Mounted filesystems | 11 | Local + remote mounts |
| Engineering functions | 7,355 | digitalmodel repo |
| Standards tracked | 425 | 424 indexed, 1 implemented |
Key Insights
- The largest remaining semantic gap is summary coverage, not raw indexing —
data-audit-report.md shows 1,033,933 indexed records but only 639,585 with summaries (61.9%), leaving 394,348 records needing context enrichment.
- Index-level
other still hides 44,705 project/miscellaneous files — even though standards-level other has been eliminated, the document index still has a large miscellaneous bucket worth targeted reclassification.
- 221 of 247 online resources have
download_status: not_started — massive untapped source pool.
- Intelligence discoverability is itself a gap —
intelligence-accessibility-registry.yaml flags assets like online-resource-registry.yaml as hard-to-discover / not linked from navigation surfaces.
- Knowledge seeds are NOT all indexed by query-knowledge.sh — maritime-law-cases, mooring-failures, naval-architecture-resources are excluded.
- The riser-eng-job mount has 15,449 PDF/DOC/DOCX files across 4 projects (93GB) — a major literature source.
- DDE remote mounts have 18 unique standard orgs not present in /mnt/ace (ASME, AWS, NACE, etc.).
- Session corpus (wrk-completions.jsonl, 420 records) represents tacit institutional knowledge — perfect for wiki ingest once structured.
resource-intelligence-maturity.md can be stale; YAML is authoritative — on 2026-04-13 the Markdown still said 5 docs / 0 read while YAML showed 425 docs / 29 read / 6.8%.enhancement-plan.yaml may lag current audits — it still reported by_domain.other.count: 176,527, while newer audit artifacts reported index-level other at 44,705 and standards other eliminated.
Pitfalls
- Do NOT attempt to
find across /mnt/ace recursively — there are millions of files and it will hang
- Do NOT parse index.jsonl directly (572MB) — read the summary YAML/MD files instead
- Remote mounts (
/mnt/remote/) may be unavailable — check mount status before attempting to read
- Enhancement-plan.yaml may be stale relative to later audit artifacts; verify against
data-audit-report.md before citing other counts
resource-intelligence-maturity.md is a convenience summary only; always trust the YAML ledger if numbers disagree- Enhancement-plan.yaml is large — parse selectively, don't dump it whole
- Knowledge/dark-intelligence YAML files are gitignored — they exist locally but may not be on all machines
Related Skills
llm-wiki — the target system this inventory feeds intoknowledge-pipeline — existing knowledge workflow skeletondocument-inventory — generic single-directory scanner (don't use for workspace-hub recon)
1---2name: knowledge-source-recon3description: Reconnaissance pattern to inventory all knowledge sources across the workspace-hub ecosystem's existing intelligence infrastructure. Maps raw sources for LLM Wiki ingestion planning. Leverages pre-built registries and indexes rather than re-scanning directories.4---56# Knowledge Source Reconnaissance78## When to Use910- Preparing raw source inventories for LLM Wiki implementation11- Planning knowledge base consolidation across workspace-hub12- Auditing what knowledge exists before building new pipelines13- Creating GitHub issues for knowledge infrastructure work14- Any task requiring a "what do we know and where is it" answer1516## Core Principle1718DO NOT re-scan directories. The workspace-hub ecosystem already has comprehensive intelligence infrastructure — read the registries, catalogs, and reports that already exist. Re-scanning is wasteful and misses the registry metadata (classification, status, relationships).1920## The Three Intelligence Systems2122The workspace ecosystem tracks knowledge across three layers:23241. **Document/Resource Intelligence** — Indexed local files (standards, conference papers, research literature, engineering refs)252. **Online Intelligence** — Remote resources cataloged for future download (papers, tools, APIs, data portals)263. **Repo Intelligence** — Engineering code, functions, standards implementations in digitalmodel repo2728## Scan Procedure2930Read these specific files — they are the authoritative sources:3132### Phase 1: Document/Resource Intelligence3334| File | What It Contains | Command |35|---|---|---|36| `data/document-index/index.jsonl` | Master document index (647K+ lines) | `wc -l` for line count |37| `data/document-index/enhancement-plan.yaml` | Classified files by domain (1M+ files) | Parse YAML, read `by_domain` section |38| `data/document-index/standards-transfer-ledger.yaml` | Standards tracking (status, impl, domains) | Read `summary` section |39| `data/document-index/conference-index.jsonl` | Conference paper catalog (27K+ papers) | `wc -l` for count |40| `data/document-index/conference-index-stats.yaml` | Conference stats per collection | `cat` for full stats |41| `data/document-index/research-literature-report.md` | Domain-organized research PDFs | `cat` for full breakdown |42| `data/document-index/engineering-refs-catalog.md` | Engineering reference files | `cat` for catalog |4344### Phase 2: Online Intelligence4546| File | What It Contains | Command |47|---|---|---|48| `data/document-index/online-resource-registry.yaml` | 247 remote resources (tools, repos, papers, APIs) | Read `summary` section for breakdown |49| `data/document-index/public-og-data-sources.yaml` | 38 data API/portal sources (ingested + pending) | Read `already_ingested`, `known_not_ingested`, `newly_discovered` |50| `data/document-index/conference-paper-catalog.yaml` | Conference paper metadata catalog | `wc -l` for scope |51| `data/document-index/intelligence-accessibility-registry.yaml` | Discoverability/accessibility map for intelligence assets; flags hard-to-discover registries/wikis | Read `assets` entries with `discoverability`/`gaps` |52| `data/document-index/resource-intelligence-maturity.yaml` | Canonical progress/coverage ledger for resource-intelligence parsing and review | Read `status` section |53| `data/document-index/resource-intelligence-maturity.md` | Human summary only; may be stale relative to YAML | Cross-check against YAML, do not treat as authoritative |5455### Phase 3: Repo Intelligence5657| Path | What It Contains | Command |58|---|---|---|59| `knowledge/seeds/*.yaml` | Structured knowledge (career learnings, law cases, mooring failures, naval arch resources) | `ls -la` + count entries per file |60| `knowledge-base/wrk-completions.jsonl` | Session work summaries (420 records) | `wc -l` |61| `knowledge/dark-intelligence/` | Excel-to-YAML extraction outputs | `find -name "*.yaml" | wc -l` |62| `digitalmodel/specs/module-registry.yaml` | Engineering function registry | `wc -l` for scope |63| `digitalmodel/` repo stats | 7,355 public functions, 42 standards impl | Read README.md or capability report |6465### Phase 4: Mounted Filesystem Sources6667| File | What It Contains |68|---|---|69| `data/document-index/mounted-source-registry.yaml` | 11 source roots with mount paths, dedup rules, availability checks |7071Read the `source_roots` list — each entry has `source_id`, `mount_root`, `local_or_remote`, and `canonical_storage_policy`.7273## Output Format7475Produce a markdown table organized by intelligence system with columns: Source Name, Location, Scale/Count, Status, Notes. Always include a summary table with totals.7677```78## SCALE SUMMARY7980| Category | Count | Notes |81|---|---|---|82| Classified documents | 1,033,933 | 12 domains via enhancement-plan.yaml |83| Conference papers | 27,735 | 30 collections |84| Research literature | 174 | 12 domain folders |85| Online resources | 247 | 221 pending download |86| Data API sources | 38 | 20 ingestable |87| Knowledge seeds | ~100 | YAML entries across 5 files |88| Mounted filesystems | 11 | Local + remote mounts |89| Engineering functions | 7,355 | digitalmodel repo |90| Standards tracked | 425 | 424 indexed, 1 implemented |91```9293## Key Insights94951. **The largest remaining semantic gap is summary coverage, not raw indexing** — `data-audit-report.md` shows 1,033,933 indexed records but only 639,585 with summaries (61.9%), leaving 394,348 records needing context enrichment.962. **Index-level `other` still hides 44,705 project/miscellaneous files** — even though standards-level `other` has been eliminated, the document index still has a large miscellaneous bucket worth targeted reclassification.973. **221 of 247 online resources have `download_status: not_started`** — massive untapped source pool.984. **Intelligence discoverability is itself a gap** — `intelligence-accessibility-registry.yaml` flags assets like `online-resource-registry.yaml` as hard-to-discover / not linked from navigation surfaces.995. **Knowledge seeds are NOT all indexed** by query-knowledge.sh — maritime-law-cases, mooring-failures, naval-architecture-resources are excluded.1006. **The riser-eng-job mount has 15,449 PDF/DOC/DOCX files** across 4 projects (93GB) — a major literature source.1017. **DDE remote mounts have 18 unique standard orgs** not present in /mnt/ace (ASME, AWS, NACE, etc.).1028. **Session corpus (wrk-completions.jsonl, 420 records)** represents tacit institutional knowledge — perfect for wiki ingest once structured.1039. **`resource-intelligence-maturity.md` can be stale; YAML is authoritative** — on 2026-04-13 the Markdown still said 5 docs / 0 read while YAML showed 425 docs / 29 read / 6.8%.10410. **`enhancement-plan.yaml` may lag current audits** — it still reported `by_domain.other.count: 176,527`, while newer audit artifacts reported index-level `other` at 44,705 and standards `other` eliminated.105106## Pitfalls107108- Do NOT attempt to `find` across /mnt/ace recursively — there are millions of files and it will hang109- Do NOT parse index.jsonl directly (572MB) — read the summary YAML/MD files instead110- Remote mounts (`/mnt/remote/`) may be unavailable — check mount status before attempting to read111- Enhancement-plan.yaml may be stale relative to later audit artifacts; verify against `data-audit-report.md` before citing `other` counts112- `resource-intelligence-maturity.md` is a convenience summary only; always trust the YAML ledger if numbers disagree113- Enhancement-plan.yaml is large — parse selectively, don't dump it whole114- Knowledge/dark-intelligence YAML files are gitignored — they exist locally but may not be on all machines115116## Related Skills117118- `llm-wiki` — the target system this inventory feeds into119- `knowledge-pipeline` — existing knowledge workflow skeleton120- `document-inventory` — generic single-directory scanner (don't use for workspace-hub recon)
vamseeachanta