Atlas ingest
A source is anything the knowledge base's claims can rest on. Ingesting does three things: converts it to Markdown the other skills can read
(sources/normalized/<id>.md), records it in sources/manifest.yaml (type, location, trust tier, commit or fetch date), and tells the team how far to
trust it. The other skills read only sources/normalized/ and the code repositories themselves, so a source that is not ingested does not exist
as far as they are concerned.
AGENTS.md has the trust order. Remember why it matters here: a source's tier decides how its claims are treated, and an AI-generated or undated
document is a set of claims to check, however fluent.
Ingest is also what makes citations checkable. Pages cite lines of sources/normalized/<id>.md, with a verbatim quote, and the manifest records a hash of that file at ingest (normalized_hash). lint-kb refuses a quote against a normalized file that no longer matches its hash, so
a source cannot be quietly edited to agree with a page. Never edit a normalized file by hand: fix the problem in the export and re-ingest. Re-ingesting changed content changes the hash, and pages that quoted the old text will fail lint until they are re-checked, which is the intended
way for a changed source to reach the pages built on it.
Choose the adapter
| The thing you have |
Adapter |
How |
Needs |
| A code repository (local clone, or just a remote URL) |
repo |
script |
git |
| A Markdown file |
markdown |
script |
nothing |
| A PDF |
pdf |
script |
pdftotext (poppler) |
A meeting transcript (.vtt or .txt) |
transcript |
script |
nothing |
| Pull requests of a GitHub repo |
github-pr |
script |
gh, logged in with access |
| Confluence, OneDrive/SharePoint, Rally |
not implemented yet |
- |
export first (PDF or Markdown), then use folder, markdown or pdf |
Details, limits and the interim routes: .atlas/adapters/ (one short doc per adapter, including the three not yet implemented).
Check the tools with node scripts/doctor.mjs if you are unsure.
If the team chose a provider
Manual is the default. Run atlas adapters (or node scripts/adapters.mjs) before asking the person to export anything: if a source type shows a provider whose status is ok (a team skill, an MCP server or a command), use that to fetch, read-only, and save the raw result in sources/exports/; then continue with folder below as if the person had exported it. Never write to kb/, sources/normalized/ or the manifest from a provider, never put a credential in a file, and never set up, switch or remove a provider yourself: if the status says TODO or needs attention, or a detected provider is not chosen, tell the person and let them run atlas adapters set. .atlas/adapters/README.md has the contract.
A folder of mixed files
If the person has a pile of exports, put them in sources/exports/ and run node scripts/ingest.mjs folder (or ... folder <path>) instead of one command per file. It picks the adapter by file type and prints one line per file. Read that report to the user: it names every file it skipped and why (Word, slides, web pages, video and audio need converting first; a file that looks like it holds a credential is refused), and for each file what it could not tell. It never claims what it cannot know: a document is stored at trust ai-or-undated unless the person says people wrote them (--trust human-docs), and a transcript without a date in its file name gets the file's own date, which may only be the download date (--date, or rename the file). Ask the person about those two things for the files that matter, then continue with steps 4 to 7 below for the new sources. The per-kind "how do I get it out of Confluence, OneDrive, Teams, Zoom" steps are in .atlas/docs/sources.md; point the person there instead of improvising them.
Steps
- Pick an id: lowercase letters, digits and hyphens (
app-repo, kitchen-sync-2026-09-03). Pages cite sources by this id, so keep it stable.
- Run the adapter, from the workspace root:
node scripts/ingest.mjs repo ../app-repo --id app-repo # local clone: referenced by URL/commit, never copied
node scripts/ingest.mjs markdown sources/exports/design.md --id design-doc
node scripts/ingest.mjs pdf sources/exports/spec.pdf --id requirements-spec
node scripts/ingest.mjs transcript sources/exports/sync.vtt --id kitchen-sync --date 2026-09-03
node scripts/ingest.mjs github-pr owner/name --id app-repo-prs --limit 50 --since 2026-06-01
Add --ai-generated if the document is machine-written or you cannot tell (it sets the trust tier to ai-or-undated), --trust <tier> to override the default,
--note "<text>" for context. Re-running with the same id refreshes that source.
- Put raw files in
sources/exports/ first when you can. That folder is git-ignored on purpose: Confluence, OneDrive and Rally exports often contain
credentials, customer data or HR content, and only the manifest should be committed. If a file is outside sources/exports/, tell the user the manifest
will record only its file name.
- Read what was produced. Open
sources/normalized/<id>.md and the new manifest entry. Check the adapter's notes it printed (truncated PRs, no text
extracted from a scanned PDF, uncommitted changes in a repo). If the extraction is poor (garbled tables, empty text), say so and consider an alternative route
(export to Markdown from the source tool). Check the file for secrets or personal data before it is committed. If there are any, remove them from the raw export and re-ingest (so the recorded hash matches), or do not commit the normalized file, and tell the user.
Do not redact the normalized file in place: the lint will then report it as edited after ingest.
- Judge reliability by spot-checking (skip for
repo, which is the reference for the others):
- Pick five to ten specific claims (file paths, service names, endpoints, config keys, command lines, dates) and look for them in the code.
- Compare the source's date with how much the code it describes has changed since (
git log --since=<date> -- <path>).
- Verdict: reliable (claims check out), mixed (some right, some wrong), stale (describes an older system), unverified (not checked yet, or nothing checkable).
- Record it:
node scripts/ingest.mjs mark <id> --reliability <verdict> --note "<what you checked and found>".
- File a finding for each contradiction between this source and the code (or another source), each side cited with a citation from
node scripts/cite.mjs, never a line number typed from memory: node scripts/findings.mjs new ... (node scripts/findings.mjs list first). Do not edit either side and do not decide which is right; a person closes the finding.
- Report to the user: what was registered, the tier, the verdict with the evidence behind it, and anything that could not be read.
- Add a line under
## Unreleased in CHANGELOG.md (Added source <id> (<type>)) and run node scripts/lint-kb.mjs so any page citing the source still resolves.
Notes on specific adapters
- repo: the normalized file is an orientation snapshot (layout, file types, recent commits, high-churn files), not a copy of the code. The manifest stores the remote URL and
the commit; the path of your local clone goes in
.atlas/local.yaml (git-ignored) because it differs per machine. check-stale needs that clone to say which files changed.
With --url <remote> and no path you get a reference-only entry.
- transcript: a record of one discussion. Its facts about why and who are valuable; its claims about what was built need confirming against code, PRs or tickets.
Speech-to-text mangles names, numbers and product terms, so tell the user which of those in the transcript look doubtful (a product name that matches nothing in the code, a figure that seems off) and confirm them elsewhere before any page states them. Pages cite a turn by its timestamp
(
@00:12:31) with a verbatim quote, so keep timestamps: ask for the .vtt export rather than a .txt where you can. Pass --date so the meeting date is recorded.
- github-pr: bodies and comments are clipped; this gives you what changed, why, and who reviewed. It does not replace reading the diff of a PR that matters.
- pdf: text layer only, no OCR. Tables and columns can come out jumbled.
Not implemented yet
For Confluence, OneDrive/SharePoint and Rally, do not pretend: tell the user the adapter does not exist yet, and walk them through the interim route in
.atlas/adapters/<name>.md (export to sources/exports/, then ingest as Markdown, PDF or CSV). Record in the --note where the content originally lived (for example
"exported from Confluence space PAY, page 'Checkout design', 2026-11-15") so it can be re-exported later.
1---2name: atlas-ingest3description: Registers a new source (code repo, Markdown, PDF, meeting transcript, GitHub pull requests) in the team knowledge-base workspace, normalizes it through the matching adapter, updates the manifest, and judges how reliable the source is by spot-checking it. Use when someone wants to add a source, import a document or transcript, connect a repo, or has an export from Confluence, OneDrive, SharePoint or Rally.4disable-model-invocation: true5metadata:6 source: https://skillmd.com/skills/chauthehienp3c1/atlas-ingest7---89# Atlas ingest1011A source is anything the knowledge base's claims can rest on. Ingesting does three things: converts it to Markdown the other skills can read12(`sources/normalized/<id>.md`), records it in `sources/manifest.yaml` (type, location, trust tier, commit or fetch date), and tells the team how far to13trust it. The other skills read **only** `sources/normalized/` and the code repositories themselves, so a source that is not ingested does not exist14as far as they are concerned.1516`AGENTS.md` has the trust order. Remember why it matters here: a source's tier decides how its claims are treated, and an AI-generated or undated17document is a set of claims to check, however fluent.1819**Ingest is also what makes citations checkable.** Pages cite lines of `sources/normalized/<id>.md`, with a verbatim quote, and the manifest records a hash of that file at ingest (`normalized_hash`). `lint-kb` refuses a quote against a normalized file that no longer matches its hash, so20a source cannot be quietly edited to agree with a page. Never edit a normalized file by hand: fix the problem in the export and re-ingest. Re-ingesting changed content changes the hash, and pages that quoted the old text will fail lint until they are re-checked, which is the intended21way for a changed source to reach the pages built on it.2223## Choose the adapter2425| The thing you have | Adapter | How | Needs |26|---|---|---|---|27| A code repository (local clone, or just a remote URL) | `repo` | script | `git` |28| A Markdown file | `markdown` | script | nothing |29| A PDF | `pdf` | script | `pdftotext` (poppler) |30| A meeting transcript (`.vtt` or `.txt`) | `transcript` | script | nothing |31| Pull requests of a GitHub repo | `github-pr` | script | `gh`, logged in with access |32| Confluence, OneDrive/SharePoint, Rally | not implemented yet | - | export first (PDF or Markdown), then use `folder`, `markdown` or `pdf` |3334Details, limits and the interim routes: `.atlas/adapters/` (one short doc per adapter, including the three not yet implemented).35Check the tools with `node scripts/doctor.mjs` if you are unsure.3637## If the team chose a provider3839Manual is the default. Run `atlas adapters` (or `node scripts/adapters.mjs`) before asking the person to export anything: if a source type shows a provider whose status is ok (a team skill, an MCP server or a command), use **that** to fetch, read-only, and save the raw result in `sources/exports/`; then continue with `folder` below as if the person had exported it. Never write to `kb/`, `sources/normalized/` or the manifest from a provider, never put a credential in a file, and never set up, switch or remove a provider yourself: if the status says TODO or needs attention, or a detected provider is not chosen, tell the person and let them run `atlas adapters set`. `.atlas/adapters/README.md` has the contract.4041## A folder of mixed files4243If the person has a pile of exports, put them in `sources/exports/` and run `node scripts/ingest.mjs folder` (or `... folder <path>`) instead of one command per file. It picks the adapter by file type and prints one line per file. **Read that report to the user**: it names every file it skipped and why (Word, slides, web pages, video and audio need converting first; a file that looks like it holds a credential is refused), and for each file what it could not tell. It never claims what it cannot know: a document is stored at trust `ai-or-undated` unless the person says people wrote them (`--trust human-docs`), and a transcript without a date in its file name gets the file's own date, which may only be the download date (`--date`, or rename the file). Ask the person about those two things for the files that matter, then continue with steps 4 to 7 below for the new sources. The per-kind "how do I get it out of Confluence, OneDrive, Teams, Zoom" steps are in `.atlas/docs/sources.md`; point the person there instead of improvising them.4445## Steps46471. **Pick an id**: lowercase letters, digits and hyphens (`app-repo`, `kitchen-sync-2026-09-03`). Pages cite sources by this id, so keep it stable.482. **Run the adapter**, from the workspace root:49 ```bash50 node scripts/ingest.mjs repo ../app-repo --id app-repo # local clone: referenced by URL/commit, never copied51 node scripts/ingest.mjs markdown sources/exports/design.md --id design-doc52 node scripts/ingest.mjs pdf sources/exports/spec.pdf --id requirements-spec53 node scripts/ingest.mjs transcript sources/exports/sync.vtt --id kitchen-sync --date 2026-09-0354 node scripts/ingest.mjs github-pr owner/name --id app-repo-prs --limit 50 --since 2026-06-0155 ```56 Add `--ai-generated` if the document is machine-written or you cannot tell (it sets the trust tier to `ai-or-undated`), `--trust <tier>` to override the default,57 `--note "<text>"` for context. Re-running with the same id refreshes that source.583. **Put raw files in `sources/exports/`** first when you can. That folder is git-ignored on purpose: Confluence, OneDrive and Rally exports often contain59 credentials, customer data or HR content, and only the manifest should be committed. If a file is outside `sources/exports/`, tell the user the manifest60 will record only its file name.614. **Read what was produced.** Open `sources/normalized/<id>.md` and the new manifest entry. Check the adapter's notes it printed (truncated PRs, no text62 extracted from a scanned PDF, uncommitted changes in a repo). If the extraction is poor (garbled tables, empty text), say so and consider an alternative route63 (export to Markdown from the source tool). **Check the file for secrets or personal data** before it is committed. If there are any, remove them from the raw export and re-ingest (so the recorded hash matches), or do not commit the normalized file, and tell the user.64 Do not redact the normalized file in place: the lint will then report it as edited after ingest.655. **Judge reliability by spot-checking** (skip for `repo`, which is the reference for the others):66 - Pick five to ten specific claims (file paths, service names, endpoints, config keys, command lines, dates) and look for them in the code.67 - Compare the source's date with how much the code it describes has changed since (`git log --since=<date> -- <path>`).68 - Verdict: **reliable** (claims check out), **mixed** (some right, some wrong), **stale** (describes an older system), **unverified** (not checked yet, or nothing checkable).69 - Record it: `node scripts/ingest.mjs mark <id> --reliability <verdict> --note "<what you checked and found>"`.70 - File a finding for each contradiction between this source and the code (or another source), each side cited with a citation from `node scripts/cite.mjs`, never a line number typed from memory: `node scripts/findings.mjs new ...` (`node scripts/findings.mjs list` first). Do not edit either side and do not decide which is right; a person closes the finding.716. **Report** to the user: what was registered, the tier, the verdict with the evidence behind it, and anything that could not be read.727. Add a line under `## Unreleased` in `CHANGELOG.md` (`Added source <id> (<type>)`) and run `node scripts/lint-kb.mjs` so any page citing the source still resolves.7374## Notes on specific adapters7576- **repo**: the normalized file is an orientation snapshot (layout, file types, recent commits, high-churn files), not a copy of the code. The manifest stores the remote URL and77 the commit; the path of your local clone goes in `.atlas/local.yaml` (git-ignored) because it differs per machine. `check-stale` needs that clone to say which files changed.78 With `--url <remote>` and no path you get a reference-only entry.79- **transcript**: a record of one discussion. Its facts about *why* and *who* are valuable; its claims about *what was built* need confirming against code, PRs or tickets.80 Speech-to-text mangles names, numbers and product terms, so tell the user which of those in the transcript look doubtful (a product name that matches nothing in the code, a figure that seems off) and confirm them elsewhere before any page states them. Pages cite a turn by its timestamp81 (`@00:12:31`) with a verbatim quote, so keep timestamps: ask for the `.vtt` export rather than a `.txt` where you can. Pass `--date` so the meeting date is recorded.82- **github-pr**: bodies and comments are clipped; this gives you what changed, why, and who reviewed. It does not replace reading the diff of a PR that matters.83- **pdf**: text layer only, no OCR. Tables and columns can come out jumbled.8485## Not implemented yet8687For Confluence, OneDrive/SharePoint and Rally, do not pretend: tell the user the adapter does not exist yet, and walk them through the interim route in88`.atlas/adapters/<name>.md` (export to `sources/exports/`, then ingest as Markdown, PDF or CSV). Record in the `--note` where the content originally lived (for example89"exported from Confluence space PAY, page 'Checkout design', 2026-11-15") so it can be re-exported later.