# Atlas Ingest

> Registers a new source (code repo, Markdown, PDF, meeting transcript, GitHub pull requests) in the team knowledge-base workspace, normalizes it through the matching adapter, updates the manifest, and judges how reliable the source is by spot-checking it. Use when someone wants to add a source, import a document or transcript, connect a repo, or has an export from Confluence, OneDrive, SharePoint or Rally.

- Skill: `chauthehienp3c1/atlas-ingest` (Agent Skill)
- Install (CLI): `npx get-skill add chauthehienp3c1/atlas-ingest`
- Raw SKILL.md: https://api.skillmd.com/api/skills/chauthehienp3c1/atlas-ingest/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: chauthehienp3c1 (https://skillmd.com/u/chauthehienp3c1)
- Updated: 2026-11-15
- Page: https://skillmd.com/skills/chauthehienp3c1/atlas-ingest

---


# Atlas ingest

A source is anything the knowledge base's claims can rest on. Ingesting does three things: converts it to Markdown the other skills can read
(`sources/normalized/<id>.md`), records it in `sources/manifest.yaml` (type, location, trust tier, commit or fetch date), and tells the team how far to
trust it. The other skills read **only** `sources/normalized/` and the code repositories themselves, so a source that is not ingested does not exist
as far as they are concerned.

`AGENTS.md` has the trust order. Remember why it matters here: a source's tier decides how its claims are treated, and an AI-generated or undated
document is a set of claims to check, however fluent.

**Ingest is also what makes citations checkable.** Pages cite lines of `sources/normalized/<id>.md`, with a verbatim quote, and the manifest records a hash of that file at ingest (`normalized_hash`). `lint-kb` refuses a quote against a normalized file that no longer matches its hash, so
a source cannot be quietly edited to agree with a page. Never edit a normalized file by hand: fix the problem in the export and re-ingest. Re-ingesting changed content changes the hash, and pages that quoted the old text will fail lint until they are re-checked, which is the intended
way for a changed source to reach the pages built on it.

## Choose the adapter

| The thing you have | Adapter | How | Needs |
|---|---|---|---|
| A code repository (local clone, or just a remote URL) | `repo` | script | `git` |
| A Markdown file | `markdown` | script | nothing |
| A PDF | `pdf` | script | `pdftotext` (poppler) |
| A meeting transcript (`.vtt` or `.txt`) | `transcript` | script | nothing |
| Pull requests of a GitHub repo | `github-pr` | script | `gh`, logged in with access |
| Confluence, OneDrive/SharePoint, Rally | not implemented yet | - | export first (PDF or Markdown), then use `folder`, `markdown` or `pdf` |

Details, limits and the interim routes: `.atlas/adapters/` (one short doc per adapter, including the three not yet implemented).
Check the tools with `node scripts/doctor.mjs` if you are unsure.

## If the team chose a provider

Manual is the default. Run `atlas adapters` (or `node scripts/adapters.mjs`) before asking the person to export anything: if a source type shows a provider whose status is ok (a team skill, an MCP server or a command), use **that** to fetch, read-only, and save the raw result in `sources/exports/`; then continue with `folder` below as if the person had exported it. Never write to `kb/`, `sources/normalized/` or the manifest from a provider, never put a credential in a file, and never set up, switch or remove a provider yourself: if the status says TODO or needs attention, or a detected provider is not chosen, tell the person and let them run `atlas adapters set`. `.atlas/adapters/README.md` has the contract.

## A folder of mixed files

If the person has a pile of exports, put them in `sources/exports/` and run `node scripts/ingest.mjs folder` (or `... folder <path>`) instead of one command per file. It picks the adapter by file type and prints one line per file. **Read that report to the user**: it names every file it skipped and why (Word, slides, web pages, video and audio need converting first; a file that looks like it holds a credential is refused), and for each file what it could not tell. It never claims what it cannot know: a document is stored at trust `ai-or-undated` unless the person says people wrote them (`--trust human-docs`), and a transcript without a date in its file name gets the file's own date, which may only be the download date (`--date`, or rename the file). Ask the person about those two things for the files that matter, then continue with steps 4 to 7 below for the new sources. The per-kind "how do I get it out of Confluence, OneDrive, Teams, Zoom" steps are in `.atlas/docs/sources.md`; point the person there instead of improvising them.

## Steps

1. **Pick an id**: lowercase letters, digits and hyphens (`app-repo`, `kitchen-sync-2026-09-03`). Pages cite sources by this id, so keep it stable.
2. **Run the adapter**, from the workspace root:
   ```bash
   node scripts/ingest.mjs repo ../app-repo --id app-repo            # local clone: referenced by URL/commit, never copied
   node scripts/ingest.mjs markdown sources/exports/design.md --id design-doc
   node scripts/ingest.mjs pdf sources/exports/spec.pdf --id requirements-spec
   node scripts/ingest.mjs transcript sources/exports/sync.vtt --id kitchen-sync --date 2026-09-03
   node scripts/ingest.mjs github-pr owner/name --id app-repo-prs --limit 50 --since 2026-06-01
   ```
   Add `--ai-generated` if the document is machine-written or you cannot tell (it sets the trust tier to `ai-or-undated`), `--trust <tier>` to override the default,
   `--note "<text>"` for context. Re-running with the same id refreshes that source.
3. **Put raw files in `sources/exports/`** first when you can. That folder is git-ignored on purpose: Confluence, OneDrive and Rally exports often contain
   credentials, customer data or HR content, and only the manifest should be committed. If a file is outside `sources/exports/`, tell the user the manifest
   will record only its file name.
4. **Read what was produced.** Open `sources/normalized/<id>.md` and the new manifest entry. Check the adapter's notes it printed (truncated PRs, no text
   extracted from a scanned PDF, uncommitted changes in a repo). If the extraction is poor (garbled tables, empty text), say so and consider an alternative route
   (export to Markdown from the source tool). **Check the file for secrets or personal data** before it is committed. If there are any, remove them from the raw export and re-ingest (so the recorded hash matches), or do not commit the normalized file, and tell the user.
   Do not redact the normalized file in place: the lint will then report it as edited after ingest.
5. **Judge reliability by spot-checking** (skip for `repo`, which is the reference for the others):
   - Pick five to ten specific claims (file paths, service names, endpoints, config keys, command lines, dates) and look for them in the code.
   - Compare the source's date with how much the code it describes has changed since (`git log --since=<date> -- <path>`).
   - Verdict: **reliable** (claims check out), **mixed** (some right, some wrong), **stale** (describes an older system), **unverified** (not checked yet, or nothing checkable).
   - Record it: `node scripts/ingest.mjs mark <id> --reliability <verdict> --note "<what you checked and found>"`.
   - File a finding for each contradiction between this source and the code (or another source), each side cited with a citation from `node scripts/cite.mjs`, never a line number typed from memory: `node scripts/findings.mjs new ...` (`node scripts/findings.mjs list` first). Do not edit either side and do not decide which is right; a person closes the finding.
6. **Report** to the user: what was registered, the tier, the verdict with the evidence behind it, and anything that could not be read.
7. Add a line under `## Unreleased` in `CHANGELOG.md` (`Added source <id> (<type>)`) and run `node scripts/lint-kb.mjs` so any page citing the source still resolves.

## Notes on specific adapters

- **repo**: the normalized file is an orientation snapshot (layout, file types, recent commits, high-churn files), not a copy of the code. The manifest stores the remote URL and
  the commit; the path of your local clone goes in `.atlas/local.yaml` (git-ignored) because it differs per machine. `check-stale` needs that clone to say which files changed.
  With `--url <remote>` and no path you get a reference-only entry.
- **transcript**: a record of one discussion. Its facts about *why* and *who* are valuable; its claims about *what was built* need confirming against code, PRs or tickets.
  Speech-to-text mangles names, numbers and product terms, so tell the user which of those in the transcript look doubtful (a product name that matches nothing in the code, a figure that seems off) and confirm them elsewhere before any page states them. Pages cite a turn by its timestamp
  (`@00:12:31`) with a verbatim quote, so keep timestamps: ask for the `.vtt` export rather than a `.txt` where you can. Pass `--date` so the meeting date is recorded.
- **github-pr**: bodies and comments are clipped; this gives you what changed, why, and who reviewed. It does not replace reading the diff of a PR that matters.
- **pdf**: text layer only, no OCR. Tables and columns can come out jumbled.

## Not implemented yet

For Confluence, OneDrive/SharePoint and Rally, do not pretend: tell the user the adapter does not exist yet, and walk them through the interim route in
`.atlas/adapters/<name>.md` (export to `sources/exports/`, then ingest as Markdown, PDF or CSV). Record in the `--note` where the content originally lived (for example
"exported from Confluence space PAY, page 'Checkout design', 2026-11-15") so it can be re-exported later.

