Ingest

Document Ingestion Skill

Turn external documents into structured, linkable knowledge notes with semantic deduplication.

Quick Start

/ingest ~/Downloads/attention-paper.pdf
/ingest https://example.com/api-docs --internal
/ingest spec.docx --title "API Specification v2"
/ingest paper.pdf --dry-run  # Preview without writing

Features

• Semantic deduplication: Merges similar concepts instead of creating duplicates
• Atomic notes: Extracts 3-15 typed concepts per document
• Multi-source accumulation: Concepts grow richer with each new source
• Dual mode: Literature (research) vs Internal (team docs)

Modes

• Literature (default): Creates literature/lit-{slug}.md for external research, papers, articles
• Internal (--internal): Creates internal/int-{slug}.md for team docs, internal specs, processes

Arguments

Prerequisites

This skill requires claude-note CLI for full functionality (deduplication, PDF parsing).

Check if installed:

claude-note --version

Install if needed:

# Using uv (recommended)
uv tool install git+https://github.com/crimeacs/claude-note.git

# Or using pipx
pipx install git+https://github.com/crimeacs/claude-note.git

Without CLI: The skill still works for basic ingestion using built-in tools (pdftotext, pandoc), but advanced features like semantic deduplication require the CLI.

Process

When the user invokes /ingest, follow these steps:

1. Read the Document

For files:

• Use the Read tool for .md and .txt files
• For .pdf files: use Bash with pdftotext or pandoc to extract text
• For .docx files: use Bash with pandoc -t plain to extract text

For URLs:

• Use WebFetch to retrieve the content

2. Get Configuration

Read the vault path from ~/.config/claude-note/config.toml:

cat ~/.config/claude-note/config.toml

Look for vault_root = "..." to find where notes should be created.

If no config exists, ask the user where to create notes.

3. Extract Knowledge

Analyze the content and extract:

• Summary: 2-3 sentence overview of what the document covers
• Key Concepts: Main ideas, terminology, and definitions (3-7 concepts)
• Highlights: Important findings, quotes, or actionable insights
• Questions: Open questions or things worth exploring further
• Related Topics: Connections to other knowledge areas

4. Check for Duplicates

Before creating a new note, check if a similar note exists:

ls {vault_path}/literature/ 2>/dev/null | grep -i "{keywords}"
ls {vault_path}/internal/ 2>/dev/null | grep -i "{keywords}"

If a similar note exists, ask the user if they want to:

1. Merge new content into the existing note
2. Create a separate note anyway
3. Cancel

5. Create the Note

Write a structured note using the appropriate format:

• Literature notes: See literature-format.md
• Internal notes: See internal-format.md

Filename conventions:

• Literature: literature/lit-{slug}.md
• Internal: internal/int-{slug}.md

Where {slug} is a kebab-case version of the title (max 50 chars).

6. Confirm

Tell the user what was created and offer to open the note.

Example Workflow

User: /ingest ~/Downloads/transformer-paper.pdf

1. Extract text from PDF:

   pdftotext ~/Downloads/transformer-paper.pdf - 2>/dev/null || pandoc ~/Downloads/transformer-paper.pdf -t plain
   

2. Read config:

   cat ~/.config/claude-note/config.toml
   

3. Analyze the content and extract knowledge

4. Create note at {vault_root}/literature/lit-attention-is-all-you-need.md

5. Confirm to user what was created

URL Ingestion

For URLs, use WebFetch to get the content:

/ingest https://docs.example.com/api-guide --internal

WebFetch will retrieve the content, then follow the same extraction process.

Dry Run

Preview what would be extracted without writing files:

/ingest paper.pdf --dry-run

This shows the extracted knowledge and where it would be saved, but doesn't create any files.

Tips

• Use descriptive filenames or --title for better slugs
• Review created notes and add manual annotations
• Link to existing topic notes in your vault using [[note-name]] syntax
• For large documents, focus on the most important 3-5 concepts
• Use --dry-run to preview before committing

See Also

• literature-format.md - Template for literature notes
• internal-format.md - Template for internal notes
• sample-output.md - Example of a created note

CLI Alternative

For batch processing or automation, use the CLI command instead:

claude-note ingest ~/papers/*.pdf

The CLI calls Claude API programmatically and is better for processing multiple documents.

Source: crimeacs/claude-note — distributed by TomeVault.