# Nlp Guide

> Overview of the NLP layer for text extraction and mapping Use when this capability is needed.

- Skill: `tomevault-io/nlp-guide` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add tomevault-io/nlp-guide`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/nlp-guide/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: tomevault-io (https://skillmd.com/u/tomevault-io)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/tomevault-io/nlp-guide

---


# NLP Layer Guide

Overview of the NLP layer in `narrativegraphs/nlp/`.

## Pipelines

| Pipeline                 | Purpose                                         |
| ------------------------ | ----------------------------------------------- |
| **Pipeline**             | Full narrative graph (triplets + cooccurrences) |
| **CooccurrencePipeline** | Simpler cooccurrence-only extraction            |

Both handle: document ingestion → extraction → mapping → stats calculation.

## Extraction Components

### Entity Extraction (`entities/`)

- **EntityExtractor** - Abstract base class
- **SpacyEntityExtractor** - NER + noun chunks with length filtering, non-overlapping selection

### Triplet Extraction (`triplets/`)

- **TripletExtractor** - Abstract base class
- **DependencyGraphExtractor** - Verb-first traversal; fine-grained boolean flags per relation type; no sentence-length limits
- **EntityPairExtractor** - Entity-pair traversal; declarative `PathPattern` list; supports compound "verb prep" predicates; guards: `max_sentence_length=60`, `max_entity_distance=10`

### Cooccurrence Extraction (`tuplets/`)

- **CooccurrenceExtractor** - Abstract base class
- **ChunkCooccurrenceExtractor** - Sentence-windowed cooccurrences (default)
- **DocumentCooccurrenceExtractor** - All entity pairs in document

## Mapping (`mapping/`)

Maps surface forms to canonical labels: `dict[str, str]`

- **Mapper** - Abstract base class
- **StemmingMapper** - Groups by Porter stemmed form
- **SubgramStemmingMapper** - Stemming + subgram matching
- **SubgramLemmatizationMapper** - Lemmatization + subgram matching (default)

## Data Models (`common/`)

- **SpanAnnotation** - Text span with offsets (`text`, `start_char`, `end_char`)
- **AnnotationContext** - Context window (`text`, `doc_offset`)
- **Triplet** - `subj`, `pred`, `obj` (SpanAnnotation) + optional `context`
- **Tuplet** - `entity_one`, `entity_two` (SpanAnnotation) + optional `context`

## Architecture

```
Pipeline / CooccurrencePipeline
    │
    ├── TripletExtractor → Triplet
    ├── EntityExtractor → SpanAnnotation
    ├── CooccurrenceExtractor → Tuplet
    └── Mapper → canonical labels
```

## Default Components

**Pipeline:**

- `DependencyGraphExtractor` (triplets, default; `EntityPairDependencyExtractor` is an alternative)
- `ChunkCooccurrenceExtractor` (cooccurrences)
- `SubgramLemmatizationMapper("noun")` (entities)
- `SubgramLemmatizationMapper("verb")` (predicates)

**CooccurrencePipeline:**

- `SpacyEntityExtractor` (entities)
- `ChunkCooccurrenceExtractor` (cooccurrences)
- `SubgramLemmatizationMapper("noun")` (entities)

---
> Converted and distributed by [TomeVault](https://tomevault.io/claim/kasperfyhn) — claim your Tome and manage your conversions.
<!-- tomevault:4.0:skill_md:2026-04-13 -->