NLP Layer Guide
Overview of the NLP layer in narrativegraphs/nlp/.
Pipelines
| Pipeline | Purpose |
|---|---|
| Pipeline | Full narrative graph (triplets + cooccurrences) |
| CooccurrencePipeline | Simpler cooccurrence-only extraction |
Both handle: document ingestion → extraction → mapping → stats calculation.
Extraction Components
Entity Extraction (entities/)
- EntityExtractor - Abstract base class
- SpacyEntityExtractor - NER + noun chunks with length filtering, non-overlapping selection
Triplet Extraction (triplets/)
- TripletExtractor - Abstract base class
- DependencyGraphExtractor - Verb-first traversal; fine-grained boolean flags per relation type; no sentence-length limits
- EntityPairExtractor - Entity-pair traversal; declarative
PathPatternlist; supports compound "verb prep" predicates; guards:max_sentence_length=60,max_entity_distance=10
Cooccurrence Extraction (tuplets/)
- CooccurrenceExtractor - Abstract base class
- ChunkCooccurrenceExtractor - Sentence-windowed cooccurrences (default)
- DocumentCooccurrenceExtractor - All entity pairs in document
Mapping (mapping/)
Maps surface forms to canonical labels: dict[str, str]
- Mapper - Abstract base class
- StemmingMapper - Groups by Porter stemmed form
- SubgramStemmingMapper - Stemming + subgram matching
- SubgramLemmatizationMapper - Lemmatization + subgram matching (default)
Data Models (common/)
- SpanAnnotation - Text span with offsets (
text,start_char,end_char) - AnnotationContext - Context window (
text,doc_offset) - Triplet -
subj,pred,obj(SpanAnnotation) + optionalcontext - Tuplet -
entity_one,entity_two(SpanAnnotation) + optionalcontext
Architecture
Pipeline / CooccurrencePipeline
│
├── TripletExtractor → Triplet
├── EntityExtractor → SpanAnnotation
├── CooccurrenceExtractor → Tuplet
└── Mapper → canonical labels
Default Components
Pipeline:
DependencyGraphExtractor(triplets, default;EntityPairDependencyExtractoris an alternative)ChunkCooccurrenceExtractor(cooccurrences)SubgramLemmatizationMapper("noun")(entities)SubgramLemmatizationMapper("verb")(predicates)
CooccurrencePipeline:
SpacyEntityExtractor(entities)ChunkCooccurrenceExtractor(cooccurrences)SubgramLemmatizationMapper("noun")(entities)
Converted and distributed by TomeVault — claim your Tome and manage your conversions.