Content Pipeline

Content Pipeline

Script-driven content production: ingest raw source, repurpose it into platform-native drafts, and gate the drafts before publish. All steps are non-interactive Python; chain them or run any stage standalone.

quote-mining ─┐
              ├─► content atoms ─► content-transform ─► drafts ─► quality-scorer ─► quality-gate ─► publish
editorial-brain ┘                       │
                          (optional in-loop expert panel from content-ops)

Setup

pip install -r requirements.txt      # anthropic, feedparser
cp .env.example .env                 # set ANTHROPIC_API_KEY; configure optional feeds/voice

All scripts read/write a data directory (default ./data/, override with CONTENT_OPS_DATA_DIR). Each stage writes a *-latest.json the next stage picks up.

Stages

1. Ingest — quote mining. Scan podcast RSS feeds + local meeting notes for quotable, contrarian, viral-worthy moments; emit scored candidates.

   python scripts/quote-mining-engine.py --days 90 --top 50 --min-score 60 \
     --feeds config/feeds.json --notes-dir ./notes/ --speaker "Name"
   

   Feeds come from --feeds <json>, QUOTE_MINING_FEEDS_FILE, or inline QUOTE_MINING_FEEDS. See config/feeds.example.json.

2. Ingest — editorial brain. Two-pass LLM clip discovery on a video transcript: pass 1 finds candidate hook→build→payoff moments, pass 2 deep-scores each on hook/build/payoff/clean-cut (0–100). Only clips at/above --min-score (default 90) are cut. Needs ANTHROPIC_API_KEY; video cutting needs yt-dlp + ffmpeg (see requirements.txt).

   python scripts/editorial-brain.py --url "https://youtube.com/watch?v=..." --max-clips 5
   python scripts/editorial-brain.py --vtt file.vtt --video-id ID --skip-cut   # analysis only
   

3. Transform. Repurpose long-form "content atoms" into platform-native drafts — X threads/posts, LinkedIn posts, YouTube Short scripts, newsletter sections. LLM mode is default; --template-only runs without the API. The optional in-loop expert panel (--no-expert-panel to disable) reuses content-ops's experts/ and scoring-rubrics/content-quality.md — see Cross-skill dependency below.

   python scripts/content-transform.py --atoms atoms.json --top-n 10
   python scripts/content-transform.py --atoms atoms.json --template-only
   

4. Score (batch, heuristic). Score a batch of drafts on five dimensions — voice similarity, specificity, AI-slop penalty, length appropriateness, engagement potential — and emit pass/fail per draft. No LLM; purely heuristic and fast. Default threshold 60; tune weights via --init-weights then edit data/quality-scorer-weights.json.

   python scripts/content-quality-scorer.py --input drafts.json --verbose
   python scripts/content-quality-scorer.py --threshold 75 --input drafts.json
   

5. Gate (publish filter). CI-style gate that runs the scorer and filters drafts below threshold; nothing publishes without passing. --conservative passes everything but annotates quality flags instead of dropping.

   python scripts/content-quality-gate.py --input drafts.json --threshold 75
   

Input formats

Content atoms (transform input):

{ "atoms": [ { "id": "atom-001", "content": "Long-form source…", "tags": ["AI"], "platforms_missing": ["x","linkedin"], "repurpose_score": 8 } ] }

Drafts (scorer/gate input):

{ "drafts": [ { "id": "draft-001", "platform": "x", "draft": "Content text…" } ] }

Cross-skill dependency

content-transform.py's optional in-loop expert panel does not duplicate the rubric — it reads the sibling content-ops skill's experts/ panels and scoring-rubrics/content-quality.md. The path resolves to ../content-ops/ by default; override with CONTENT_OPS_SKILL_DIR if the skills live elsewhere. content-ops remains the single source of truth for panel definitions.

Related skills

• content-ops — interactive expert-panel scorer; the canonical quality gate for a single artifact, and the source of the panels this pipeline reuses in content-transform
• autoresearch — pre-launch variant generation + multi-round optimization for conversion copy