raven-sourav Skill: Newsletter Scraper
Scrape newsletter posts from Substack and Beehiiv publications. No API keys needed.
Trigger
User wants to scrape, research, or pull posts from a newsletter URL.
Supported Platforms
Substack
Any Substack publication (subdomain or custom domain).
python3 skills/scraper/scripts/scrape_substack.py --url "https://newsletter.substack.com" --days 30 --output summary
python3 skills/scraper/scripts/scrape_substack.py --url "https://www.lennysnewsletter.com" --keywords "AI" --output json
python3 skills/scraper/scripts/scrape_substack.py --url "https://newsletter.substack.com" --full-content --output json
Beehiiv
Any Beehiiv publication (including custom domains).
python3 skills/scraper/scripts/scrape_beehiiv.py --url "https://www.growthunhinged.com" --days 30 --output summary
python3 skills/scraper/scripts/scrape_beehiiv.py --url "https://www.growthunhinged.com" --keywords "pricing" --output json
python3 skills/scraper/scripts/scrape_beehiiv.py --url "https://www.growthunhinged.com" --full-content --output json
CLI Flags (both scripts)
| Flag | Default | Description |
|---|---|---|
--url |
required | Newsletter URL |
--keywords |
none | Filter by keywords (comma-separated, OR logic) |
--days |
all | Only include posts from last N days |
--max-posts |
all | Max posts to return |
--full-content |
false | Fetch full post body (slower) |
--output |
summary | summary (table) or json |
How It Works
- Substack: Uses public
/api/v1/archiveJSON endpoint. Returns title, subtitle, word count, reactions, comments, tags, author. - Beehiiv: Uses hidden
/postsJSON endpoint. Returns title, subtitle, tags, reading time, author.
Dependencies
Only requests — install via pip install requests.