Pulse Template

Pulse - Daily Curated News Digest

A personalized daily news digest skill with resilient multi-tier fetching. Goes out, reads the internet for you, picks the best stuff, and emails you a curated digest.

Features

• Multiple pulses — Separate digests for different topics
• Multi-source — Twitter/X, RSS/Atom blogs, Reddit, arXiv, YouTube/conferences
• Topic search — Searches Reddit and YouTube by topic (not just specific channels/subreddits)
• Resilient fetch chain — 5-tier fallback so nothing blocks you
• AI curation — Round-robin source diversity + engagement ranking
• Email delivery — Dark-themed HTML emails

Resilient Fetch Chain

Every source tries multiple methods in order. If one fails, it automatically tries the next:

Tier 1: Native tool (bird CLI, RSS parser, Reddit API, arXiv API)
  ↓ fails
Tier 2: Web search (Brave Search API → DuckDuckGo HTML scrape)
  ↓ fails
Tier 3: curl / urllib with rotating user agents
  ↓ fails
Tier 4: Playwright headless browser (handles JS-heavy sites)
  ↓ fails
Tier 5: Chrome DevTools Protocol (connects to running Chrome instance)

Per-source fallback chains:

Search fallback:

Brave Search API (needs free API key)
  ↓ no key or fails
DuckDuckGo HTML scrape (no key needed, always available)

Quick Start

1. Edit config.json — your topics, sources, and delivery
2. Optional: Set BRAVE_API_KEY env var for better search (free at brave.com/search/api)
3. Optional: pip install playwright && playwright install chromium for Tier 4
4. Test: python3 fetch_pulse.py my-digest --test
5. Schedule via cron

Configuration

What to Edit

Open config.json and customize:

Global (fetch_chain section):

• brave_api_key — Free Brave Search API key (recommended, massively improves fallbacks)
• cdp_url — Chrome DevTools URL if you run Chrome with --remote-debugging-port=9222

Per-pulse:

• id — Unique identifier (used in CLI: python3 fetch_pulse.py <id>)
• name — Display name in email header
• recipient — Email address to deliver to
• itemCount — How many items per digest (default 15)
• topics — Keywords that filter ALL sources for relevance
• sources — Enable/disable and configure each source type

Source Details

Twitter/X

"twitter": {
  "enabled": true,
  "accounts": ["person1", "person2"],
  "searches": ["search phrase"]
}

If bird CLI isn't installed, automatically falls back to web search for site:x.com queries, then tries fxtwitter API.

Blogs/RSS

"blogs": {
  "feeds": [
    {"name": "Display Name", "url": "https://example.com/feed.xml"}
  ]
}

Supports both RSS and Atom feeds. Uses full fetch chain for resilience.

Reddit

"reddit": {
  "subreddits": ["sub1", "sub2"]
}

Now does BOTH:

1. Monitors your listed subreddits (filtered by topic keywords)
2. Searches ALL of Reddit for your topics via the search API

YouTube/Conferences

"conferences": {
  "youtube_channels": [
    {"name": "Channel", "url": "https://www.youtube.com/feeds/videos.xml?channel_id=XXX"}
  ],
  "keywords": ["filter", "terms"]
}

Now does BOTH:

1. Monitors listed channels via RSS
2. If few results, searches YouTube broadly via web search

arXiv

"arxiv": {
  "categories": ["cs.AI"],
  "keywords": ["your keywords"]
}

Priority System

Curation Algorithm

1. Fetch — Hit all enabled sources with fallback chain
2. Bucket — Group items by source type
3. Sort — Each bucket sorted by priority × engagement
4. Round-robin — Take 1-2 from each bucket (ensures diversity)
5. Fill — Remaining slots filled by best items across all buckets
6. Dedupe — No duplicate URLs

Email Delivery

Edit the send_email() function in fetch_pulse.py for your email provider:

• Default: himalaya CLI (IMAP/SMTP)
• Alternatives: SendGrid, Mailgun, Gmail API, Python smtplib

Requirements

• Python 3.6+ (no pip dependencies for basic usage)
• Optional: pip install playwright && playwright install chromium (Tier 4 headless browser)
• Optional: pip install websocket-client (Tier 5 Chrome DevTools)
• Optional: Brave Search API key (free, improves search fallbacks significantly)
• Optional: bird CLI (Twitter native access)

Scheduling

# OpenClaw cron
cron add --schedule "0 7 * * *" --text "python3 /path/to/fetch_pulse.py my-digest"

# System crontab
0 7 * * * cd /path/to/pulse && python3 fetch_pulse.py my-digest