Event Finder

Event Finder — Search + Crawl + Email

Discovers events via SearXNG metasearch, extracts details with Crawl4AI, and emails the results.

Infrastructure

SearXNG (metasearch on localhost:8888):

• JSON API — Google, Bing, Brave, DuckDuckGo aggregated
• No rate limits, no API keys
• Start: source ~/.hermes/scripts/searxng-start.sh
• Source: ~/searxng/, config: ~/searxng/settings.yml

Crawl4AI (LLM-optimized crawler):

• Extracts clean markdown from any page (JS or static)
• Uses existing Playwright browsers
• API: result.markdown_v2.fit_markdown for clean output (NOT result.fit_markdown — removed in 0.9.x)
• LLMExtractionStrategy for structured data extraction

Email delivery:

• Uses existing ~/.hermes/outreach/outreach_toolkit.py (already configured)
• Email: luca.d.romeo@gmail.com
• Just call: python3 ~/.hermes/outreach/outreach_toolkit.py send --to luca.d.romeo@gmail.com --subject "..." --body "..."

Workflow

Step 1: MANDATORY — Clarify before searching

Do NOT start searching until you've asked these questions. User's definition of "tech" is narrow (transhumanism, EA, rationalism, AI alignment, futurology, economic/technological movements) — NOT mainstream AWS/Ruby/Rust meetups. Default assumptions produce irrelevant results.

Ask (via clarify tool or inline) until you have answers to ALL of these:

1. Topic focus: What specific domains? (e.g., transhumanism, effective altruism, AI alignment, neurotech, crypto/web3, biotech/longevity, economic systems, rationalist community, futurism, climate tech, etc.)

2. Format preference: Meetups? Conferences? Informal dinners/drinks? Lectures/talks? Workshops? Online OK or in-person only?

3. Date precision: Exact dates or flexible? How far out? Only events with confirmed dates, or "coming soon" with TBD dates OK?

4. Existing communities: Any specific groups/orgs you already know about? (e.g., LessWrong Vancouver, EA Vancouver, specific Meetup groups) — these should be prioritized.

5. Quality bar: Mainstream/general-interest OK or only niche/specialized? Free only or paid OK?

6. Location radius: City only or metro area? Would you travel to suburbs?

Minimum: get answers to #1 and #3 before searching. The rest can be inferred from past interactions but confirm if ambiguous.

Step 2: Search via SearXNG

Run multiple queries to cover different sources:

# General
curl -s "http://127.0.0.1:8888/search?q=<city>+<type>+events+<month>&format=json"

# Site-specific
curl -s "http://127.0.0.1:8888/search?q=site:meetup.com+<topic>+<city>&format=json"
curl -s "http://127.0.0.1:8888/search?q=site:eventbrite.com+<topic>+<city>&format=json"
curl -s "http://127.0.0.1:8888/search?q=site:lu.ma+<topic>+<city>&format=json"

# Concerts
curl -s "http://127.0.0.1:8888/search?q=site:songkick.com+<city>+concerts+<month>&format=json"

Parse with Python to deduplicate and collect URLs.

Step 3: Deep-crawl with Crawl4AI

For top results, extract event details. Write the crawl script to /tmp/ first, then run it — Python heredocs (python3 << 'PYEOF') get blocked by the security scanner.

# Save to /tmp/crawl_events.py, then: python3 /tmp/crawl_events.py
import asyncio
from crawl4ai import AsyncWebCrawler

async def extract():
    async with AsyncWebCrawler() as crawler:
        result = await crawler.arun(
            url='<event_url>',
            word_count_threshold=1
        )
        # Use markdown_v2.fit_markdown (NOT result.fit_markdown — removed in 0.9.x)
        md = result.markdown_v2.fit_markdown if hasattr(result, 'markdown_v2') else str(result.markdown)
        print(md[:3000])

asyncio.run(extract())

Extract from the markdown: event name, date, time, venue, price, link.

Fallback when crawling fails: If Crawl4AI times out (Eventbrite antibot, Luma JS overload), use SearXNG search result snippets directly — they often contain dates and key details already extracted by the search engine. T-Net BC Tech Events Calendar (bctechnology.com/events/) is particularly good for Vancouver — it lists dates and times inline in search snippets.

Step 4: Compile

Include price/cost and attendance requirements for EVERY event. User wants to know if they can just show up or need to register/pay.

Format as clean text, with two sections:

EVENTS IN <CITY> — <DATE RANGE> | <DOMAIN FILTER> | <FORMAT FILTER>
=====================================================================

CONFIRMED EVENTS
================
(only events with explicit confirmed dates in the window)

1. Event Name
   Date: Friday July 17, 7:00 PM
   Location: Venue Name, Address
   Price: Free / $XX / Members only
   Requirements: RSVP on Meetup / Register at link below / Just show up
   Domain: Transhumanism / EA / AI alignment / Crypto / etc.
   Link: https://...
   Notes: One-line relevance to user's interests

ACTIVE COMMUNITIES (no confirmed events in window, but worth tracking)
======================================================================
(communities user should join/follow for future events)

COMMUNITY NAME
   What: Monthly gatherings for X discussion
   Where: https://...
   Notes: Why relevant to user's domains

Honesty rule: If niche communities are sparse, say so upfront. Never pad with irrelevant events just to make the list longer. A short list of genuine matches is better than a long list of noise.

Step 5: Email

Use the existing outreach toolkit. Must pass env vars inline:

EMAIL_ADDRESS=luca.d.romeo@gmail.com \
EMAIL_PASSWORD=$(grep EMAIL_PASSWORD ~/.hermes/.env | cut -d= -f2) \
python3 ~/.hermes/outreach/outreach_toolkit.py send \
  --to luca.d.romeo@gmail.com \
  --subject "Vancouver Events — July 13-20" \
  --body "$(cat /tmp/events.txt)"

Credentials in ~/.hermes/.env. Use ASCII-safe subjects (hyphens, not em dashes — Unicode causes warnings).

Cron Job Setup (for recurring monitoring)

After a manual search confirms the pipeline works, set up automated weekly monitoring:

cronjob create "0 9 * * 1" \
  --name "vancouver-niche-events" \
  --skill event-finder \
  --prompt "<self-contained search+email instructions>" \
  --deliver local

Key patterns:

• --deliver local: The cron agent sends email via outreach_toolkit during execution — delivery is self-contained. No need for --deliver origin.
• --skill event-finder: Attaches this skill so the cron agent knows the tools and pitfalls.
• Prompt must be self-contained: specify domains, sources, email address, and the exact env-var-prefixed outreach command.
• Schedule: 0 9 * * 1 = Mondays 9 AM. Good for weekly event roundups.
• If zero events, still send email — don't go silent.

Current cron jobs for Vancouver:

• 8dc23f — niche (transhumanism, EA, AI alignment, crypto)
• 56a84c — startup/tech (deep tech, general startup, VC, crypto)

Event Sources

Reference Files

• references/vancouver-communities.md — Vancouver-specific communities for transhumanism, EA, rationalism, crypto/web3, and AI. Includes search patterns and confirmed active groups from real sessions.

Pitfalls

• SearXNG not running / health check: Don't trust ~/.searxng.pid — it can be stale after crashes. Verify with ss -tlnp | grep 8888. Quick health check: curl -s -o /dev/null -w "%{http_code}" "http://127.0.0.1:8888/". First request after server start may timeout — retry with --max-time 20.
• Crawl4AI fit_markdown removed: Use result.markdown_v2.fit_markdown, not result.fit_markdown. The old attribute was removed in 0.9.x. Fallback: str(result.markdown).
• Crawl4AI output size: 73KB+ per page is normal. Limit to first 3000 chars. Never dump raw markdown into context.
• Eventbrite antibot: Eventbrite blocks Crawl4AI (60s timeout on navigation). Don't waste time retrying — use SearXNG snippets for Eventbrite listings instead.
• Browser timeout on JS-heavy sites: Luma, Meetup, and Eventbrite can timeout the browser stack (60s+). Prefer SearXNG snippets + targeted Crawl4AI on lighter pages.
• execute_code sandbox: Does NOT have crawl4ai installed. Always run Crawl4AI via terminal (write script to /tmp/, then python3 /tmp/script.py).
• Python heredocs blocked: python3 << 'PYEOF' gets blocked by the security scanner. Write scripts to /tmp/ first, then run them.
• SearXNG snippet fallback: When deep crawling fails, search result snippets often already contain dates, times, and venues. Parse these directly — skip the crawl step entirely for sources like T-Net (bctechnology.com) and Meetup event listings that inline event details in search results. This is often FASTER and MORE RELIABLE than deep crawling.
• Outreach toolkit env vars: The outreach_toolkit.py send command needs EMAIL_ADDRESS and EMAIL_PASSWORD in the environment. Pattern: EMAIL_ADDRESS=luca.d.romeo@gmail.com EMAIL_PASSWORD=$(grep EMAIL_PASSWORD ~/.hermes/.env | cut -d= -f2) python3 ~/.hermes/outreach/outreach_toolkit.py send ...
• Outreach toolkit unicode: Em dashes and other Unicode in subject lines cause warnings (email still sends). Use ASCII-safe subjects: hyphens instead of em dashes.
• Duplicate events: Cross-reference across sources — same event often appears on multiple platforms.

Example Session

User: "Find tech meetups in Austin this week and email them to me"

1. curl "localhost:8888/search?q=Austin+tech+meetups+July+2026&format=json"
2. curl "localhost:8888/search?q=site:meetup.com+tech+Austin&format=json"
3. curl "localhost:8888/search?q=site:eventbrite.com+tech+Austin&format=json"
4. Parse, deduplicate, take top 8 URLs
5. Crawl each with Crawl4AI → extract dates/times/venues
6. Compile into formatted text
7. python3 ~/.hermes/outreach/outreach_toolkit.py send --to luca.d.romeo@gmail.com --subject "Tech Meetups in Austin This Week" --body "..."