Agent Reach Internet Access

---
name: agent-reach-internet-access
description: Give AI agents internet access to Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu and more — one CLI, zero API fees.
triggers:
  - give my agent internet access
  - install agent reach
  - help my AI search twitter reddit youtube
  - set up web scraping for my agent
  - agent reach setup and configuration
  - read tweets without paying for API
  - scrape social media for my AI agent
  - configure agent reach platforms
---

# Agent Reach

> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.

Agent Reach is a scaffolding tool that gives AI coding agents (Claude Code, Cursor, Windsurf, etc.) the ability to read and search the internet — Twitter/X, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu, WeChat, Weibo, RSS, and more — using free upstream CLI tools, zero paid APIs.

## Installation

### One-liner (recommended — tell your agent)

帮我安装 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md

Or in English:

Install Agent Reach for me: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md

### Manual install

```bash
pip install agent-reach

Safe mode (no auto system installs)

Install Agent Reach (safe mode): https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
Use the --safe flag

Update

帮我更新 Agent Reach：https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md

What the installer does

1. Runs pip install agent-reach
2. Installs system deps: Node.js, gh CLI, mcporter, xreach
3. Configures Exa search engine via MCP (free, no API key needed)
4. Detects environment (local vs server) and gives proxy advice
5. Registers SKILL.md in agent's skills directory

Key CLI Commands

# Check all channel statuses
agent-reach doctor

# Diagnose a specific channel
agent-reach doctor --channel twitter

# List all available channels
agent-reach channels

# Show version
agent-reach --version

Platform Support & Status

How Agents Use It (no commands to memorize)

After install, the agent reads SKILL.md and knows which upstream tool to call:

# Read any webpage
curl https://r.jina.ai/https://example.com

# Read a tweet
xreach tweet https://x.com/user/status/123 --json

# Extract YouTube subtitles
yt-dlp --skip-download --write-auto-sub --sub-format json3 "https://youtube.com/watch?v=VIDEO_ID"

# Search YouTube
yt-dlp "ytsearch5:LLM framework comparison 2025" --dump-json

# Read GitHub repo
gh repo view owner/repo-name

# Search GitHub
gh search repos "LLM framework" --language python --sort stars

# Search GitHub issues
gh search issues "bug label:help-wanted" --repo owner/repo

# Read RSS feed
python -c "import feedparser; f = feedparser.parse('https://example.com/feed.xml'); [print(e.title, e.link) for e in f.entries[:5]]"

# Search web (via Exa MCP — agent calls this automatically)
# No direct CLI; agent invokes via MCP tool call

Configuration Workflows

Twitter/X Cookie Setup

# 1. Install Chrome extension: Cookie-Editor
# 2. Log into twitter.com in browser
# 3. Click Cookie-Editor → Export → Copy JSON
# 4. Tell your agent: "帮我配 Twitter" and paste the cookie JSON
# Agent will save it to the correct location for xreach

Or tell agent: "帮我配 Twitter" — it will guide you step by step.

GitHub Authentication

gh auth login
# Follow interactive prompts: GitHub.com → HTTPS → browser auth

Or tell agent: "帮我登录 GitHub"

XiaoHongShu Setup

# Tell agent: "帮我配小红书"
# Agent will prompt for:
#   1. Cookie-Editor export from xiaohongshu.com
#   2. Starts xiaohongshu-mcp Docker container

Docker-based XHS MCP server:

docker run -d \
  -e XHS_COOKIE="$XHS_COOKIE" \
  -p 8080:8080 \
  ghcr.io/xpzouying/xiaohongshu-mcp:latest

Proxy Setup (for servers)

# Tell agent: "帮我配代理"
# Needed only on cloud servers (~$1/month), NOT on local machines
# Agent will configure HTTP_PROXY / HTTPS_PROXY env vars
export HTTP_PROXY=http://your-proxy:port
export HTTPS_PROXY=http://your-proxy:port

Podcast Transcription (小宇宙)

# Requires free Whisper API key
# Tell agent: "帮我配小宇宙播客"
export OPENAI_API_KEY=your_whisper_key

Channel Architecture

Each platform is a pluggable module in channels/:

channels/
├── web.py          → Jina Reader (r.jina.ai)
├── twitter.py      → xreach CLI
├── youtube.py      → yt-dlp
├── github.py       → gh CLI
├── bilibili.py     → yt-dlp
├── reddit.py       → JSON API + Exa MCP
├── xiaohongshu.py  → xiaohongshu-mcp (Docker)
├── douyin.py       → douyin-mcp-server
├── linkedin.py     → linkedin-mcp-server
├── wechat.py       → camoufox + miku_ai
├── weibo.py        → direct scraping
├── v2ex.py         → V2EX public API
├── rss.py          → feedparser
├── exa_search.py   → mcporter MCP
└── __init__.py     → channel registry

Each channel implements a check() method used by agent-reach doctor.

Custom channel example

# channels/hackernews.py
from agent_reach.base import BaseChannel

class HackerNewsChannel(BaseChannel):
    name = "hackernews"
    description = "Read Hacker News stories and comments"

    def check(self) -> dict:
        """Returns status dict for agent-reach doctor."""
        try:
            import urllib.request
            urllib.request.urlopen("https://hacker-news.firebaseio.com/v0/topstories.json", timeout=5)
            return {"status": "ok", "message": "HN API reachable"}
        except Exception as e:
            return {"status": "error", "message": str(e)}

    def get_top_stories(self, limit: int = 10) -> list:
        import urllib.request, json
        url = "https://hacker-news.firebaseio.com/v0/topstories.json"
        with urllib.request.urlopen(url) as r:
            ids = json.loads(r.read())[:limit]
        stories = []
        for sid in ids:
            with urllib.request.urlopen(f"https://hacker-news.firebaseio.com/v0/item/{sid}.json") as r:
                stories.append(json.loads(r.read()))
        return stories

Real Usage Patterns for Agents

Pattern 1: Research a topic across platforms

# Agent workflow for "research LLM frameworks":

# 1. Web search
# → MCP Exa tool call: search("best LLM frameworks 2025")

# 2. GitHub trending
import subprocess
result = subprocess.run(
    ["gh", "search", "repos", "LLM framework", "--sort", "stars", "--limit", "10", "--json", "name,description,stargazersCount,url"],
    capture_output=True, text=True
)
import json
repos = json.loads(result.stdout)

# 3. YouTube tutorials
result = subprocess.run(
    ["yt-dlp", "ytsearch5:LLM framework tutorial 2025", "--dump-json", "--flat-playlist"],
    capture_output=True, text=True
)

# 4. Reddit discussion
# → MCP Exa tool call: search("LLM framework site:reddit.com")

# 5. Read a specific article
import urllib.request
article = urllib.request.urlopen("https://r.jina.ai/https://example.com/llm-article").read().decode()

Pattern 2: Monitor Twitter for a topic

# Search recent tweets
xreach search "LLM framework" --limit 20 --json

# Get a user's timeline
xreach timeline @username --limit 50 --json

# Read a specific tweet thread
xreach tweet https://x.com/user/status/TWEET_ID --json --thread

Pattern 3: YouTube video summarization

# Extract subtitles for summarization
yt-dlp \
  --skip-download \
  --write-auto-sub \
  --sub-lang en \
  --sub-format vtt \
  --output "/tmp/%(id)s.%(ext)s" \
  "https://youtube.com/watch?v=VIDEO_ID"

# Get video metadata
yt-dlp --dump-json "https://youtube.com/watch?v=VIDEO_ID"

# Parse VTT subtitles into clean text
import re

def vtt_to_text(vtt_path: str) -> str:
    with open(vtt_path) as f:
        content = f.read()
    # Remove VTT headers and timestamps
    lines = content.split('\n')
    text_lines = []
    for line in lines:
        line = line.strip()
        if not line or line.startswith('WEBVTT') or '-->' in line or re.match(r'^\d+$', line):
            continue
        # Remove HTML tags
        line = re.sub(r'<[^>]+>', '', line)
        if line:
            text_lines.append(line)
    return ' '.join(text_lines)

Pattern 4: Read Weibo trending & posts

import subprocess, json

# Get trending topics
result = subprocess.run(
    ["agent-reach", "weibo", "trending"],
    capture_output=True, text=True
)
trends = json.loads(result.stdout)

# Search Weibo
result = subprocess.run(
    ["agent-reach", "weibo", "search", "AI大模型", "--limit", "20"],
    capture_output=True, text=True
)

Pattern 5: RSS feed aggregation

import feedparser

def read_feed(url: str, limit: int = 10) -> list[dict]:
    feed = feedparser.parse(url)
    return [
        {
            "title": entry.get("title", ""),
            "link": entry.get("link", ""),
            "summary": entry.get("summary", ""),
            "published": entry.get("published", ""),
        }
        for entry in feed.entries[:limit]
    ]

# Usage
hn_feed = read_feed("https://news.ycombinator.com/rss")
arxiv_feed = read_feed("https://export.arxiv.org/rss/cs.AI")
github_trending = read_feed("https://github.com/trending?since=daily")

Pattern 6: WeChat public account articles

# Search articles
agent-reach wechat search "AI Agent 2025" --limit 10

# Read full article as Markdown
agent-reach wechat read "https://mp.weixin.qq.com/s/ARTICLE_ID"

OpenClaw Users: Enable exec First

# Without this, agent can't run shell commands
openclaw config set tools.profile "coding"

# Or edit ~/.openclaw/openclaw.json:
# { "tools": { "profile": "coding" } }

# Restart gateway
openclaw gateway restart

Troubleshooting

agent-reach doctor output interpretation

agent-reach doctor

# Example output:
# ✅ web        Jina Reader reachable
# ✅ youtube    yt-dlp v2025.x.x installed
# ✅ github     gh CLI authenticated as @username
# ✅ rss        feedparser 6.x installed
# ⚠️  twitter   xreach installed, no cookie configured
# ❌ bilibili   yt-dlp blocked (server IP) — configure proxy
# ❌ reddit     proxy required for full access
# ✅ search     Exa MCP connected via mcporter

Common issues

Twitter returns empty results

# Cookie expired — re-export from browser
# 1. Open twitter.com, log in
# 2. Cookie-Editor → Export All → JSON
# 3. Tell agent: "帮我配 Twitter" and paste new cookies
xreach config --cookie-file ~/.xreach/cookies.json

Bilibili blocked on server

# Set proxy environment variables
export HTTP_PROXY=http://proxy-host:port
export HTTPS_PROXY=http://proxy-host:port
# Test
yt-dlp --dump-json "https://www.bilibili.com/video/BV_ID"

yt-dlp outdated (YouTube changes frequently)

pip install -U yt-dlp
# or
yt-dlp -U

gh CLI not authenticated

gh auth login
gh auth status  # verify

mcporter / Exa MCP not connecting

# Reinstall mcporter
npm install -g mcporter
# Check Node version (requires 18+)
node --version
# Restart MCP server
mcporter restart exa

XiaoHongShu Docker container down

docker ps | grep xiaohongshu
docker restart xiaohongshu-mcp
# Or re-run with updated cookie:
docker run -d \
  -e XHS_COOKIE="$XHS_COOKIE" \
  -p 8080:8080 \
  ghcr.io/xpzouying/xiaohongshu-mcp:latest

Jina Reader returns garbled content

# Try with Accept header
curl -H "Accept: text/plain" https://r.jina.ai/https://target-url.com
# Or use markdown mode
curl -H "X-Return-Format: markdown" https://r.jina.ai/https://target-url.com

Environment Variables Reference

# Proxy (server deployments only)
HTTP_PROXY=http://host:port
HTTPS_PROXY=http://host:port
NO_PROXY=localhost,127.0.0.1

# XiaoHongShu
XHS_COOKIE=<exported cookie JSON string>

# Douyin MCP
DOUYIN_COOKIE=<exported cookie JSON string>

# LinkedIn MCP
LINKEDIN_EMAIL=your@email.com
LINKEDIN_PASSWORD=$LINKEDIN_PASSWORD  # use secret manager

# Podcast transcription (Whisper)
OPENAI_API_KEY=<your key>

# Agent Reach config dir (default: ~/.agent-reach/)
AGENT_REACH_CONFIG_DIR=/custom/path

Project Structure

agent-reach/
├── agent_reach/
│   ├── cli.py          # CLI entry point (agent-reach command)
│   ├── doctor.py       # Health check logic
│   ├── base.py         # BaseChannel interface
│   └── channels/       # One file per platform
├── docs/
│   ├── install.md      # Agent-readable install instructions
│   ├── update.md       # Agent-readable update instructions
│   └── README_en.md    # English README
├── skills/
│   └── SKILL.md        # Installed to agent's skills directory
└── pyproject.toml

Quick Reference Card