Social Scraping

Social Media Scraping with StableSocial

Scrape profiles, posts, comments, followers, and search across 20+ platforms: TikTok, Instagram, YouTube, LinkedIn, X/Twitter, Facebook, Reddit, GitHub, Rumble, Threads, Bluesky, Pinterest, Twitch, Spotify, and more — plus ad libraries and a UGC research agent (Lightreel). All endpoints cost $0.06 per call.

Three endpoint families:

• Legacy platform endpoints (/api/tiktok/*, /api/instagram/*, /api/facebook/*, /api/reddit/*) — profiles, posts, comments, followers, search
• Scrape Creators suite (/api/sc/*) — much broader platform and data coverage, including transcripts, ad libraries, and niche platforms
• Lightreel UGC research agent (/api/lightreel/*) — durable-job research tasks (hooks, trends, creator search, scripts, briefs)

Setup

See rules/getting-started.md for installation and wallet setup.

Notes

Use npx agentcash@latest fetch for paid POST triggers. Use npx agentcash@latest fetch for free GET polling.

IMPORTANT: Use exact endpoint paths from the Quick Reference tables below. All paths include a platform prefix (e.g. https://stablesocial.dev/api/tiktok/...).

How It Works: Async Two-Step Flow

Every request follows a trigger-then-poll pattern:

Step 1: Trigger (paid, $0.06)

npx agentcash@latest fetch https://stablesocial.dev/api/instagram/profile -m POST -b '{"handle": "natgeo"}'

Returns 202 Accepted with a durable job ID, poll URL, and legacy JWT token:

{"jobId": "abc123", "status": "pending", "pollUrl": "https://stablesocial.dev/api/jobs/abc123", "token": "eyJhbGciOiJIUzI1NiIs..."}

Step 2: Poll (free)

Poll the durable job by ID (requires SIWX wallet auth from the paying wallet — the agentcash CLI handles this automatically):

npx agentcash@latest fetch https://stablesocial.dev/api/jobs/abc123

Or poll with the legacy token (no wallet auth needed):

npx agentcash@latest fetch "https://stablesocial.dev/api/jobs?token=eyJhbGciOiJIUzI1NiIs..."

• {"status": "pending"} — poll again in 3-5 seconds
• {"status": "finished", "data": {...}} — data is ready
• {"status": "failed", "error": "..."} — collection failed (not charged)

Tokens expire after 30 minutes; durable jobs remain pollable by jobId. Jobs typically finish in 5-60 seconds (Lightreel jobs can take longer). GET /api/jobs (no token) lists your durable jobs.

Quick Reference — TikTok

Input: {"handle": "username"} for profile/posts/followers. {"query": "keyword"} for search.

Quick Reference — Instagram

Input: {"handle": "username"} for profile/posts/followers. {"query": "keyword"} for search.

Quick Reference — Facebook

Input: {"handle": "username"} or {"profile_id": "id"} for profile. {"query": "keyword"} for search.

Quick Reference — Reddit

Input: {"post_id": "id"} for post details. {"query": "keyword"} for search. {"subreddit": "name"} for subreddit.

Quick Reference — Scrape Creators Suite (/api/sc/*)

A larger, newer suite (~155 endpoints, all $0.06, same trigger-then-poll flow) covering many more platforms and data types. Coverage by platform:

Representative examples:

npx agentcash@latest fetch https://stablesocial.dev/api/sc/tiktok/profile -m POST -b '{"handle": "username"}'
npx agentcash@latest fetch https://stablesocial.dev/api/sc/youtube/video/transcript -m POST -b '{"url": "https://youtube.com/watch?v=..."}'
npx agentcash@latest fetch https://stablesocial.dev/api/sc/linkedin/profile -m POST -b '{"url": "https://linkedin.com/in/someone"}'
npx agentcash@latest fetch https://stablesocial.dev/api/sc/facebook/adLibrary/search/ads -m POST -b '{"query": "brand name"}'

Input: varies per endpoint — many accept {"handle": ...} or an ID, some accept a full {"url": ...} (e.g. LinkedIn profile). Check the endpoint schema (npx agentcash@latest discover https://stablesocial.dev) when unsure.

Quick Reference — Lightreel UGC Research Agent (/api/lightreel/*)

An AI research agent for UGC/short-form content. All $0.06 per call; returns a durable job immediately — poll GET /api/jobs/{jobId}. Jobs are research tasks and can take minutes, not seconds.

Input: /chat takes {"question": "..."} (required), optional conversation_id to continue a conversation and response_fields (up to 5 fields, type string or array) for structured output. Task endpoints take task-specific fields — e.g. /top-hooks takes {"topic": "..."} (required) plus optional timeframe, platform, max_hooks, conversation_id.

npx agentcash@latest fetch https://stablesocial.dev/api/lightreel/top-hooks -m POST -b '{"topic": "skincare UGC ads"}'

Free SIWX-authed extras: GET /api/lightreel/chats lists your Lightreel chats; GET /api/lightreel/chat/{conversationId} fetches a transcript (must be the wallet that paid).

Data Dependencies

Some endpoints require a prior collection. For example, to get followers you must first trigger the profile:

# 1. Trigger profile collection
npx agentcash@latest fetch https://stablesocial.dev/api/instagram/profile -m POST -b '{"handle": "natgeo"}'
# Poll until finished...

# 2. Now fetch followers (depends on profile)
npx agentcash@latest fetch https://stablesocial.dev/api/instagram/followers -m POST -b '{"handle": "natgeo"}'
# Poll until finished...

Pagination

When results are paginated, the response includes page_info.has_next_page and a cursor. Pass the cursor to fetch the next page (each page is a new paid POST):

npx agentcash@latest fetch https://stablesocial.dev/api/tiktok/followers -m POST -b '{"handle": "username", "cursor": "abc123"}'

Key Parameters

• handle / profile_id — target account
• max_page_size — results per page (default varies, max 100)
• max_followers — how many followers to collect (default 500)
• max_posts / max_results — item limits (default 50)
• cursor — pagination cursor from previous response
• order_by — sort order: date_desc, date_asc, id_desc

Workflows

Profile Deep Dive

• (Optional) Check balance: npx agentcash@latest balance
• Trigger profile collection
• Poll until finished
• Trigger posts collection
• Poll until finished
• Optionally fetch comments, followers

Cross-Platform Search

• Search same keyword across multiple platforms
• Compare results and synthesize findings

npx agentcash@latest fetch https://stablesocial.dev/api/instagram/search -m POST -b '{"query": "brand name"}'
npx agentcash@latest fetch https://stablesocial.dev/api/tiktok/search -m POST -b '{"query": "brand name"}'
npx agentcash@latest fetch https://stablesocial.dev/api/reddit/search -m POST -b '{"query": "brand name"}'

Influencer Analysis

• Get profile on target platform
• Fetch recent posts with engagement
• Get follower list for audience analysis
• Check comments for sentiment

Competitive Intelligence

• Search for competitor on target platform
• Get competitor posts and engagement
• Monitor competitor mentions across platforms
• Analyze audience sentiment via comments

npx agentcash@latest fetch https://stablesocial.dev/api/instagram/profile -m POST -b '{"handle": "competitor"}'
npx agentcash@latest fetch https://stablesocial.dev/api/reddit/search -m POST -b '{"query": "competitor name"}'

Cost Estimation

All endpoints are $0.06 per trigger call. Polling is free.

vs social-intelligence Skill

The social-intelligence skill uses Reddit (stableenrich.dev). Use it for quick Reddit post lookups and discussions.

Use social-scraping (this skill) when you need:

• TikTok, Instagram, YouTube, LinkedIn, X/Twitter, Facebook, Reddit, GitHub, or other platform data (beyond basic Reddit search)
• Profiles, followers, following — not just search
• Comments, replies, reactions, transcripts on posts and videos
• Ad libraries, Marketplace, Events data
• UGC research (hooks, trends, creators, scripts) via Lightreel
• Cross-platform research