How do I install the Wsb Data Pipeline skill?

Run npx skillmds@latest add faaz17/wsb-data-pipeline in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Wsb Data Pipeline safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Wsb Data Pipeline?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Wsb Data Pipeline free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Wsb Data Pipeline?

Faaz17 (@faaz17) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Wsb Data Pipeline by faaz17 — Agent Skill

Name: Wsb Data Pipeline
Author: Faaz17

Wsb Data Pipeline

Scrape r/wallstreetbets posts for specific stock/forex tickers (TSLA, AAPL, XAUUSD, NVDA, GME) over historical date ranges and export Post Title, Date, Upvotes, Number of Comments, and Ticker Mentions to an Excel workbook. Use when the user asks to collect, update, or re-run WSB Reddit data for the herd-behavior research project, or asks about ticker date ranges, the data source, or output columns for that dataset.

Faaz17 3df9032 3.7 KB Updated Sep 17, 2026

File contents

WSB Data Pipeline

Collects historical r/wallstreetbets post data per ticker into output/wsb_ticker_data.xlsx (one sheet per ticker) for the herd-behavior sentiment research (see Research Points.docx).

Data source

Reddit's live API/search does not return multi-year-old posts reliably. This pipeline uses the Arctic Shift API (https://arctic-shift.photon-reddit.com/api/posts/search), a free public mirror of the Pushshift historical Reddit archive, instead. No Reddit credentials are required.

Running the pipeline

Always run the bundled script — do not hand-write scraping/Excel code, the script already handles pagination, rate-limit backoff, caching, and deduping.

cd "<project root>"
.venv/Scripts/python.exe scripts/scrape_wsb.py                 # all 5 tickers, full configured ranges
.venv/Scripts/python.exe scripts/scrape_wsb.py --ticker GME     # one ticker only
.venv/Scripts/python.exe scripts/scrape_wsb.py --ticker TSLA --after 2024-01-01 --before 2024-02-01  # custom range
.venv/Scripts/python.exe scripts/scrape_wsb.py --no-cache       # force re-fetch, ignore cached API pages

On Windows PowerShell use .\.venv\Scripts\python.exe instead of the forward-slash form.

Each run merges into the existing output/wsb_ticker_data.xlsx (sheets for tickers not re-run are preserved), so a single ticker can be refreshed without re-scraping the rest.

Configured ticker ranges (in `scripts/scrape_wsb.py`, `TICKERS` dict)

Ticker	Aliases searched	Date range
TSLA	TSLA, Tesla	last 5 years (2021-06-30 to 2026-06-30)
AAPL	AAPL, Apple	last 3 years (2023-06-30 to 2026-06-30)
XAUUSD	XAUUSD, XAU/USD, Gold	last 3 years (2023-06-30 to 2026-06-30)
NVDA	NVDA, Nvidia, NVIDIA	last 4 years (2022-06-30 to 2026-06-30)
GME	GME, GameStop	2020-01-01 to 2021-12-31 (fixed, per research scope)

If the user wants different ranges or aliases, edit the TICKERS dict at the top of scripts/scrape_wsb.py rather than building one-off scripts.

Output columns

Post Title, Date (UTC, ISO YYYY-MM-DD), Upvotes (Reddit score), Number of Comments, Ticker Mentions (regex word-boundary count of the ticker's aliases across title + selftext — use this as the per-post mention intensity, not a list of tickers).

Things to watch for

Long runs: TSLA/NVDA over multi-year windows can mean hundreds of API pages. Run tickers one at a time (--ticker X) if you want incremental progress/output rather than waiting for all 5.
Caching: raw API pages are cached under cache/<alias>/<after_ts>.json. A re-run with the same date range is fast and free of API calls; use --no-cache only if the underlying data may have changed.
Rate limits: the script checks the X-Ratelimit-Remaining response header and sleeps automatically when low — do not remove this or run multiple scrape processes in parallel against the same API.
XAUUSD is barely discussed on WSB by exact ticker; the "Gold" alias is included to catch gold-related posts, but expect a much smaller dataset than the equities.
After scraping, do not manually edit output/wsb_ticker_data.xlsx — re-run the script with updated ranges/aliases instead, so the file stays reproducible from scripts/scrape_wsb.py.

---name: wsb-data-pipelinedescription: Scrape r/wallstreetbets posts for specific stock/forex tickers (TSLA, AAPL, XAUUSD, NVDA, GME) over historical date ranges and export Post Title, Date, Upvotes, Number of Comments, and Ticker Mentions to an Excel workbook. Use when the user asks to collect, update, or re-run WSB Reddit data for the herd-behavior research project, or asks about ticker date ranges, the data source, or output columns for that dataset.---# WSB Data PipelineCollects historical r/wallstreetbets post data per ticker into `output/wsb_ticker_data.xlsx` (one sheet per ticker) for the herd-behavior sentiment research (see `Research Points.docx`).## Data sourceReddit's live API/search does not return multi-year-old posts reliably. This pipeline uses the **Arctic Shift API** (`https://arctic-shift.photon-reddit.com/api/posts/search`), a free public mirror of the Pushshift historical Reddit archive, instead. No Reddit credentials are required.## Running the pipelineAlways run the bundled script — do not hand-write scraping/Excel code, the script already handles pagination, rate-limit backoff, caching, and deduping.```bashcd "<project root>".venv/Scripts/python.exe scripts/scrape_wsb.py # all 5 tickers, full configured ranges.venv/Scripts/python.exe scripts/scrape_wsb.py --ticker GME # one ticker only.venv/Scripts/python.exe scripts/scrape_wsb.py --ticker TSLA --after 2024-01-01 --before 2024-02-01 # custom range.venv/Scripts/python.exe scripts/scrape_wsb.py --no-cache # force re-fetch, ignore cached API pages```On Windows PowerShell use `.\.venv\Scripts\python.exe` instead of the forward-slash form.Each run merges into the existing `output/wsb_ticker_data.xlsx` (sheets for tickers not re-run are preserved), so a single ticker can be refreshed without re-scraping the rest.## Configured ticker ranges (in `scripts/scrape_wsb.py`, `TICKERS` dict)| Ticker | Aliases searched | Date range ||---|---|---|| TSLA | TSLA, Tesla | last 5 years (2021-06-30 to 2026-06-30) || AAPL | AAPL, Apple | last 3 years (2023-06-30 to 2026-06-30) || XAUUSD | XAUUSD, XAU/USD, Gold | last 3 years (2023-06-30 to 2026-06-30) || NVDA | NVDA, Nvidia, NVIDIA | last 4 years (2022-06-30 to 2026-06-30) || GME | GME, GameStop | 2020-01-01 to 2021-12-31 (fixed, per research scope) |If the user wants different ranges or aliases, edit the `TICKERS` dict at the top of `scripts/scrape_wsb.py` rather than building one-off scripts.## Output columns`Post Title`, `Date` (UTC, ISO `YYYY-MM-DD`), `Upvotes` (Reddit score), `Number of Comments`, `Ticker Mentions` (regex word-boundary count of the ticker's aliases across title + selftext — use this as the per-post mention intensity, not a list of tickers).## Things to watch for- **Long runs**: TSLA/NVDA over multi-year windows can mean hundreds of API pages. Run tickers one at a time (`--ticker X`) if you want incremental progress/output rather than waiting for all 5.- **Caching**: raw API pages are cached under `cache/<alias>/<after_ts>.json`. A re-run with the same date range is fast and free of API calls; use `--no-cache` only if the underlying data may have changed.- **Rate limits**: the script checks the `X-Ratelimit-Remaining` response header and sleeps automatically when low — do not remove this or run multiple scrape processes in parallel against the same API.- **XAUUSD is barely discussed on WSB** by exact ticker; the "Gold" alias is included to catch gold-related posts, but expect a much smaller dataset than the equities.- After scraping, do not manually edit `output/wsb_ticker_data.xlsx` — re-run the script with updated ranges/aliases instead, so the file stays reproducible from `scripts/scrape_wsb.py`.

Frequently asked questions