bbrysonelite-max Mine — run the agnostic Data Mine
One job: run datamine against a named vertical and show the ore. It's a one-liner.
Before every Mine invocation
Surface this reminder before every Mine run:
Last30Days freshness reminder: Before relying on Last30Days to choose what to mine, check or update the installed skill from
https://github.com/mvanhorn/last30days-skill. Do not silently update it; if the vertical is already selected, show this reminder and continue.
Last30Days supplies recent market-selection intelligence. Mine produces raw ore, and the refinery turns that raw ore into commercially valuable leads. This reminder is non-blocking for an already-selected vertical and does not authorize an update, credential change, or claim that Last30Days is current without verification.
Run it
The Data Mine is the Datamine package — GitHub bbrysonelite-max/Datamine, and on the iMac at ~/Datamine (2020iMac.local, SSH brentbryson w/ ~/.ssh/trashcan).
# from the repo (clone it if you don't have it: gh repo clone bbrysonelite-max/Datamine)
cd ~/Datamine
[ -d .venv ] || { python3 -m venv .venv && .venv/bin/pip install -q -e .; }
# Reddit hard-rate-limits a single IP (429s). Pull the Oxylabs residential-proxy
# creds from Secret Manager so the mine routes Reddit through them and runs fast
# (~80s vs 25+ min). No-op if not gcloud-authed — it falls back to direct fetch.
export OXYLABS_USERNAME="$(gcloud secrets versions access latest --secret=oxylabs-username 2>/dev/null)"
export OXYLABS_PASSWORD="$(gcloud secrets versions access latest --secret=oxylabs-password 2>/dev/null)"
.venv/bin/datamine --vertical <vertical-name> --out /tmp/mine-<vertical-name>
<vertical-name>= any file indatamine/verticals/(e.g.ssdi-work-fear). To target a NEW group, add adatamine/verticals/<name>.yaml(build phrases, fear phrases, keywords, per-surface queries) — no code change.- Output:
/tmp/mine-<name>.csv/.jsonl/.md— raw ore, one row per person: source, handle, url, build/fear signal, date, found_via, text. NOT enriched leads (no name/email/phone — that's the refinery). CSV is spreadsheet-safe; JSONL is the raw feed for the refinery. - After the run: report the count and the by-source / by-signal breakdown. Hand the JSONL to the refinery, or the CSV to Brent.
Notes
- 6 surfaces run by default: Reddit (Oxylabs), YouTube + TikTok (ScrapeCreators), X (xAI Grok), Quora (Serper), Hacker News. Keys auto-load from
~/.config/last30days/.env+~/Desktop/GitSync/kloop.env; Oxylabs is exported from Secret Manager above. - Ledger (
~/.datamine/<vertical>.json): re-runs skip already-mined people and dead surfaces. Use--no-ledgerfor a full fresh run;--ledger <path>to override. - The mine finds ore; the refinery turns ore into the diamond — identity (maigret/blue-healer) → contact → qualified. Different, gated step. See [[the-rebuild]].
- Override surfaces/window if needed:
--surfaces reddit youtube --days 14.
Codex Runtime
Resolve sibling Codex skills from installed skill roots (including ~/.codex/skills) or the current collection; do not assume a legacy skill root.
Never expose or print secret, credential, or token values.
Mandatory dependencies:
Datamine repository and environmentlast30days data-source credentials
Preflight each dependency using MCP/app capability discovery, CLI availability/version checks, read-only filesystem or Git checks for repositories, and provider auth-status commands without printing secrets, credentials, or tokens. If any mandatory dependency is unavailable, stop and report a concise blocked state naming the missing dependency and the next action needed.