How do I install the Gated Content Retrieval skill?

Run npx skillmds add theheavenlyd3mon/gated-content-retrieval in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Gated Content Retrieval safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Gated Content Retrieval?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Gated Content Retrieval free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Gated Content Retrieval?

theheavenlyd3mon (@theheavenlyd3mon) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Gated Content Retrieval by theheavenlyd3mon

Name: Gated Content Retrieval
Rating: 4.2 (5 reviews)
Author: theheavenlyd3mon

File contents

gated-content-retrieval

Reference notes for pulling web content when normal scrapers return empty, timeout, or say "not supported" on a protected URL.

xurl article API (no browser)

xurl can fetch an X Article body: request tweet.fields=article on /2/tweets/<ID> and read data.article.plain_text (+ .title).

Watch out: xurl may print a non-JSON warning line before the JSON. Piping its stdout straight into python3 -c "json.load(...)" fails with JSONDecodeError: Extra data (line 7). Redirect to a file first, then parse. If the API answers 402 ... credits-depleted, this route is dead — fall through to the browser route.

GitHub file URLs (PDFs, notebooks, binaries)

A github.com/<org>/<repo>/blob/main/<file> URL returns the HTML viewer page, NOT the file — web_extract on it yields GitHub chrome ("Uh oh!", file tree, sign-in prompts). Rewrite to raw.githubusercontent.com/<org>/<repo>/main/<file> before extracting. Works for PDFs (web_extract parses them) and any raw text.

Browserbase dual-control session

browser_navigate creates ONE Browserbase session. That same session is controllable by the agent over CDP and simultaneously viewable/operable from the Browserbase dashboard → Sessions → Live View. keepAlive defaults true, so the session survives across turns. Because one cloud browser is both agent-driven and human-visible, a person can advance it past any step the agent cannot automate, then the agent resumes.

For long articles the accessibility snapshot truncates — prefer a browser_console JS eval that returns the article's plain_text field; fallback browser_snapshot(full=true).

Browserbase is a paid backend; on a free plan the plugin auto-drops proxies/keepAlive on a 402 but the session still works. This is NOT computer-use.

---name: gated-content-retrievaldescription: Notes on fetching web content ordinary scrapers cannot reach — the xurl article API and the Browserbase dual-control browser session.--- # gated-content-retrieval Reference notes for pulling web content when normal scrapers return empty, timeout, or say "not supported" on a protected URL. ## xurl article API (no browser) `xurl` can fetch an X Article body: request `tweet.fields=article` on `/2/tweets/<ID>` and read `data.article.plain_text` (+ `.title`). Watch out: `xurl` may print a non-JSON warning line before the JSON. Piping its stdout straight into `python3 -c "json.load(...)"` fails with `JSONDecodeError: Extra data (line 7)`. Redirect to a file first, then parse. If the API answers `402 ... credits-depleted`, this route is dead — fall through to the browser route. ## GitHub file URLs (PDFs, notebooks, binaries) A `github.com/<org>/<repo>/blob/main/<file>` URL returns the HTML viewer page, NOT the file — web_extract on it yields GitHub chrome ("Uh oh!", file tree, sign-in prompts). Rewrite to `raw.githubusercontent.com/<org>/<repo>/main/<file>` before extracting. Works for PDFs (web_extract parses them) and any raw text. ## Browserbase dual-control session `browser_navigate` creates ONE Browserbase session. That same session is controllable by the agent over CDP and simultaneously viewable/operable from the Browserbase dashboard → Sessions → Live View. `keepAlive` defaults true, so the session survives across turns. Because one cloud browser is both agent-driven and human-visible, a person can advance it past any step the agent cannot automate, then the agent resumes. For long articles the accessibility snapshot truncates — prefer a `browser_console` JS eval that returns the article's plain_text field; fallback `browser_snapshot(full=true)`. Browserbase is a paid backend; on a free plan the plugin auto-drops proxies/keepAlive on a 402 but the session still works. This is NOT computer-use.

Frequently asked questions