Site routing
Use this only after the tier index identifies an
external recovery procedure. web browse itself remains static public HTTP(S)
GET and never runs this table automatically.
| Target |
First explicit route |
Important limit |
| PDF, DOI, arXiv ID |
Tier 0 / academic pipeline |
Use a public or authorized OA copy; do not bypass a paywall. |
| Academic metadata or structured identifier |
Tier 1 / academic APIs |
Confirm current API policy and key requirements. |
| Static article, blog, docs |
Tier 1.5 / Trafilatura |
It cannot execute JavaScript or authenticate. |
| Lists, tables, metadata, Reddit/GitHub HTML |
Tier 2 / BeautifulSoup |
Prefer the platform's public API where documented. |
| JS-rendered/protected public page |
Tier 3 / Playwright |
Nature/Springer use domcontentloaded, not networkidle. |
| Forms, login, SPA, upload, human proof |
Agent-native browser |
Separate Chrome DevTools MCP procedure; never web browse. |
| News/RSS, social, finance, weather, facts |
The matching domain reference below |
Follow its public endpoint and rate/access rules. |
extract_page.py::auto_tier() is the source of truth for that retained helper's
selection. Its --fallback option is a standalone script feature and is not a
promise of automatic fallback in the public web action. For one documented
public HTTP retry, follow the operation contract's crawler User-Agent rule;
never impersonate a user, chain identities, solve CAPTCHA, or bypass login,
paywall, robots, or other access control.
Domain routes: news/RSS, social media,
real-time data, and academic pipeline.
Before executing a retained helper or trusting an asset, read its
known limitations.
1---2name: web-manual-routing-and-sites3description: Site-class and recovery routing for the external web procedures, including endpoint pointers and access limitations.4---5# Site routing67Use this only after the [tier index](../tier-quick-refs/SKILL.md) identifies an8external recovery procedure. `web browse` itself remains static public HTTP(S)9GET and never runs this table automatically.1011| Target | First explicit route | Important limit |12|---|---|---|13| PDF, DOI, arXiv ID | Tier 0 / [academic pipeline](../academic-pipeline.md) | Use a public or authorized OA copy; do not bypass a paywall. |14| Academic metadata or structured identifier | Tier 1 / [academic APIs](../tier-1-apis.md) | Confirm current API policy and key requirements. |15| Static article, blog, docs | Tier 1.5 / [Trafilatura](../tier-1-5-trafilatura.md) | It cannot execute JavaScript or authenticate. |16| Lists, tables, metadata, Reddit/GitHub HTML | Tier 2 / [BeautifulSoup](../tier-2-beautifulsoup.md) | Prefer the platform's public API where documented. |17| JS-rendered/protected public page | Tier 3 / [Playwright](../tier-3-playwright.md) | Nature/Springer use `domcontentloaded`, not `networkidle`. |18| Forms, login, SPA, upload, human proof | [Agent-native browser](../agent-native-browser.md) | Separate Chrome DevTools MCP procedure; never `web browse`. |19| News/RSS, social, finance, weather, facts | The matching domain reference below | Follow its public endpoint and rate/access rules. |2021`extract_page.py::auto_tier()` is the source of truth for that retained helper's22selection. Its `--fallback` option is a standalone script feature and is not a23promise of automatic fallback in the public `web` action. For one documented24public HTTP retry, follow the operation contract's crawler User-Agent rule;25never impersonate a user, chain identities, solve CAPTCHA, or bypass login,26paywall, robots, or other access control.2728Domain routes: [news/RSS](../news-and-rss.md), [social media](../social-media.md),29[real-time data](../realtime-data.md), and [academic pipeline](../academic-pipeline.md).3031Before executing a retained helper or trusting an asset, read its32[known limitations](../maintenance-bundles/SKILL.md#known-legacy-limitations).
lingtai-ai