web-research
Web search, page scraping, and text summarization using only the Python standard library. No API keys, no third-party packages.
Run CLIs from this skill's directory (the folder that contains this SKILL.md):
uv run scripts/<name>.py or python3 scripts/<name>.py. Do not prefix
skills/web-research/ — that path is only valid from this monorepo's root.
Available Scripts
Search the web
uv run scripts/search.py --query "climate policy 2024" --limit 10
Searches via the DuckDuckGo HTML endpoint. Outputs JSON:
{"query": ..., "count": N, "results": [{"title", "url", "snippet"}]}.
Exit 2 if no results.
Scrape a page
uv run scripts/scrape.py --url "https://example.com/article" --max-chars 5000
Fetches and extracts the readable title, meta description, and text. Outputs
JSON: {"url", "title", "description", "text"}. Exit 2 if no text extracted.
Summarize text
uv run scripts/scrape.py --url "https://example.com" | \
uv run scripts/summarize.py --sentences 5
Reads plain text OR JSON with a text field (e.g. scrape.py output) on
stdin. Produces an extractive, frequency-ranked summary. Outputs JSON:
{"sentences": N, "summary": [...]}. Works offline once text is on stdin.
Pipeline
Chain the scripts: search.py → pick a URL → scrape.py → summarize.py.
Output & Exit Codes
- All scripts emit JSON to stdout; diagnostics to stderr.
- Exit
0success,1error,2not found.
Notes
- Search uses DuckDuckGo's HTML endpoint; heavy use may be rate-limited.
- Summarization is extractive (no LLM), so it needs no API key and runs offline.