dvcrn Fetch
Turn public URLs into usable local content.
Core Philosophy
- Fetch only public web content.
- Prefer clean extracted text over noisy raw HTML.
- Save both the raw response and structured extraction locally.
- Keep a simple local job history so previous fetches are easy to inspect.
Runtime Requirements
- Python 3 must be available as
python3 - No external packages required
Safety Boundaries
- Public URLs only
- No login flows
- No cookies or browser automation
- No API keys or credentials
- No external uploads or cloud sync
- All fetched data is stored locally only
Local Storage
All data is stored under:
~/.openclaw/workspace/memory/fetch/jobs.json~/.openclaw/workspace/memory/fetch/pages/
Key Workflows
- Fetch URL:
fetch_url.py --url "https://example.com" - Save cleaned output:
save_output.py --url "https://example.com" --title "Example" - List history:
list_jobs.py - Show job details:
show_job.py --id JOB-XXXX
Scripts
| Script | Purpose |
|---|---|
init_storage.py |
Initialize local storage |
fetch_url.py |
Fetch a public URL and extract content |
save_output.py |
Save cleaned output with a custom title |
list_jobs.py |
List previous fetch jobs |
show_job.py |
Show one saved fetch job |