DaveSteadman Gen2WebText
Gen2 derivative of DLoggedWebTextPlus, with centralized folder resolution via CommonCode/FolderNavigator.py.
Execution
python gen2_web_text.py --max-links --words --timeout-ms
Arguments
- domain required alphabetic string only A-Z or a-z
- url required start URL
- --max-links optional default 8 range 1 to 25
- --words optional default 200 range 80 to 400
- --timeout-ms optional default 15000
Output
On success returns JSON with:
- domain
- url
- start_page_summary
- start_page_words
- links_processed
- results with rank, anchor_text, url, page_summary, page_words, page_status
- log_path
- status
On error returns JSON with:
- domain
- url
- error
- results
- links_processed
- log_path
- status
Logging
Writes to: {OPENCLAW_ROOT}/data/datastore3/01-Mine//YYYY/MM/DD/{PageName}.txt
Path resolution is handled by FolderNavigator.from_fixed_point() and get_today_path(area="mine", domain=..., create=True).