agentskillexchange Download every archived snapshot of a URL before site migrations, takedowns, or investigations
Use waybackpack when an agent needs the full historical record for a URL, not a few clicks through the Wayback Machine UI. The agent can list or download snapshots, constrain by date, deduplicate archives, and preserve evidence locally before a site changes or disappears.
Prerequisites
Python 3, pip, a target URL, local disk space for archived snapshots, and optional tqdm for progress output.
Installation
Use the upstream install or setup path that matches your environment:
- pip install waybackpack
Requirements and caveats from upstream:
- [](https://pypi.python.org/pypi/w...
- Requires tqdm to be installed.
- Waypackback is written in pure Python, depends only on requests, and should work wherever Python works. Requires Python 3.3+.
Basic usage or getting-started notes:
For instance, to download every copy of the Department of Labor's homepage through 1996 (which happens to be the first year the site was archived), you'd run:
[--from-date FROM_DATE] [--to-date TO_DATE]
[--user-agent USER_AGENT] [--follow-redirects]
Extracted from upstream docs: https://raw.githubusercontent.com/jsvine/waybackpack/HEAD/README.md