Hf Resume

hf-resume

hf download cannot resume across runs. Since ~`huggingface_hub1.17 each attempt writes to..incompleteand deletes it in afinally` — the library's own comment says the partial "could not be reused anyway since the temporary name is unique to this download". Kill a 17 GB download at 90% and you start again at byte 0.

This skill downloads through a pinned huggingface_hub==1.16.4 (run by uv), the last line that opens a deterministic <etag>.incomplete and passes resume_size to http_get so a real Range request continues the file. A native Swift window shows progress.

Use it

S=~/.claude/skills/hf-resume/scripts/hf-resume     # or wherever this skill is installed

$S pull mlx-community/flux2-klein-9b-8bit          # detached + window; source auto-picked
$S pull Qwen/Qwen3-8B --include '*.safetensors' --workers 4
$S pull HuggingFaceFW/fineweb --type dataset --local-dir ./fineweb
$S pull <repo> --modelscope   # force the CN mirror  (--hf forces direct)
$S status            # text progress (any time, any shell)
$S window            # re-open the window
$S stop              # stop; partials stay resumable
$S list              # every run
$S verify <repo>     # checksum-verify what landed
$S doctor [repo]     # environment check + which source it would pick, and why

Re-running the same pull is how you resume. It is idempotent: finished files are skipped, half-finished ones continue from their byte offset, and the window reports resumed from N MB already on disk.

Downloads land in the HF cache by default, so mflux / diffusers / mlx resolve them by repo id with no path juggling. --local-dir gives a flat copy instead.

The daemon is detached: it keeps running after the command returns, after the window is closed, and after this session ends. status and window reattach to it.

Sources: direct or ModelScope, and why the choice is measured

--source auto (default) probes direct HuggingFace and uses it whenever it is reachable at ≥2 MB/s. Only when HF is unreachable or crawling does it probe ModelScope and switch. Override with --hf / --modelscope, or hf-resume doctor <repo> to see the decision.

Do not try to detect the proxy instead. Clash/mihomo in TUN mode routes at the network layer: scutil --proxy reports HTTPEnable: 0 and no proxy env vars exist, while huggingface.co is in fact fast. Config-based detection reports "no proxy" exactly when direct is the right answer. Proxy signals are logged as explanation; the measurement decides.

Short probes are noisy in both directions — TCP slow start undershoots (a 3 MB read measured 1.2 MB/s on a link sustaining 8 MB/s), a warm CDN edge overshoots. So the probe skips its first 1.5 MB and its verdict is only ever "usable vs not", never a close race.

Either source fills the same cache. For LFS files the HF cache names each blob by its sha256, and ModelScope publishes the same sha256 for the same repo (verified byte-identical, and non-LFS files get their git-blob-sha1 computed locally). So ModelScope bytes are valid HF blobs: after a ModelScope pull, an --hf pull of the same repo finishes in seconds and re-downloads nothing — measured, with the blob's mtime unchanged. Switch freely.

Rules — each one is a bug someone already shipped

1. Never aria2c -x8 / any multi-connection downloader on Hugging Face. A signed CDN URL pins one byte range in its CloudFront policy; reusing that URL for a different range returns 403. With preallocation the file ends up full-size with holes — silent corruption that only a checksum catches. Keep every file single-stream. Concurrency comes from --workers (several files at once), never from splitting one file.
2. Xet must stay disabled (HF_HUB_DISABLE_XET=1, set by the downloader). Even in 1.16.4, resume_size is passed to http_get but never to xet_get, while the incomplete file is opened "ab" — so on a Xet repo a leftover partial can get a full reconstruction appended onto it. Xet is faster when it works, but it cannot resume here.
3. hf_transfer cannot resume at all. HF_HUB_ENABLE_HF_TRANSFER is stripped.
4. Mirrors that redirect are worse than nothing. hf-mirror.com now 308-redirects to huggingface.co (zero acceleration) and breaks modern hf metadata resolution outright (Local entry not found. Distant resource does not seem to be on huggingface.co). So an HF_ENDPOINT that does not point at huggingface.co is ignored and reported, never silently dropped. A genuinely working mirror is still worth having: pass --endpoint URL (wins unconditionally) or set HF_RESUME_KEEP_ENDPOINT=1 to keep the ambient one. If doctor finds a stale mirror in ~/.zshrc, offer to delete that line.
5. Progress comes from the filesystem, not tqdm. tqdm's desc is the CDN basename truncated to 40 chars, and one repo can hold several files named 0.safetensors. The downloader maps each file to its {etag}.incomplete instead — exact for both cache and --local-dir.
6. --force-download throws away resumable bytes. Only reach for it when a checksum fails.

Shape

scripts/hf-resume      bash dispatch; detaches the daemon, compiles+launches the window
scripts/downloader.py  uv PEP-723 script pinning huggingface_hub==1.16.4; publishes state.json
scripts/Window.swift   SwiftUI reader of state.json (compiled on first use, cached by hash)

Runtime state lives in ~/.cache/hf-resume/runs/<slug>/ (state.json, downloader.log, pid). state.json is written atomically (temp + rename), so the window never reads a torn file.

The window is a pure reader and owns no download — closing it stops nothing.

Two things about it are load-bearing and were found the hard way:

• It sets NSApp.setActivationPolicy(.regular); a non-bundled binary is .prohibited by default and would never show a window.
• It creates its NSWindow explicitly and hosts the SwiftUI tree in an NSHostingView, rather than using a SwiftUI App/Window scene. In a bare binary a Scene works while the App holds only value @State, but the moment the App owns a model object (@StateObject on an ObservableObject, or @State on an @Observable class, with or without @MainActor) the process launches, idles in a healthy event loop, and creates zero windows — no crash, no log. Because it is top-level code it must be compiled without -parse-as-library.

Troubleshooting