Docker Images On Ssd

Docker images on SSD

Goal

Guarantee Docker image layers are written under /local-ssd, so the root filesystem (/) never fills up from image data.

The core rule (read this first)

Docker 29 has TWO possible image stores, and each keeps images in a DIFFERENT place. You must check which store is active, then verify the matching root.

Key facts learned on these machines:

• Docker 29 fresh installs default to the containerd store (overlayfs), even with NO "features" block in daemon.json. Do not add a containerd-snapshotter feature flag — it is not needed and not what the fresh machines use.
• An in-place upgrade that finds existing overlay2 data keeps using overlay2. That is why some machines are overlay2 and freshly-imaged ones are overlayfs.
• The classic trap: setting data-root in daemon.json looks correct, but on a containerd-store machine it does NOTHING for image location — images still go to containerd's root (default /var/lib/containerd, which is on /). This is the usual cause of / filling up.
• Verify the EFFECTIVE containerd root with containerd config dump, not just the file — drop-ins or defaults can differ from what config.toml appears to say.

Assumptions / policy (confirmed with the user)

• SSD mount is /local-ssd. Confirm it is a SEPARATE mount from / (via df). Targets: overlay2 → /local-ssd/docker; containerd → /local-ssd/containerd.
• Report first, then fix only after the user confirms each change. Never auto-edit or restart daemons unattended.
• Re-pull/rebuild is acceptable. No need to preserve or migrate existing image layers when switching stores or roots — losing image visibility is OK.

Workflow

1. Audit (read-only, always safe)

Run the bundled audit script as root:

sudo bash scripts/audit.sh

It reports: OS, docker version, storage driver, docker root dir, both config files, effective containerd root, directory existence, disk usage per candidate store path, and df for / vs /local-ssd. Makes zero changes.

2. Diagnose

From the audit output determine:

• Is /local-ssd a separate mount with free space? (If not, STOP — tell the user.)
• Which store is active (overlay2 vs overlayfs)?
• Is the matching root already on /local-ssd?
  • overlay2 → DockerRootDir should be /local-ssd/docker.
  • overlayfs → effective containerd root should be /local-ssd/containerd.
• Where does the disk usage actually sit? The path holding gigabytes is the real store. If that path is on / (e.g. /var/lib/containerd/...content), that is the bug.

If everything already points at /local-ssd, report "already correct" and stop.

3. Report

Present a short table: store type, current root, whether it is on SSD, and the exact change needed. Then ask the user to confirm before editing anything.

4. Fix (only after confirmation)

Apply the minimal change for the active store, then restart:

• overlay2: set "data-root": "/local-ssd/docker" in /etc/docker/daemon.json.
• overlayfs / containerd: set root = "/local-ssd/containerd" in /etc/containerd/config.toml (keep existing lines like disabled_plugins = ["cri"]). Optionally also set data-root in daemon.json for consistency, but note it does not govern image location in this mode.
• Create the target dir if missing: sudo mkdir -p <target>.
• Restart in order: sudo systemctl restart containerd && sudo systemctl restart docker.
• Because re-pull is acceptable, do not attempt to migrate old layers; the user can docker pull / rebuild as needed. Optionally note the old store path (e.g. /local-ssd/docker/overlay2 or /var/lib/...) that can be reclaimed.

5. Verify

Re-run sudo bash scripts/audit.sh and confirm:

• Active store's root is now on /local-ssd.
• New image data lands there (pull a small image, e.g. docker pull alpine, and confirm size grows under the SSD path, not on /).

Notes

• Restarting docker on a machine that switches store type will hide images stored in the previous store. That is expected and acceptable per policy (re-pull).
• Never run chmod -R on store directories; they are root-only by design. Use sudo for reads instead of loosening permissions.