# Image Pull Debug > Diagnose container image pull failures (ErrImagePull / ImagePullBackOff). Checks pod status, containerd logs, and events to identify root cause. - Skill: `scitix/image-pull-debug` (Agent Skill) - Install (CLI): `npx skillmds@latest add scitix/image-pull-debug` - Raw SKILL.md: https://api.skillmd.com/api/skills/scitix/image-pull-debug/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: scitix (https://skillmd.com/u/scitix) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/scitix/image-pull-debug --- # Image Pull Failure Diagnosis When a pod is stuck in `ErrImagePull` or `ImagePullBackOff`, follow this flow to identify the root cause. **Important:** `ErrImagePull`, `ImagePullBackOff`, and `Back-off pulling image` are NOT causes — they only indicate the pull failed. You MUST proceed through all steps below to find the actual cause. Never conclude with just these status messages. **Scope:** This skill is for **diagnosis only**. Once you identify the root cause, report it to the user and stop. Do NOT attempt network-level debugging (ping, curl, iptables, traceroute, etc.) — that is outside the scope of this skill and should be left to the user or network administrator. ## Diagnostic Flow ### 1. Get pod info — one call ``` k8s_inspect(kind: "pod", name: "", namespace: "") ``` All three things this step needs come back together: the **waiting reason and message** (which may already be the root cause), the **image** including its registry prefix, and the **node name** in the `node` section — which is the argument step 2 needs, and the reason this used to cost three separate reads before the node was even known. Also check the image registry: - If the image has **no registry prefix** (e.g. `nginx:latest`, `envoyproxy/gateway:v1.2.8`), it pulls from **Docker Hub** (`docker.io`). - If it has a prefix (e.g. `registry.example.com/app:v1`), it pulls from that registry. ### 2. Check containerd logs Containerd logs are the authoritative source for the root cause. Pod events are often generic ("Failed to pull image") and do not contain the actual error — always check containerd logs. Use the `node-logs` skill — via `host_script` when the node is a bound SSH host (check `host_list`), otherwise `node_script`: ``` node_script: node="", skill="node-logs", script="get-node-logs.sh", args='--unit containerd --grep-fixed "" --since "1h ago"' ``` Use `--grep-fixed` for the image reference: a registry path is a literal, and its `.`, `-` and `/` would otherwise be read as regex. Adjust `--since` to cover the pod's creation time. If the journal has nothing, try the log files — note the distro difference (`/var/log/messages` on RHEL/CentOS, `/var/log/syslog` on Ubuntu): ``` node_script: node="", skill="node-logs", script="get-node-logs.sh", args='--file /var/log/messages --include-rotated --grep-fixed ""' ``` Read the `status:` line at the end of the output before concluding anything. Only `no_match` means "read successfully, nothing matched"; `source_error`, `not_found` and `filter_error` say nothing at all about the image pull. See the `node-logs` SKILL.md for the full table, and for the tier-3 fallback when neither SSH nor a debug pod is available on that node. ### 3. Match error and conclude Match the error from containerd logs (or the `state.waiting.message` from step 1) against the patterns below. Once a pattern matches, **report the root cause to the user and stop**. Do not continue with further diagnostic commands. If containerd logs have no relevant entries, check events as a supplementary source: ```bash kubectl get events -n --field-selector involvedObject.name= ``` If still no match, report whatever error information you have found and let the user decide next steps. Do NOT start autonomous network investigation. --- #### `not found` / `manifest unknown` — Image does not exist The image name or tag does not exist in the registry. Inform the user to verify the image name and tag. --- #### `unauthorized` / `access denied` / `denied` — Authentication failed The registry rejected the request. Advise the user to: 1. Configure an imagePullSecret for the pod with valid registry credentials 2. Or adjust image permissions on the registry side to allow access --- #### `x509` / `certificate` / `tls` — Certificate not trusted The node's containerd does not trust the registry's CA certificate. Advise the user to add the registry CA to the node's containerd trust config (`/etc/containerd/certs.d/`) or system trust store. --- #### `no such host` / `lookup.*failed` — DNS resolution failed The registry hostname cannot be resolved. Advise the user to check the hostname spelling and node DNS config. --- #### `connection reset by peer` — Remote reset TCP reached the registry but was reset by the remote end — a server-side or intermediary issue. --- #### `i/o timeout` / `dial tcp.*timeout` — Connection timed out The node cannot establish a TCP connection to the registry. Common causes: firewall blocking, proxy misconfiguration, or registry unreachable from the node's network. **Docker Hub specific:** If the image is from Docker Hub (`docker.io`) and the node is in mainland China, this is almost certainly caused by network restrictions (GFW). Advise the user to use a registry mirror or re-tag the image to a domestically accessible registry. Report the timeout to the user and stop. Do NOT attempt network diagnostics (ping, curl, iptables, etc.). --- #### `connection refused` — Connection refused TCP reached the host but the port is not listening. The registry service is down or on a different port. --- #### `too many requests` / `429` / `rate limit` — Rate limited The registry is throttling requests. Inform the user to wait for the rate limit window to expire, or configure a registry mirror to reduce direct requests. --- #### `no space left on device` — Disk full Containerd cannot unpack image layers due to insufficient disk space on the node. --- #### `invalid reference format` — Malformed image name The image reference contains illegal characters or has incorrect format. Inform the user to fix the image field in the pod spec. --- #### `server gave HTTP response to HTTPS client` — Protocol mismatch The registry serves HTTP but containerd expects HTTPS. Advise the user to configure the registry as insecure in containerd config, or enable TLS on the registry. --- #### `does not match the specified platform` — Architecture mismatch The image exists but has no manifest for this node's CPU architecture. Inform the user to use a multi-arch image or the correct platform-specific tag. --- #### `ErrImageNeverPull` — Pull policy forbids pulling `imagePullPolicy` is `Never` but the image is not present on the node. Inform the user to pre-load the image or change `imagePullPolicy`. ## Notes - The `--since` parameter should cover the pod's creation time. If the pod was created long ago, increase accordingly (e.g. `--since "24h ago"`). - If both containerd logs and events are empty, the `state.waiting.message` from step 1 is your best available information — report it directly.