Docker Troubleshooting

Docker Troubleshooting — 问题排查与调试

Systematic approach to diagnosing and resolving common Docker issues.

When to Use

ALWAYS use this skill when the user mentions:

• "docker 排查", "docker debug", "容器问题"
• "容器启动失败", "exit code", "crash"
• "OOM", "磁盘满", "disk full"
• "网络不通", "network issues"
• "docker daemon", "daemon not running"

Diagnostic Framework

1. Check container state    → docker ps -a, docker inspect
2. Check logs               → docker logs --tail 100
3. Check resources          → docker stats, docker system df
4. Check exit code          → docker inspect --format='{{.State.ExitCode}}'
5. Interactive debug        → docker run --rm -it image sh

Container Startup Failure

Exit Code Reference

# Quick diagnosis
docker ps -a | grep Exited
docker inspect --format='{{.State.ExitCode}} {{.State.Error}}' container
docker logs container

Disk Space Exhaustion

# Check disk usage
docker system df

# What's using space?
docker system df -v   # Verbose breakdown

# Clean up
docker system prune -a     # Remove all unused: images, containers, volumes, networks
docker image prune -a      # Unused images only
docker volume prune        # Unused volumes only

# Log rotation (prevent future issues)
docker run --log-opt max-size=10m --log-opt max-file=3 myapp

OOM (Out of Memory)

# Check if container was OOM-killed
docker inspect --format='{{.State.OOMKilled}}' container

# Check memory usage
docker stats --no-stream

# Solution: increase memory limit
docker update --memory=1g --memory-swap=1g container

Network Connectivity

# Check network configuration
docker network inspect bridge
docker exec container ip addr show
docker exec container ping other-container

# DNS check
docker exec container nslookup other-container
docker exec container cat /etc/resolv.conf

# Host networking check
iptables -t nat -L DOCKER -n   # Port forwarding rules

Permission Issues

# Bind mount permission denied
# Solution 1: Match UID
docker run -u $(id -u):$(id -g) -v $PWD:/app myapp

# Solution 2: Fix permissions in entrypoint
# In entrypoint script: chown -R appuser:appgroup /data

# Docker socket permission
sudo usermod -aG docker $USER   # Add user to docker group
newgrp docker                    # Refresh group (or re-login)

Workflow — 排查流程

Step 1: 查看状态: docker ps -a → 确认退出码和 OOMKilled Step 2: 查看日志: docker logs --tail 100 <container> 获取错误信息 Step 3: 深入检查: docker inspect → 检查网络/挂载/资源限制 Step 4: 交互调试: docker run -it --rm --entrypoint sh <image> 进入排查 Step 5: 资源排查: docker stats + docker system df 检查 CPU/内存/磁盘

Gotchas — Common Pitfalls

• docker system prune is destructive: Removes ALL unused objects. Don't run blindly in production. → Recovery: Use docker system df first to preview; add --filter "until=24h" for safe cleanup; never run prune -a without confirmation.
• Logs fill disk silently: JSON-file driver has NO rotation by default. A noisy app can fill the disk in hours. → Recovery: docker run --log-opt max-size=10m --log-opt max-file=3 app; existing: truncate -s 0 $(docker inspect -f '{{.LogPath}}' myapp).
• OOMKilled=TRUE but no error in logs: SIGKILL can't be logged. → Recovery: Check docker inspect -f '{{.State.OOMKilled}}' myapp; increase memory: docker update --memory 512m myapp.
• Exited (0) but service not working: Application started but crashed after health check passed. → Recovery: docker logs --tail 100 myapp; check if app listens on 0.0.0.0 (not 127.0.0.1); verify with docker exec myapp netstat -tlnp.

Boundary — 能力边界（适用与不适用场景）

When NOT to Use This Skill

Security & Stability

• docker exec into production containers can expose sensitive data. Prefer log-based debugging.
• docker system prune -a removes ALL unused images and volumes — use with caution in production.
• Inspecting containers with --privileged debug tools can bypass security controls.
• Never expose Docker socket (/var/run/docker.sock) to containers for debugging — it grants root access.
• Rotate debug logs — they may contain sensitive information. Use log drivers with TTL.

📚 官方文档参考

🧭 Docker Skills Journey

📍 You are here: docker-troubleshooting — 问题排查

← Previous: docker-production