Coolify Troubleshoot

Coolify Troubleshooting

Diagnostic flows for HTTP errors, container issues, build failures, SSL problems, and Coolify self-repair for v4 self-hosted.

Self-Hosted Only: All diagnostic commands assume self-hosted Coolify v4.x with SSH access to the server. Coolify Cloud users have limited access to container-level debugging.

Master Diagnostic Decision Tree

START: What's the symptom?
│
├─ HTTP error in browser?
│  ├─ 502 Bad Gateway → See "502 Diagnostic Flow"
│  ├─ 504 Gateway Timeout → See "504 Diagnostic Flow"
│  ├─ SSL certificate error → See "SSL Diagnostic Flow"
│  ├─ "404 page not found" → Domain not configured or Traefik misconfigured
│  └─ Traefik "no server" error → Container not running or wrong network
│
├─ Deployment failed?
│  ├─ Build failed → See "Build Failure Diagnosis"
│  └─ Build succeeded but container won't start → See "Container Lifecycle"
│
├─ Container issues?
│  ├─ Restart loop → See "Restart Loop Diagnosis"
│  ├─ OOM killed → See "OOM Diagnosis"
│  └─ Container exited → Check exit code (see table below)
│
├─ Coolify itself is broken?
│  └─ See "Coolify Self-Repair"
│
└─ None of the above?
   └─ Check Coolify server resources (disk, RAM, CPU)
      docker system df
      df -h
      free -m

502 Bad Gateway — Diagnostic Flow

A 502 means Traefik received the request but could not reach the backend container.

502 Bad Gateway
│
├─ 1. Is the container running?
│     docker ps | grep <APP_NAME>
│     ├─ Not running → Check deployment logs in Coolify UI
│     │  └─ Container crashed? Check: docker logs <CONTAINER>
│     └─ Running → Continue
│
├─ 2. Is the container healthy?
│     docker inspect <CONTAINER> | grep -A5 Health
│     ├─ unhealthy → App is running but health check fails
│     │  └─ Check health check path returns 200
│     └─ healthy or no health check → Continue
│
├─ 3. Is the app listening on the right port?
│     docker exec <CONTAINER> curl -s localhost:<PORT>/
│     ├─ Connection refused → App not binding to configured port
│     │  └─ Check PORT env var matches app's listen port
│     └─ Returns response → Container is fine, issue is Traefik → Continue
│
├─ 4. Are Traefik labels correct?
│     docker inspect <CONTAINER> | grep traefik
│     ├─ Missing or wrong labels → Coolify misconfigured routing
│     │  └─ Redeploy the application from Coolify UI
│     └─ Labels look correct → Continue
│
└─ 5. Is Traefik itself healthy?
      docker logs coolify-proxy --tail 50
      └─ Check for error messages about the backend

Common 502 causes (ranked by frequency):

1. App crashes on startup — check docker logs <CONTAINER>
2. Wrong port — app listens on 8080 but Coolify expects 3000
3. App takes too long to start — Traefik tries to route before app is ready
4. Out of memory — container killed before it can serve requests

504 Gateway Timeout — Diagnostic Flow

A 504 means Traefik reached the container, but the response took too long.

504 Gateway Timeout
│
├─ Is it a specific endpoint or all requests?
│  ├─ Specific endpoint → Application-level timeout (slow query, external API)
│  │  └─ Profile the endpoint; check database queries, external calls
│  └─ All requests → Container overloaded or unresponsive
│
├─ Check container resource usage:
│  docker stats <CONTAINER> --no-stream
│  ├─ CPU near 100% → App is CPU-bound; scale up or optimize
│  └─ Memory near limit → OOM pressure; increase memory limit
│
└─ Check Traefik timeout configuration:
   Default timeout is 30 seconds
   For long-running requests, increase via Traefik labels or Coolify settings

Container Lifecycle Issues

Exit Code Reference

Restart Loop Diagnosis

# Check restart count
docker inspect <CONTAINER> --format '{{.RestartCount}}'

# Check last exit code
docker inspect <CONTAINER> --format '{{.State.ExitCode}}'

# Check last N log lines (may show crash reason)
docker logs <CONTAINER> --tail 50

# Check if OOM killed
docker inspect <CONTAINER> --format '{{.State.OOMKilled}}'

Resolution by exit code:

• Exit 1: Read the logs — usually a config error or missing env var
• Exit 137: Increase memory limit or optimize the app
• Exit 127: Fix the start command in Coolify settings
• Exit 126: Fix file permissions in Dockerfile

OOM Kill Confirmation

# Method 1: Docker inspect
docker inspect <CONTAINER> --format '{{.State.OOMKilled}}'
# Returns "true" if OOM killed

# Method 2: System logs
dmesg | grep -i "oom\|killed" | tail -10

# Method 3: Docker events
docker events --filter 'event=oom' --since '1h'

Fix: Increase the memory limit in Coolify application settings, or optimize the app to use less memory.

Build Failure Diagnosis

Nixpacks Build Failures

Reading Build Logs

Build logs in Coolify contain both Coolify orchestration output and the actual build output. Focus on:

1. Skip the initial "Pulling repository..." and "Starting build..." lines
2. Look for the first ERROR, error, FAILED, or non-zero exit code
3. The actual error is usually 5-10 lines BEFORE "Build failed" at the end
4. Ignore Coolify's wrapper messages; focus on the build tool's output (npm, pip, cargo, etc.)

Force Clean Build

# Via Coolify API
curl -X POST "https://<COOLIFY>/api/v1/applications/<UUID>/deploy" \
  -H "Authorization: Bearer <TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"force": true}'

# Or from Coolify UI: Deploy → "Force Rebuild" toggle

Build cache invalidation: Coolify caches Docker layers between builds. A force rebuild ignores the cache entirely.

SSL Certificate Issues

Diagnostic Flow

SSL Error in Browser
│
├─ 1. Is DNS pointing to the Coolify server?
│     dig <DOMAIN> A    (should show server IP)
│     ├─ Wrong IP → Fix DNS records
│     └─ Correct → Continue
│
├─ 2. Is port 80 open? (Required for HTTP-01 challenge)
│     curl http://<DOMAIN>/.well-known/acme-challenge/test
│     ├─ Connection refused → Firewall blocking port 80
│     │  └─ ufw allow 80/tcp
│     └─ Returns something → Port is open → Continue
│
├─ 3. Check Traefik ACME logs:
│     docker logs coolify-proxy 2>&1 | grep -i "acme\|certificate\|letsencrypt"
│     ├─ Rate limit → Too many cert requests; wait 1 hour
│     ├─ DNS challenge failed → DNS not propagated yet
│     └─ Challenge succeeded but cert not applied → Restart Traefik
│
└─ 4. Force certificate renewal:
      # Restart Traefik to re-trigger ACME
      docker restart coolify-proxy
      # Or delete the ACME storage and restart
      # WARNING: This renews ALL certificates

Let's Encrypt Challenge Types

Custom Certificates

For non-Let's Encrypt certificates:

1. Upload cert and key files via Coolify UI (Application → SSL settings)
2. Or mount them as volumes and configure Traefik labels manually

Coolify Self-Repair

Restart Coolify Without Affecting Apps

# SSH into the server, then:
cd /data/coolify/source
docker compose restart

# Or restart specific Coolify services:
docker restart coolify          # Main Coolify app
docker restart coolify-proxy    # Traefik reverse proxy
docker restart coolify-realtime # Websocket/realtime service

Running app containers are NOT affected by restarting Coolify. They continue serving traffic independently.

Coolify Self-Update

# Recommended: Use the UI → Settings → Update
# Or from CLI:
curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash

What can go wrong:

• Database migration failure → Coolify won't start → check docker logs coolify
• Port conflict → another service grabbed Coolify's port
• Docker version incompatibility → rare, but check Docker version requirements

"Coolify Is Unreachable" Diagnosis

# 1. Check if Coolify containers are running
docker ps | grep coolify

# 2. Check Coolify logs
docker logs coolify --tail 100

# 3. Check if port 8000 is listening
ss -tlnp | grep 8000

# 4. Check disk space (Coolify fails silently when disk is full)
df -h

# 5. Check if Docker is running
systemctl status docker

# 6. Check server resources
free -m
top -bn1 | head -5

Anti-Patterns

Related Skills

• coolify-deploy — Deployment configuration, build pack selection
• coolify-security — Resource limits, OOM prevention
• coolify-observability — Monitoring, log drains, alerting
• coolify-databases — Database connection issues, OOM

Additional Resources

Reference Files

• references/traefik-debugging.md — Traefik v3 routing diagnosis, label inspection, dynamic config
• references/common-errors.md — Expanded error pattern table with fixes for 50+ common issues