Infrastructure Runbook
Quick Diagnosis
# What's running?
pm2 jlist | jq -r '.[] | "\(.name)\t\(.pm2_env.status)\t\(.pm2_env.restart_time)"'
# What's listening?
lsof -iTCP -sTCP:LISTEN -nP | grep -E ":(18789|3110|5678|3100|11434|1234)" | awk '{print $1, $9}'
# Disk pressure?
df -h /System/Volumes/Data | tail -1
Service-Specific Runbooks
OpenClaw Gateway (:18789)
Symptom: Agents not responding, messages not flowing
# Check status
curl -s http://localhost:18789/health # NOTE: /health not /api/health
# Check logs
openclaw gateway status
# Restart (REQUIRES KEVIN'S APPROVAL)
openclaw gateway restart
Common causes: Config error after patch, OOM, node crash
Paperclip (:3110)
Symptom: Tickets not creating, API 500s
pm2 show paperclip
pm2 logs paperclip --lines 30 --nostream
# Restart
pm2 restart paperclip
Common causes: SQLite lock, port conflict (OrbStack), migration needed
BlueBubbles (:1234)
Symptom: iMessage not working, 401 errors (normal — needs auth), no response at all (bad)
# Check if alive
curl -s http://localhost:1234/api/v1/server/info -H "Authorization: Bearer $BB_PASSWORD"
# Force restart
killall -9 BlueBubbles; sleep 2; open -a BlueBubbles
Common causes: macOS killed it, port stuck after crash
n8n (:5678)
Symptom: Workflows not triggering
pm2 show n8n 2>/dev/null || docker ps | grep n8n
pm2 logs n8n --lines 20 --nostream
Langfuse (:3100)
Symptom: Observability blind
curl -s http://localhost:3100/api/health
docker ps | grep langfuse
docker logs langfuse-web --tail 20
Caddy (Reverse Proxy)
Symptom: HTTPS not working, domains not resolving
# Check config
cat ~/your-infra/caddy/Caddyfile | grep -E "^[a-z].*\{" | head -20
# Reload
caddy reload --config ~/your-infra/caddy/Caddyfile
# Check TLS certs
caddy validate --config ~/your-infra/caddy/Caddyfile
Common causes: Cloudflare DNS token expired, config syntax error
Ollama (:11434)
Symptom: Local models not responding
curl -s http://localhost:11434/api/tags | jq '.models[].name'
ollama list
# Restart
killall ollama; sleep 1; ollama serve &
Escalation
If you can't fix it:
- Write diagnostic output to
~/reports/architecture/incident-YYYYMMDD.md - Post to #morpheus with findings
- Tag Kevin if it's blocking work