Health Check

Service Health Checker

Check the health and status of all services, databases, and dependencies.

Arguments

• $ARGUMENTS: Specific service name (optional - checks all if not specified)

Services Configuration

Health Check Commands

HTTP Services

# Basic health check
curl -sf http://localhost:$PORT/health && echo "OK" || echo "FAILED"

# With timeout
curl -sf --max-time 5 http://localhost:$PORT/health

# Get response details
curl -sf -w "\nStatus: %{http_code}\nTime: %{time_total}s\n" http://localhost:$PORT/health

# Check multiple endpoints
for port in 8000 8002 3000; do
    echo "Checking port $port..."
    curl -sf --max-time 5 http://localhost:$port/health && echo " OK" || echo " FAILED"
done

Database Services

MongoDB:

# Check connection
mongosh --eval "db.adminCommand('ping')" --quiet
# or
mongosh mongodb://localhost:27017/test --eval "db.runCommand({ping: 1})"

PostgreSQL:

# Check connection
pg_isready -h localhost -p 5432
# or with psql
psql -h localhost -U postgres -c "SELECT 1" > /dev/null 2>&1 && echo "OK" || echo "FAILED"

Redis:

redis-cli ping
# Should return "PONG"

Process Status

Systemd Services:

systemctl is-active erudition-service
systemctl status erudition-service --no-pager

Docker Containers:

docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
docker inspect --format='{{.State.Health.Status}}' $CONTAINER

PM2 Processes:

pm2 status
pm2 show $APP_NAME

Port Checks

# Check if port is listening
lsof -i :$PORT
# or
netstat -tlnp | grep $PORT
# or on macOS
lsof -nP -iTCP:$PORT | grep LISTEN

Comprehensive Health Check

Run all checks at once:

echo "=== Service Health Check ==="
echo ""

# HTTP Services
echo "HTTP Services:"
for service in "8000:eruditiontx-api" "8002:mathmatterstx-api" "3000:frontend"; do
    port=$(echo $service | cut -d: -f1)
    name=$(echo $service | cut -d: -f2)
    status=$(curl -sf --max-time 3 http://localhost:$port/health && echo "UP" || echo "DOWN")
    printf "  %-20s %s\n" "$name" "$status"
done

echo ""
echo "Databases:"
# MongoDB
mongo_status=$(mongosh --eval "db.adminCommand('ping')" --quiet 2>/dev/null && echo "UP" || echo "DOWN")
printf "  %-20s %s\n" "MongoDB" "$mongo_status"

# PostgreSQL
pg_status=$(pg_isready -h localhost -p 5432 2>/dev/null && echo "UP" || echo "DOWN")
printf "  %-20s %s\n" "PostgreSQL" "$pg_status"

# Redis
redis_status=$(redis-cli ping 2>/dev/null | grep -q PONG && echo "UP" || echo "DOWN")
printf "  %-20s %s\n" "Redis" "$redis_status"

echo ""
echo "Docker Containers:"
docker ps --format "  {{.Names}}: {{.Status}}" 2>/dev/null || echo "  Docker not running"

Output Format

Health Check Report
Generated: [timestamp]

Services:
  Service                 Status    Response Time
  ---------------------   ------    -------------
  eruditiontx-api         UP        45ms
  mathmatterstx-api       UP        32ms
  frontend                UP        28ms

Databases:
  Database                Status    Connection
  ---------------------   ------    ----------
  MongoDB                 UP        localhost:27017
  PostgreSQL              DOWN      Connection refused
  Redis                   UP        localhost:6379

Infrastructure:
  Component               Status
  ---------------------   ------
  Docker                  Running (5 containers)
  Nginx                   Running

Issues Found:
  - PostgreSQL is not running
  - Port 5432 not listening

Recommendations:
  1. Start PostgreSQL: brew services start postgresql
  2. Check PostgreSQL logs: tail -f /usr/local/var/log/postgresql.log

Alerting

If critical services are down:

# Send alert (example with curl to Slack webhook)
if ! curl -sf http://localhost:8000/health > /dev/null; then
    curl -X POST -H 'Content-type: application/json' \
        --data '{"text":"ALERT: eruditiontx-api is DOWN"}' \
        $SLACK_WEBHOOK_URL
fi

Troubleshooting

If a service is down:

1. Check if process is running: ps aux | grep $SERVICE
2. Check port binding: lsof -i :$PORT
3. Check logs: /logs $SERVICE error
4. Restart service: sudo systemctl restart $SERVICE