Skastvnt AI Assistant Service Health Check Audit

name: service-health-check-audit description: "Debug startup failures, port drift, and service health for the core chatbot stack. Use when: a service fails to start, a health check returns unexpected results, ports or entry points are mismatched across scripts and docs, dependency profiles need verification, Docker or CI startup assumptions need checking, or runtime behavior no longer matches documentation."

Service Health Check Audit

When to use this skill

• A chatbot or MCP service fails to start or returns errors on health endpoints.
• Port numbers, entry points, or startup commands appear inconsistent across files.
• A change to startup behavior needs verification against scripts, Docker, and CI.
• You suspect a dependency profile mismatch (venv-core vs venv-image).
• You need to reconcile docs with actual runtime behavior.
• A health check script reports a service down when it should be running (or vice versa).

Do not use this skill for ComfyUI, Stable Diffusion, or image pipeline issues unless the failing path actually reaches those services.

Canonical service facts

Known stale references (do not trust):

Authoritative source for ports: README.md service table.

Chatbot startup modes

Note: run.py is the canonical dispatcher and starts the Flask monolith only.

Health check endpoints (chatbot)

Docker healthcheck uses: curl -f http://localhost:5000/health

Startup debug sequence

When a service fails to start, follow this sequence exactly. Do not skip steps.

Step 1 — Verify the environment

Check                                   How
─────────────────────────────────────── ────────────────────────────────
Python version                          python --version (expect 3.10+)
Active venv                             which python / where python
Correct venv for service                venv-core for chatbot/MCP, venv-image for SD/edit
Required packages installed             pip list | grep flask (or mcp, etc.)
Shared env file exists                  ls app/config/.env or .env_dev
Startup env vars set                    echo $FLASK_PORT, echo $CHATBOT_PORT

Step 2 — Verify the entry point

Check                                   How
─────────────────────────────────────── ────────────────────────────────
Entry file exists                       ls services/chatbot/run.py
Entry file is syntactically valid       python -m py_compile services/chatbot/run.py
Shared env import works                 python -c "from services.shared_env import load_shared_env"
Core config imports cleanly             cd services/chatbot && python -c "from core.config import *"

Step 3 — Attempt a controlled start

# Local (Flask monolith)
cd services/chatbot && python chatbot_main.py

# Docker
docker-compose -f app/config/docker-compose.yml up chatbot

Watch for:

• ImportError / ModuleNotFoundError → wrong venv or missing package.
• Address already in use → port 5000 already occupied.
• KeyError or missing env var → shared env not loaded or variable missing from .env.
• ConnectionRefusedError to MongoDB/Redis → database not running.

Step 4 — Verify health

# HTTP health check
curl http://localhost:5000/health

# Database health
curl http://localhost:5000/api/db-health

# Expected healthy response
{"status": "healthy", "service": "chatbot"}

Step 5 — Report findings

Use the output format below.

MCP server health check

The MCP server uses stdio, not HTTP. It has no port and no /health endpoint.

Verify MCP

# Check the module imports cleanly
cd services/mcp-server && python -c "from server import mcp; print('OK')"

# Check FastMCP is installed
pip show mcp

# Run interactively (stdio — will wait for input)
cd services/mcp-server && python server.py

If scripts or docs reference MCP on port 8000, that is stale. Do not add HTTP listeners.

Docker health check sequence

# Start stack
docker-compose -f app/config/docker-compose.yml up -d

# Check container status
docker-compose -f app/config/docker-compose.yml ps

# Check chatbot health
docker exec ai-chatbot curl -f http://localhost:5000/health

# Check logs for startup errors
docker-compose -f app/config/docker-compose.yml logs chatbot --tail 50

# Verify depends_on services
docker-compose -f app/config/docker-compose.yml logs mongodb --tail 20
docker-compose -f app/config/docker-compose.yml logs redis --tail 20

Docker healthcheck config (from docker-compose.yml):

healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 40s

CI / workflow considerations

When changing startup behavior, verify these workflow files still work:

CI assumptions to preserve:

• Working directory: ./services/chatbot
• Python version: 3.10
• Dependencies: tests/requirements-test.txt + requirements.txt
• TESTING=True disables live service connections
• MONGODB_ENABLED=False uses mocks instead of real DB
• No actual service startup — tests run against imported modules

If you change any of these, update the workflow file:

• Entry point file name or location
• requirements.txt path or name
• Test directory path
• Env vars the test suite checks

Where ports are defined (drift risk map)

Port 5000 (chatbot) appears in 20+ files. When changing the chatbot port, all of these must be updated:

This is why ports must not be hardcoded in new code. Read from env: int(os.getenv('CHATBOT_PORT', '5000')).

Dependency profile verification

# Check chatbot deps (venv-core)
venv-core/Scripts/python -m pip check

# Check for image-ai leakage into core
venv-core/Scripts/pip list | findstr /i "torch diffusers transformers"
# Should return nothing — these belong in venv-image

# Check profile file exists
ls app/requirements/profile_core_services.txt
ls app/requirements/profile_image_ai_services.txt

Docs-sync reminder

After resolving any startup or health issue, check:

• README.md service table — ports, entry points, and commands still match reality.
• app/scripts/README.md — if you touched a script, update the script docs.
• app/config/config.yml — if ports changed, update the YAML.
• app/config/model_config.py — ServiceConfig port values match.
• Docker healthcheck URLs — if the health endpoint path changed.

Main README.md is authoritative. If app/scripts/README.md conflicts, README.md wins.

Do not touch unless the failing path reaches them

• ComfyUI/ and services/edit-image/ComfyUI/ — external dependency subtrees.
• app/image_pipeline/ — image pipeline internals.
• services/stable-diffusion/ — unless SD is the actual failing service.
• services/edit-image/ — unless edit-image is the actual failing service.
• venv-core/ or venv-image/ — generated, never edit manually.

Required output format

After every health check or startup debug session, report:

1. Observed behavior — what actually happened (error message, HTTP status, log output).
2. Expected behavior — what should have happened instead.
3. Root cause guess — most likely explanation, with supporting evidence.
4. Files affected — which files contribute to the problem.
5. Fix applied — what was changed (if anything).
6. Next verification step — how to confirm the fix works.
7. Docs impact — whether any docs need updating after the fix.

Pre-fix checklist

Before applying a fix to a startup or health issue:

• Confirmed the Flask monolith startup path is being used.
• Confirmed which venv is active (venv-core for chatbot/MCP).
• Confirmed the shared env file loads successfully.
• Checked whether the issue reproduces in Docker, local scripts, or both.
• Verified the fix does not change behavior for unaffected startup modes.

Post-fix checklist

After applying a fix:

• The service starts without errors in the affected mode.
• GET /health returns {"status": "healthy"}.
• pytest services/chatbot/tests/ -v passes.
• If a port or entry point changed: all files in the drift risk map above are updated.
• If a startup env var changed: .env.example and README.md are updated.
• If a workflow assumption changed: the .github/workflows/ files are updated.
• No image-pipeline or ComfyUI files were modified for a chatbot-only fix.

Source: SkastVnT/AI-Assistant — distributed by TomeVault.