Setup Local

Setup Local

Ask for the Telegram bot token if not provided as an argument, then work through each step starting from Step 0.

Step 0: Check for Docker

Run docker compose version. If Docker is installed, set up using Docker and stop — do not continue to the remaining steps:

cd docker
cp .env.local.example .env
# Set TELEGRAM_BOT_TOKEN in .env
docker compose --profile local up --build

See docker/README.md for details. Report success and stop.

Native Setup (no Docker)

Do NOT follow these steps if Docker is installed. Go back to Step 0 and use Docker instead.

Tested on an NVIDIA RTX 5090.

Prerequisites

• uv, Node.js 22+, npm
• NVIDIA GPU with >= 24 GB VRAM, CUDA drivers and toolkit (nvidia-smi should work)
• A Telegram bot token from @BotFather

1. Install dependencies

# Python deps (from repo root)
uv sync --extra local --extra teacher --extra dev

# pyproject.toml pins torch to CPU — reinstall with CUDA
# Match cu1XX to your CUDA version from nvidia-smi (e.g. cu124, cu128)
uv pip install "torch>=2.1.0+cu128" torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/cu128 --reinstall
uv pip install "numpy<2.3"  # numba compatibility

# Flash Attention 2 — required for local training (default attn_implementation)
# Must install AFTER torch with --no-build-isolation so it links against the CUDA torch
uv pip install flash-attn --no-build-isolation

# OpenClaw
npm install -g openclaw@latest

Python headers: Triton JIT-compiles a CUDA extension at first vLLM startup and needs Python.h. System Python often lacks dev headers. Use a uv-managed Python (which ships with headers) to avoid this:

uv python install 3.12
uv venv --python 3.12 --clear .venv
# Re-run the installs above

2. Initialize LoRA and OpenClaw config

LORA_ROOT="${HOME}/.local/share/claas/loras"
OPENCLAW_HOME="${HOME}/.local/share/claas/openclaw-config"
mkdir -p "$LORA_ROOT" "$OPENCLAW_HOME"

CLAAS_CONFIG_NAME=local \
CLAAS_LORA_ROOT="$LORA_ROOT" \
LORA_NAME=openclaw/assistant \
MODEL=Qwen/Qwen3-8B \
VLLM_BASE_URL=http://localhost:8000/v1 \
API_KEY=sk-local \
TELEGRAM_BOT_TOKEN=<token> \
OPENCLAW_HOME="$OPENCLAW_HOME" \
  uv run python3 docker/scripts/init-stack.py

The init script writes config to $OPENCLAW_HOME/ but OpenClaw (run with HOME="$OPENCLAW_HOME") reads from $OPENCLAW_HOME/.openclaw/. Copy and fix:

cp "$OPENCLAW_HOME/openclaw.json" "$OPENCLAW_HOME/.openclaw/openclaw.json"
cp "$OPENCLAW_HOME/agents/main/agent/models.json" \
   "$OPENCLAW_HOME/.openclaw/agents/main/agent/models.json"

# Replace Docker service hostname with localhost
sed -i 's|http://claas-api:8080|http://localhost:8080|g' \
  "$OPENCLAW_HOME/.openclaw/openclaw.json"

# Feedback plugin
mkdir -p "$OPENCLAW_HOME/.openclaw/extensions"
cp -r plugins/claas-feedback "$OPENCLAW_HOME/.openclaw/extensions/claas-feedback"

# Auth credentials for the local vLLM provider
cat > "$OPENCLAW_HOME/.openclaw/agents/main/agent/auth-profiles.json" << 'EOF'
{
  "version": 1,
  "profiles": {
    "local-default": {
      "type": "api_key",
      "provider": "local",
      "key": "sk-local"
    }
  }
}
EOF

3. Start vLLM

LORA_ROOT="${HOME}/.local/share/claas/loras"
# Create the aliases file if it doesn't exist (the start script reads it)
[ -f "$LORA_ROOT/.aliases.json" ] || echo '{}' > "$LORA_ROOT/.aliases.json"

export PATH="$(pwd)/.venv/bin:$PATH"  # puts 'vllm' on PATH
export MODEL=Qwen/Qwen3-8B HOST=0.0.0.0 PORT=8000 API_KEY=sk-local
export SERVED_MODEL_NAMES=qwen3-8b MAX_MODEL_LEN=32768 GPU_MEMORY_UTILIZATION=0.70
export ENABLE_SLEEP_MODE=1 VLLM_SERVER_DEV_MODE=1 VLLM_ALLOW_RUNTIME_LORA_UPDATING=1
export ENABLE_AUTO_TOOL_CHOICE=1 TOOL_CALL_PARSER=qwen3_xml
export LORA_ROOT="$LORA_ROOT" LORA_ALIAS_FILE="$LORA_ROOT/.aliases.json" INCLUDE_ALIAS_LORAS=1
# Enable LoRA even with no initial adapters — needed for runtime LoRA loading
export EXTRA_ARGS='--enable-lora --max-lora-rank 32'

bash docker/scripts/start_vllm_qwen3_8b.sh >> /tmp/vllm.log 2>&1 &

# First run downloads Qwen3-8B (~16 GB) — expect 5-20 min
until curl -sf http://localhost:8000/health; do sleep 5; done && echo "vLLM ready"

4. Start CLaaS API

The API must be started via its Hydra entry point (not bare uvicorn) so that the runtime config is loaded and configure_web_app() is called. Override lora_root to point to the local LoRA directory (the default /loras is the Docker path).

VLLM_API_KEY=sk-local \
  uv run python -m runpy claas.api \
    lora_root="${HOME}/.local/share/claas/loras" \
    feedback_log_dir=/tmp/feedback-logs \
    'hydra.run.dir=.' \
    >> /tmp/claas-api.log 2>&1 &

curl -sf http://localhost:8080/v1/health

5. Start OpenClaw

OPENCLAW_HOME="${HOME}/.local/share/claas/openclaw-config"

TELEGRAM_BOT_TOKEN=<token> \
VLLM_BASE_URL=http://localhost:8000 \
HOME="$OPENCLAW_HOME" \
OPENCLAW_GATEWAY_TOKEN=openclaw-local-dev-token \
  openclaw gateway --port 18789 --bind lan --allow-unconfigured --verbose >> /tmp/openclaw.log 2>&1 &

6. Verify and approve pairing

curl -s http://localhost:8000/v1/models -H "Authorization: Bearer sk-local"
curl -s http://localhost:8080/v1/health
curl -s http://localhost:8080/v1/lora
tail -10 /tmp/openclaw.log  # should show "agent model: local/openclaw-assistant-latest"

When the user first messages the bot they'll receive a pairing code. Approve it:

HOME="${HOME}/.local/share/claas/openclaw-config" openclaw pairing approve telegram <CODE>

Report the status of all four components and the Telegram bot username.

Troubleshooting

Logs