# Setup Local

> Set up the full CLaaS stack (vLLM + API + OpenClaw/Telegram) locally. Uses Docker if available, falls back to native setup otherwise.

- Skill: `experientiallabs/setup-local` (Agent Skill)
- Install (CLI): `npx skillmds@latest add experientiallabs/setup-local`
- Raw SKILL.md: https://api.skillmd.com/api/skills/experientiallabs/setup-local/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: experientiallabs (https://skillmd.com/u/experientiallabs)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/experientiallabs/setup-local

---


# Setup Local

Ask for the Telegram bot token if not provided as an argument, then work through each step starting from Step 0.

## Step 0: Check for Docker

Run `docker compose version`. If Docker is installed, set up using Docker and **stop — do not continue to the remaining steps**:

```bash
cd docker
cp .env.local.example .env
# Set TELEGRAM_BOT_TOKEN in .env
docker compose --profile local up --build
```

See `docker/README.md` for details. Report success and stop.

---

## Native Setup (no Docker)

> **Do NOT follow these steps if Docker is installed.** Go back to Step 0 and use Docker instead.

**Tested on an NVIDIA RTX 5090.**

### Prerequisites

- `uv`, Node.js 22+, npm
- NVIDIA GPU with >= 24 GB VRAM, CUDA drivers and toolkit (`nvidia-smi` should work)
- A Telegram bot token from @BotFather

### 1. Install dependencies

```bash
# Python deps (from repo root)
uv sync --extra local --extra teacher --extra dev

# pyproject.toml pins torch to CPU — reinstall with CUDA
# Match cu1XX to your CUDA version from nvidia-smi (e.g. cu124, cu128)
uv pip install "torch>=2.1.0+cu128" torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/cu128 --reinstall
uv pip install "numpy<2.3"  # numba compatibility

# Flash Attention 2 — required for local training (default attn_implementation)
# Must install AFTER torch with --no-build-isolation so it links against the CUDA torch
uv pip install flash-attn --no-build-isolation

# OpenClaw
npm install -g openclaw@latest
```

> **Python headers:** Triton JIT-compiles a CUDA extension at first vLLM startup and needs `Python.h`. System Python often lacks dev headers. Use a uv-managed Python (which ships with headers) to avoid this:
> ```bash
> uv python install 3.12
> uv venv --python 3.12 --clear .venv
> # Re-run the installs above
> ```

### 2. Initialize LoRA and OpenClaw config

```bash
LORA_ROOT="${HOME}/.local/share/claas/loras"
OPENCLAW_HOME="${HOME}/.local/share/claas/openclaw-config"
mkdir -p "$LORA_ROOT" "$OPENCLAW_HOME"

CLAAS_CONFIG_NAME=local \
CLAAS_LORA_ROOT="$LORA_ROOT" \
LORA_NAME=openclaw/assistant \
MODEL=Qwen/Qwen3-8B \
VLLM_BASE_URL=http://localhost:8000/v1 \
API_KEY=sk-local \
TELEGRAM_BOT_TOKEN=<token> \
OPENCLAW_HOME="$OPENCLAW_HOME" \
  uv run python3 docker/scripts/init-stack.py
```

The init script writes config to `$OPENCLAW_HOME/` but OpenClaw (run with `HOME="$OPENCLAW_HOME"`) reads from `$OPENCLAW_HOME/.openclaw/`. Copy and fix:

```bash
cp "$OPENCLAW_HOME/openclaw.json" "$OPENCLAW_HOME/.openclaw/openclaw.json"
cp "$OPENCLAW_HOME/agents/main/agent/models.json" \
   "$OPENCLAW_HOME/.openclaw/agents/main/agent/models.json"

# Replace Docker service hostname with localhost
sed -i 's|http://claas-api:8080|http://localhost:8080|g' \
  "$OPENCLAW_HOME/.openclaw/openclaw.json"

# Feedback plugin
mkdir -p "$OPENCLAW_HOME/.openclaw/extensions"
cp -r plugins/claas-feedback "$OPENCLAW_HOME/.openclaw/extensions/claas-feedback"

# Auth credentials for the local vLLM provider
cat > "$OPENCLAW_HOME/.openclaw/agents/main/agent/auth-profiles.json" << 'EOF'
{
  "version": 1,
  "profiles": {
    "local-default": {
      "type": "api_key",
      "provider": "local",
      "key": "sk-local"
    }
  }
}
EOF
```

### 3. Start vLLM

```bash
LORA_ROOT="${HOME}/.local/share/claas/loras"
# Create the aliases file if it doesn't exist (the start script reads it)
[ -f "$LORA_ROOT/.aliases.json" ] || echo '{}' > "$LORA_ROOT/.aliases.json"

export PATH="$(pwd)/.venv/bin:$PATH"  # puts 'vllm' on PATH
export MODEL=Qwen/Qwen3-8B HOST=0.0.0.0 PORT=8000 API_KEY=sk-local
export SERVED_MODEL_NAMES=qwen3-8b MAX_MODEL_LEN=32768 GPU_MEMORY_UTILIZATION=0.70
export ENABLE_SLEEP_MODE=1 VLLM_SERVER_DEV_MODE=1 VLLM_ALLOW_RUNTIME_LORA_UPDATING=1
export ENABLE_AUTO_TOOL_CHOICE=1 TOOL_CALL_PARSER=qwen3_xml
export LORA_ROOT="$LORA_ROOT" LORA_ALIAS_FILE="$LORA_ROOT/.aliases.json" INCLUDE_ALIAS_LORAS=1
# Enable LoRA even with no initial adapters — needed for runtime LoRA loading
export EXTRA_ARGS='--enable-lora --max-lora-rank 32'

bash docker/scripts/start_vllm_qwen3_8b.sh >> /tmp/vllm.log 2>&1 &

# First run downloads Qwen3-8B (~16 GB) — expect 5-20 min
until curl -sf http://localhost:8000/health; do sleep 5; done && echo "vLLM ready"
```

### 4. Start CLaaS API

The API must be started via its Hydra entry point (not bare `uvicorn`) so that the
runtime config is loaded and `configure_web_app()` is called. Override `lora_root`
to point to the local LoRA directory (the default `/loras` is the Docker path).

```bash
VLLM_API_KEY=sk-local \
  uv run python -m runpy claas.api \
    lora_root="${HOME}/.local/share/claas/loras" \
    feedback_log_dir=/tmp/feedback-logs \
    'hydra.run.dir=.' \
    >> /tmp/claas-api.log 2>&1 &

curl -sf http://localhost:8080/v1/health
```

### 5. Start OpenClaw

```bash
OPENCLAW_HOME="${HOME}/.local/share/claas/openclaw-config"

TELEGRAM_BOT_TOKEN=<token> \
VLLM_BASE_URL=http://localhost:8000 \
HOME="$OPENCLAW_HOME" \
OPENCLAW_GATEWAY_TOKEN=openclaw-local-dev-token \
  openclaw gateway --port 18789 --bind lan --allow-unconfigured --verbose >> /tmp/openclaw.log 2>&1 &
```

### 6. Verify and approve pairing

```bash
curl -s http://localhost:8000/v1/models -H "Authorization: Bearer sk-local"
curl -s http://localhost:8080/v1/health
curl -s http://localhost:8080/v1/lora
tail -10 /tmp/openclaw.log  # should show "agent model: local/openclaw-assistant-latest"
```

When the user first messages the bot they'll receive a pairing code. Approve it:
```bash
HOME="${HOME}/.local/share/claas/openclaw-config" openclaw pairing approve telegram <CODE>
```

Report the status of all four components and the Telegram bot username.

## Troubleshooting

| Error | Fix |
|-------|-----|
| `vllm: not found` | Prepend `.venv/bin` to PATH |
| `libtorch_cuda.so not found` | Reinstall torch with CUDA index (step 1) |
| `Numba needs NumPy 2.2 or less` | `uv pip install "numpy<2.3"` |
| `Python.h: No such file or directory` | Recreate venv with uv-managed Python (step 1 note) |
| `No API key found for provider "local"` | Create `auth-profiles.json` (step 2) |
| `flash_attn seems to be not installed` | `uv pip install flash-attn --no-build-isolation` (requires CUDA torch first) |
| vLLM OOM | Lower `GPU_MEMORY_UTILIZATION` to `0.60` |

## Logs

| Service | Log |
|---------|-----|
| vLLM | `/tmp/vllm.log` |
| CLaaS API | `/tmp/claas-api.log` |
| OpenClaw | `/tmp/openclaw.log` |