Local Testing

Local Testing Skill

This skill documents how to properly set up a local development environment for testing, including running unit tests and CLI E2E tests with mock Claude.

Quick Start

Unit Tests Only

# 1. Install dependencies
cd turbo && pnpm install

# 2. Run unit tests
cd turbo && pnpm vitest run

CLI E2E Tests

# 1. Start dev server with tunnel (required for webhooks)
/dev-server:start

# 2. Wait for server to be ready, then authenticate CLI
/dev-server:auth

# 3. Deploy runner (needed for vm0 run, takes several minutes)
/dev-server:runner

# 4. Run CLI E2E tests
VM0_API_URL=http://localhost:3000 USE_MOCK_CLAUDE=true BATS_TEST_TIMEOUT=60 \
  ./e2e/test/libs/bats/bin/bats -T ./e2e/tests/01-serial/*.bats

Prerequisites

1. Environment Variables

Ensure the following are set in turbo/apps/web/.env.local:

If environment variables are missing, ask the user to run the sync script:

scripts/sync-env.sh

Note: sync-env.sh requires 1Password authentication and can only be executed by the user directly. If you encounter missing environment variable errors, request the user to run this script.

2. Add Mock Claude Configuration

# Add to turbo/apps/web/.env.local
echo "USE_MOCK_CLAUDE=true" >> turbo/apps/web/.env.local
echo "CONCURRENT_RUN_LIMIT_CAP=0" >> turbo/apps/web/.env.local

Starting the Dev Server

Using the Skill

/dev-server:start

This will:

• Start the Turbo dev server in background
• Automatically start a Cloudflare tunnel for the web app
• Set VM0_API_URL to the tunnel URL
• Enable webhooks to reach your local server

Manual Start

cd turbo && pnpm dev

Verify Server is Ready

Check for these indicators in the logs (/dev-server:logs):

• [tunnel] Tunnel URL: https://xxx.trycloudflare.com
• Ready in XXXms for each app
• No fatal errors

Important: The tunnel URL changes each time you restart. Sandbox webhooks use this URL to send events back to your local server.

Running Unit Tests

Prerequisites

Before running unit tests, ensure dependencies are installed:

cd turbo && pnpm install

Run All Tests

cd turbo && pnpm vitest run

Expected output:

Test Files  173 passed (173)
Tests       2348 passed (2348)

Run Tests in Watch Mode

cd turbo && pnpm vitest

Run Specific Test File

cd turbo && pnpm vitest run apps/web/src/lib/__tests__/my-test.test.ts

Run Tests for a Specific Package

cd turbo && pnpm vitest run --project @vm0/cli
cd turbo && pnpm vitest run --project web

Running CLI E2E Tests

Test Structure

e2e/tests/
├── 01-serial/              # Tests that MUST run serially (scope setup)
└── 03-runner/ # All parallel tests (runs on runner)

Environment Variables for E2E

Running Serial Tests

VM0_API_URL=http://localhost:3000 \
USE_MOCK_CLAUDE=true \
BATS_TEST_TIMEOUT=30 \
./e2e/test/libs/bats/bin/bats -T ./e2e/tests/01-serial/*.bats

Running Parallel Tests

VM0_API_URL=http://localhost:3000 \
USE_MOCK_CLAUDE=true \
BATS_TEST_TIMEOUT=60 \
./e2e/test/libs/bats/bin/bats -T -j 10 --no-parallelize-within-files ./e2e/tests/03-runner/*.bats

Running a Single Test File

VM0_API_URL=http://localhost:3000 \
USE_MOCK_CLAUDE=true \
BATS_TEST_TIMEOUT=60 \
./e2e/test/libs/bats/bin/bats -T ./e2e/tests/03-runner/t17-vm0-simplified-compose.bats

Running a Specific Test by Name

VM0_API_URL=http://localhost:3000 \
USE_MOCK_CLAUDE=true \
BATS_TEST_TIMEOUT=60 \
./e2e/test/libs/bats/bin/bats -T ./e2e/tests/03-runner/t17-vm0-simplified-compose.bats \
  --filter "vm0 compose with both instructions and skills"

How Mock Claude Works

When USE_MOCK_CLAUDE=true:

1. The web server passes this to the sandbox via environment variable
2. Inside sandbox, run-agent.ts checks for USE_MOCK_CLAUDE
3. Instead of running real claude CLI, it runs the mock Claude script
4. Mock Claude executes the prompt as a bash command and outputs Claude-compatible JSONL

Troubleshooting

Unit Test Issues

Problem: "@radix-ui/react-select could not be resolved" or Missing Dependencies

Cause: Dependencies not installed or out of sync

Solution:

cd turbo && pnpm install

Problem: Tests Fail After Pulling New Changes

Solution:

cd turbo && pnpm install
cd turbo && pnpm vitest run

CLI E2E Test Issues

Problem: "Failed to fetch events" or Run Hangs

Cause: Sandbox cannot reach the tunnel URL (webhooks fail)

Solution:

1. Check if tunnel is active: /dev-server:logs tunnel
2. Test tunnel connectivity:

   curl -v https://<tunnel-url>/api/webhooks/agent/events
   

3. If SSL errors occur, restart dev server to get a fresh tunnel:

   /dev-server:stop
   /dev-server:start
   

Problem: "Concurrent agent run limit" Error

Cause: CONCURRENT_RUN_LIMIT_CAP not set or not 0

Solution:

echo "CONCURRENT_RUN_LIMIT_CAP=0" >> turbo/apps/web/.env.local
# Then restart dev server

Problem: Tests Timeout

Cause: vm0 run takes ~15-30 seconds per execution

Solution:

• Ensure BATS_TEST_TIMEOUT is set appropriately (60s for parallel tests)
• Check server logs for errors: /dev-server:logs error
• Verify sandbox is starting: /dev-server:logs sandbox

Problem: "SECRETS_ENCRYPTION_KEY" Missing or Other Environment Variables Missing

Cause: Environment variables not synced from 1Password

Solution: Ask the user to run the sync script (requires 1Password authentication):

scripts/sync-env.sh

Note: This script can only be executed by the user directly as it requires interactive 1Password authentication.

Problem: SSL Certificate Errors

Cause: CF_DNS_AND_TUNNEL_API_TOKEN not set. Caddy needs this to provision Let's Encrypt certificates via DNS-01 challenge.

Solution:

scripts/sync-env.sh

Problem: "parallel: command not found" When Running Parallel E2E Tests

Cause: GNU Parallel not installed

Solution:

sudo apt-get install -y parallel

Problem: Port Already in Use

Solution:

# Kill processes on dev ports
fuser -k 3000/tcp 3001/tcp 3002/tcp 3003/tcp

# Or use dev-stop
/dev-server:stop

Problem: "SyntaxError: Unexpected end of JSON input" During Parallel Tests

Cause: High concurrency causing request body truncation (local dev environment limitation)

Solution:

• This is a transient issue in local dev, CI environment doesn't have this problem
• Run fewer parallel jobs: -j 4 instead of -j 10
• Or run tests individually to verify they pass

CI vs Local Differences

Useful Commands

# Check dev server status
/dev-server:logs

# Filter logs by pattern
/dev-server:logs error
/dev-server:logs tunnel
/dev-server:logs sandbox

# Stop dev server
/dev-server:stop

# Authenticate CLI with local server
/dev-server:auth

# Check CLI auth status
vm0 auth status

# Manual test of vm0 run with mock Claude
vm0 run <agent-name> --artifact-name <artifact> "echo hello"

Reference

• CLI E2E Testing Patterns: .claude/skills/cli-e2e-testing/skill.md
• Dev Server Management: .claude/skills/dev-server/skill.md
• Runner Executor: turbo/apps/web/src/lib/run/executors/runner-executor.ts