Claude Speed Test

Claude Speed Test

Benchmark Claude Code response time across different CLAUDE_CONFIG_DIR locations to find the fastest configuration.

Quick Start

# Run 3 trials (default)
python ~/.claude/skills/claude-speed-test/benchmark.py

# Run 10 trials
python ~/.claude/skills/claude-speed-test/benchmark.py 10

# Run without auto-cleanup (keep session files for inspection)
python ~/.claude/skills/claude-speed-test/benchmark.py 5 --no-cleanup

What It Tests

By default, benchmarks these 2 configurations:

Key Parameters

• CLAUDE_CODE_EFFORT_LEVEL=0: Minimizes agent thinking time for consistent measurements
• cwd=/tmp/tests: Isolated working directory so test sessions go to -private-tmp-tests folder
• UUIDv7 in prompt: Each run has unique ID for traceability (chatId=<uuid7>)

Output

Trial 1: ~/.claude (default)    = 5.51s
Trial 1: /tmp/claude-sessions   = 4.35s
...
============================================================
SUMMARY
============================================================
Config                   Mean      Std      Min      Max
------------------------------------------------------------
~/.claude (default)     5.25s    0.28s    4.90s    5.72s
/tmp/claude-sessions    4.47s    0.37s    4.06s    5.41s
------------------------------------------------------------
Fastest: /tmp/claude-sessions (4.47s)

Log File

Each run is logged to benchmark_runs.jsonl (in the script directory) with:

• run_id: UUIDv7 for tracing
• config_dir, config_label: Which config was tested
• prompt: Full prompt sent (includes chatId)
• cwd: Working directory used
• env_vars: Environment variables set
• start_time, end_time, elapsed_seconds: Timing data
• stdout, stderr, exit_code: Response data

Auto-Cleanup

By default, the script:

1. Pre-cleanup: Deletes previous test sessions before running
2. Post-cleanup: Deletes test sessions after running

Test sessions are stored in -private-tmp-tests folders (based on cwd /tmp/tests).

To disable cleanup: --no-cleanup

Customizing Configurations

Edit CONFIG_OPTIONS in benchmark.py:

CONFIG_OPTIONS = [
    {"dir": None, "label": "~/.claude (default)"},
    {"dir": "/tmp/claude-sessions", "label": "/tmp/claude-sessions"},
    {"dir": "/tmp/my-custom-dir", "label": "my-custom"},  # Add your own
]

OAuth / Authentication

New config directories need OAuth tokens to authenticate. Two methods:

Method 1: Copy Existing Token (Recommended)

Copy oauth_token_dump.json from a working config dir:

# Create new config dir
mkdir -p /tmp/new-config-dir

# Copy OAuth token from an existing working config
cp ~/.claude/oauth_token_dump.json /tmp/new-config-dir/

Token location:

• ~/.claude/oauth_token_dump.json

Method 2: Generate New Token (Requires Claude Code 2.0.42)

If no token exists, generate one using Claude Code version 2.0.42 which supports CLAUDE_CODE_ENABLE_OAUTH_TOKEN_DUMP:

# Path to Claude Code 2.0.42
CLAUDE_2042=~/swe/claude-code-2.0.42/cli.js

# Generate token dump (runs a quick session)
CLAUDE_CODE_ENABLE_OAUTH_TOKEN_DUMP=1 \
  CLAUDE_CONFIG_DIR=/tmp/new-config-dir \
  $CLAUDE_2042 -p "OAuth refresh ping"

# Verify token was created
cat /tmp/new-config-dir/oauth_token_dump.json | python -m json.tool

Note: This requires ~/swe/claude-code-2.0.42/cli.js to exist. Newer versions (2.1.x) removed this feature.

Token Structure

{
  "event": "G5_store_load",
  "ts": "2025-11-15T12:34:56.789Z",
  "claudeAiOauth": {
    "accessToken": "sk-ant-oat01-...",
    "refreshToken": "sk-ant-ort01-...",
    "expiresAt": 1766530380177,
    "scopes": ["user:inference", "user:profile", "user:sessions:claude_code"],
    "subscriptionType": "max",
    "rateLimitTier": "default_claude_max_20x"
  }
}

Using Token Explicitly (Advanced)

Instead of copying the file, you can set the token via environment variable:

# Extract access token
export CLAUDE_CODE_OAUTH_TOKEN=$(cat ~/.claude/oauth_token_dump.json | \
  python -c "import json,sys; print(json.load(sys.stdin)['claudeAiOauth']['accessToken'])")

# Run Claude with explicit token
CLAUDE_CODE_OAUTH_TOKEN="$CLAUDE_CODE_OAUTH_TOKEN" claude -p "test"

Session Folder Naming

Claude creates session folders based on cwd with / replaced by -:

This makes cleanup safe and predictable.

Findings

Typical results show:

• Fresh config dirs (fewer accumulated sessions) are faster
• ~1s difference between fastest and slowest configs
• High variance due to network latency to Anthropic API

Recommended alias for speed:

alias cc='CLAUDE_CODE_EFFORT_LEVEL=0 CLAUDE_CONFIG_DIR=/tmp/claude-sessions ~/symlinks/claude --dangerously-skip-permissions'