Testdriver:customizing Devices

TestDriver Options

Configure TestDriver behavior with options passed to the TestDriver() function:

const testdriver = TestDriver(context, {
  // === Sandbox & Connection ===
  newSandbox: true,          // Force creation of a new sandbox (default: true)
  reconnect: false,          // Reconnect to last sandbox (default: false)
  keepAlive: 60000,          // Keep sandbox alive after disconnect in ms (default: 60000)
  os: "linux",               // 'linux' | 'windows' (default: 'linux')
  resolution: "1366x768",    // Sandbox resolution (e.g., '1920x1080')
  ip: "203.0.113.42",        // Direct IP for self-hosted sandbox
  sandboxAmi: "ami-1234",    // Custom AMI ID (AWS deployments)
  sandboxInstance: "i3.metal", // EC2 instance type (AWS deployments)

  // === Preview & Debugging ===
  preview: "browser",        // "browser" | "ide" | "none" (default: "browser")
  headless: false,           // @deprecated - use preview: "none" instead
  debugOnFailure: false,     // Keep sandbox alive on test failure for debugging

  // === Caching ===
  cache: true,               // Enable element caching (default: true)
  // Or use advanced caching config:
  // cache: {
  //   enabled: true,
  //   thresholds: {
  //     find: { screen: 0.05, element: 0.8 },
  //     assert: 0.05
  //   }
  // },
  cacheKey: "my-test",       // Cache key for element finding operations

  // === Recording & Screenshots ===
  dashcam: true,             // Enable/disable Dashcam video recording (default: true)
  autoScreenshots: true,     // Capture screenshots before/after each command (default: false)

  // === AI Configuration ===
  ai: {                      // Global AI sampling configuration
    temperature: 0,          // 0 = deterministic, higher = more creative
    top: {
      p: 0.9,               // Top-P nucleus sampling (0-1)
      k: 40,                // Top-K sampling (1 = most likely, 0 = disabled)
    },
  },

  // === Screen Change Detection ===
  redraw: true,              // Enable redraw detection (default: true)
  // Or use advanced redraw config:
  // redraw: {
  //   enabled: true,
  //   thresholds: {
  //     screen: 0.05,       // Pixel diff threshold (0-1), false to disable
  //     network: false,     // Monitor network activity (default: false)
  //   }
  // },

  // === Logging & Analytics ===
  logging: true,             // Enable console logging output (default: true)
  analytics: true,           // Enable analytics tracking (default: true)

  // === Advanced ===
  apiRoot: "https://...",    // API endpoint URL (for self-hosted deployments)
  environment: {},           // Additional environment variables for the sandbox
});

Preview Mode

Control how test execution is visualized. The preview option determines where the live debugger view opens:

const testdriver = TestDriver(context, {
  preview: "browser",  // Opens in default browser (default)
});

IDE Preview

For the best development experience, use preview: "ide" with the TestDriver extension for VSCode or Cursor:

const testdriver = TestDriver(context, {
  preview: "ide",  // Opens preview in IDE panel
});

Headless Mode

Run tests without any visual preview. Useful for CI/CD pipelines:

const testdriver = TestDriver(context, {
  preview: "none",  // No visual preview (headless)
});

Debug on Failure

Keep the sandbox alive when a test fails so you can reconnect and debug interactively. The sandbox ID is printed to the console along with instructions for reconnecting via MCP.

const testdriver = TestDriver(context, {
  debugOnFailure: true,
});

IP Target

If self-hosting TestDriver, use ip to specify the device IP. See Self-Hosting TestDriver for details.

const testdriver = TestDriver(context, {
  ip: "203.0.113.42",  // Your allowlisted IP
});

Operating System

Set the os property to run tests on a specific operating system. Available options are linux (default) and windows.

const testdriver = TestDriver(context, {
  os: "windows",  // Run on Windows sandbox
});

Using Environment Variables

You can make the operating system configurable via environment variables. This requires adding code to read from process.env in your test:

const testdriver = TestDriver(context, {
  os: process.env.TD_OS || "linux",  // Read from env, default to Linux
});

Then pass the variable when running tests:

# Run tests on Windows
TD_OS=windows vitest run

# Run tests on Linux (default)
TD_OS=linux vitest run

This pattern is useful for running the same test suite across multiple operating systems in CI/CD:

# Example GitHub Actions matrix
strategy:
  matrix:
    os: [linux, windows]
steps:
  - run: TD_OS=${{ matrix.os }} vitest run

Dashcam Recording

Dashcam video recording is enabled by default. Disable it to skip recording:

const testdriver = TestDriver(context, {
  dashcam: false,
});

Automatic Screenshots

Screenshots are automatically captured before and after every command (click, type, find, assert, etc.) by default. Each screenshot filename includes the line number from your test file.

Disable automatic screenshots:

const testdriver = TestDriver(context, {
  autoScreenshots: false,
});

Caching

Element caching speeds up repeated find() and assert() calls. Enabled by default.

// Disable caching
const testdriver = TestDriver(context, {
  cache: false,
});

// Advanced: custom thresholds
const testdriver = TestDriver(context, {
  cache: {
    enabled: true,
    thresholds: {
      find: { screen: 0.05, element: 0.8 },
      assert: 0.05,
    },
  },
  cacheKey: "my-test",
});

Redraw Detection

Redraw detection waits for the screen to stabilize before taking actions. Enabled by default.

// Disable redraw detection
const testdriver = TestDriver(context, {
  redraw: false,
});

// Advanced: custom thresholds with network monitoring
const testdriver = TestDriver(context, {
  redraw: {
    enabled: true,
    thresholds: {
      screen: 0.05,
      network: true,
    },
  },
});

AI Configuration

Control how the AI model generates responses for find() verification and assert() calls:

const testdriver = TestDriver(context, {
  ai: {
    temperature: 0,       // 0 = deterministic
    top: { p: 0.9, k: 40 },
  },
});

Environment Variables

Pass additional environment variables to the sandbox:

const testdriver = TestDriver(context, {
  environment: {
    MY_VAR: "value",
    DEBUG: "true",
  },
});

Keepalive

By default, sandboxes stay alive for 60 seconds after disconnect. Customize this with keepAlive:

const testdriver = TestDriver(context, {
  keepAlive: 300000,  // Keep sandbox alive for 5 minutes after test
});

Set to 0 to terminate immediately:

const testdriver = TestDriver(context, {
  keepAlive: 0,  // Terminate sandbox immediately on disconnect
});

Connecting to an Existing Sandbox

Speed up test development by connecting to an existing sandbox instead of starting fresh each time. This lets you iterate quickly on failing steps without re-running the entire test from the beginning.

Split your test into two files: one for known-good steps that set up the desired state, and another for work-in-progress steps you want to debug.

const testdriver = TestDriver(context, {
  keepAlive: 60000,  // Keep sandbox alive for 60 seconds after test
});

// Second test file: experiment.test.mjs (run within keepAlive window)
const testdriver = TestDriver(context, {
  keepAlive: 60000,
});

await testdriver.connect({ sandboxId: "sandbox-abc123" });

Then, you can run both tests in sequence:

vitest run -t known-good.test.mjs -t work-in-progress.test.mjs

And as you make changes to work-in-progress.test.mjs, you can re-run just that file to quickly iterate on the failing steps.

vitest run work-in-progress.test.mjs