# Run E2e

> Run E2E tests locally using the new-e2e framework with Pulumi-based infrastructure Use when this capability is needed.

- Skill: `tomevault-io/run-e2e` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add tomevault-io/run-e2e`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/run-e2e/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: tomevault-io (https://skillmd.com/u/tomevault-io)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/tomevault-io/run-e2e

---


Run E2E tests from `test/new-e2e/tests/` using `dda inv new-e2e-tests.run`.

## Instructions

1. **Parse `$ARGUMENTS`** to determine what to run. The user may provide:
   - A test directory path (e.g., `windows/install-test`, `agent-platform/upgrade`, `containers`)
   - A test function name (e.g., `TestInstall`, `TestUpgrade`)
   - Flags to pass through (see below)
   - A combination of the above

2. **Resolve the test target**:
   - The invoke task automatically prepends `test/new-e2e/` to targets, so targets must be relative to that directory (e.g., `./tests/agent-subcommands/flare`, NOT `./test/new-e2e/tests/...`)
   - If the user gives a directory like `agent-subcommands/flare`, use `./tests/agent-subcommands/flare` as `--targets`
   - If the user gives a partial name, search for matching directories under `test/new-e2e/tests/` using Glob, then strip the `test/new-e2e/` prefix for the target
   - If the user gives a test function name (starts with `Test`), find which package contains it using Grep under `test/new-e2e/tests/`, then set `--targets` to the package path (relative to `test/new-e2e/`) and `--run` to the test name
   - If ambiguous, list the matching options and ask the user to pick one

3. **Build the command**:
   ```
   dda inv new-e2e-tests.run --targets=./tests/<path> [flags]
   ```
   IMPORTANT: `--targets` paths are relative to `test/new-e2e/`. Do NOT include `test/new-e2e/` in the target path.

4. **Supported flags** (pass through from `$ARGUMENTS`):
   - `--run <regex>` — Only run tests matching this regex
   - `--skip <regex>` — Skip tests matching this regex
   - `--keep-stack` — Keep infrastructure up after test (for debugging)
   - `--configparams <key=value>` — Override Pulumi ConfigMap parameters
   - `--agent-image <image:tag>` — Use a specific agent image
   - `--cluster-agent-image <image:tag>` — Use a specific cluster agent image
   - `--stack-name-suffix <suffix>` — Add suffix to stack name (useful for stuck stacks)
   - `--verbose` / `--no-verbose` — Toggle verbose output (default: verbose)
   - `--max-retries <n>` — Retry failed tests up to n times
   - `--flavor <flavor>` — Package flavor (e.g., "datadog-agent")
   - `--cache` — Enable test cache (disabled by default)

5. **Before running**, confirm the full command with the user.

6. **Run the command** with a 60-minute timeout (infrastructure provisioning can take a while). Use `run_in_background` for the Bash tool since e2e tests are long-running.

7. **After completion**, summarize the results: which tests passed, which failed, and any useful error output.

## Prerequisites

The following must be configured before running e2e tests:
- `pulumi` CLI installed
- `~/.test_infra_config.yaml` exists with proper configuration

If any prerequisite is missing, inform the user what needs to be set up.

## Available Test Suites

Tests are organized under `test/new-e2e/tests/`, running `ls test/new-e2e/tests` should give the list of test packages


# Issues when running tests
If the test was previously executed and that the infra is in a weird state, that Pulumi is not aware of, trying to rerun the test with the same stack can lead to strange error, like resource being replaced, while they should not exist at all yet.
To avoid that issue it is possible to execute the test with a stack with a different name, using --stack-name-suffix <suffix>, please use short stack name suffix.
Stop the execution early when you detect that issue in the logs

## Examples

```bash
# Run all tests in a directory
dda inv new-e2e-tests.run --targets=./tests/windows/install-test

# Run a specific test
dda inv new-e2e-tests.run --targets=./tests/agent-platform/tests/upgrade --run TestUpgrade

# Keep stack for debugging
dda inv new-e2e-tests.run --targets=./tests/containers --run TestContainerLinux --keep-stack

# Run with specific agent image
dda inv new-e2e-tests.run --targets=./tests/agent-platform/tests --agent-image "my-registry/agent:latest"

# Run with stack name suffix
dda inv new-e2e-tests.run --targets=./tests/windows/install-test --stack-name-suffix 2



## Usage

- `/run-e2e windows/install-test` — Run all Windows install tests
- `/run-e2e windows/install-test --run TestInstall` — Run only TestInstall
- `/run-e2e TestUpgrade` — Auto-find and run TestUpgrade
- `/run-e2e agent-platform --keep-stack` — Run with stack kept alive

## Output

Show the user the full command before running, then report test results when done.

---
> Converted and distributed by [TomeVault](https://tomevault.io/claim/datadog) — claim your Tome and manage your conversions.
<!-- tomevault:4.0:skill_md:2026-04-11 -->