Run Integration Tests

Run Integration Tests

Integration tests live in sandbox-api/integration-tests/ and test real HTTP endpoints against a running sandbox-api instance.

CI runs them automatically on every PR touching sandbox-api/ (.github/workflows/test.yaml), in both identity modes. Running them by hand is for debugging a failure or developing a new test — see the test-changes skill for what to add.

Quick Start (Recommended)

Run against an already-running dev environment:

# 1. Start the dev environment first (in another terminal)
docker-compose up dev

# 2. Run integration tests
make integration-test

Run With Docker Auto-Start

Let the test script start and stop Docker automatically:

cd sandbox-api/integration-tests
START_API=true ./run_tests.sh

Or with a custom compose file:

DOCKER_COMPOSE_FILE=../docker-compose.yaml START_API=true ./run_tests.sh

Run Against a Remote/Custom Host

cd sandbox-api/integration-tests
API_HOST=<hostname> API_PORT=8080 ./run_tests.sh

Run a Specific Test File or Test

cd sandbox-api/integration-tests
API_BASE_URL=http://localhost:8080 go test -v ./tests/filesystem/...
API_BASE_URL=http://localhost:8080 go test -v ./tests/process/...
API_BASE_URL=http://localhost:8080 go test -v ./tests/network/...
API_BASE_URL=http://localhost:8080 go test -v ./tests/mcp/...
API_BASE_URL=http://localhost:8080 go test -v ./tests/codegen/...

# Run a single test by name
API_BASE_URL=http://localhost:8080 go test -v -run TestFilesystemRead ./tests/filesystem/...

Test Coverage Areas

tests/network/ is skipped in CI: it needs a real sandbox network stack.

Run The Unprivileged Workload Mode

When sandbox-api supervises a non-root workload user (Dockerfile USER / --user / BL_SANDBOX_USER), it stays root itself and scopes what it runs for the user. That mode has its own runner, which creates the user, builds and boots the API and runs tests/identity:

cd sandbox-api/integration-tests
sudo ./run_identity_tests.sh              # WORKLOAD_USER=sbxtest by default

Against an instance you already started, point the suite at it and tell it which identity to expect:

API_BASE_URL=http://localhost:8080 WORKLOAD_USER=app go test -v ./tests/identity/...

With WORKLOAD_USER unset, the same suite asserts the opposite contract: an API with no identity configured still does everything as root.

Adding a New Integration Test

1. Pick the matching directory in tests/ (or create one)
2. Create a *_test.go file with package tests
3. Use shared helpers from common/:

package tests

import (
    "net/http"
    "testing"

    "github.com/blaxel-ai/sandbox-api/integration_tests/common"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/require"
)

func TestMyNewEndpoint(t *testing.T) {
    resp, err := common.MakeRequest(http.MethodGet, "/my-endpoint", nil)
    require.NoError(t, err)
    defer resp.Body.Close()

    assert.Equal(t, http.StatusOK, resp.StatusCode)

    var result map[string]interface{}
    err = common.ParseJSONResponse(resp, &result)
    require.NoError(t, err)
}

Troubleshooting

• connection refused: The sandbox-api is not running. Start it with docker-compose up dev first
• Test hangs: A previous test left a process running; restart the dev container
• Codegen tests skip: RELACE_API_KEY or MORPH_API_KEY env vars not set — codegen tools require an LLM provider key