Tilt

You are managing PeerDB's local Tilt dev environment via the CLI.

Tilt connection

All tilt CLI commands MUST use --port 10352:

tilt --port 10352 <subcommand> [args...]

Starting and stopping Tilt

Use the tilt.sh script at the project root to start Tilt, keep it running until you choose to stop it. To top, kill the script execution.

If the script was stopped for some reason but Tilt is still running, use tilt down to stop.

Quick reference: tilt subcommands

NOTE: For wait commands, use --timeout 10 to avoid waiting indefinitely. It's better to iterate over several wait attempts with shorter timeouts than to risk getting stuck on a single long wait.

Architecture

Tilt manages Docker Compose services defined in docker-compose-dev.yml (core PeerDB) and ancillary-docker-compose.yml (test databases + infra). Environment variables for tests are in .env at the project root and are auto-loaded by Go tests.

Resource labels

Test resources and their dependencies

E2E tests (run go test ./e2e/ with a specific -run pattern):

All e2e tests also depend on core PeerDB services: flow-api, flow-worker, catalog, and provision-clickhouse.

Connector tests (run go test ./connectors/<connector>/...):

Connector tests only depend on catalog + their specific DB.

MySQL flavor overrides

MySQL tests use env var overrides to select flavor:

Toxiproxy and OpenSSH (test infrastructure)

Some tests require toxiproxy and/or openssh from the Ancillary-TestInfra label. These are manually started like ancillary DBs.

Toxiproxy (network chaos proxy, admin port 18474):

• Used by SSH keepalive tests in connectors/postgres/ssh_keepalive_test.go and connectors/mysql/ssh_keepalive_test.go to simulate network failures (down, latency, reset_peer)
• Used by some e2e tests in e2e/api_test.go and e2e/cancel_table_addition_test.go for latency injection during CDC flows
• Proxy ports: 9902, 9904, 42001-42005 (MySQL), 49001-49003 (Postgres)

OpenSSH (SSH tunnel server, port 2222, user testuser / password testpass):

• Used by SSH keepalive tests: toxiproxy sits in front of the openssh server to simulate tunnel failures
• The SSH proxy chain is: test -> toxiproxy proxy port -> openssh:2222 -> target DB

Examples of tests requiring these services:

Note: Tests that exercise toxiproxy or openssh can leave these services in an inconsistent state (e.g., lingering proxies, injected toxics, stuck SSH sessions) after failures or interruptions. If subsequent tests using them behave strangely (unexpected timeouts, refused connections, latency that shouldn't be there), recycle the affected resource before re-running:

tilt --port 10352 disable toxiproxy
tilt --port 10352 enable toxiproxy
tilt --port 10352 disable openssh
tilt --port 10352 enable openssh
tilt --port 10352 wait --for=condition=Ready uiresource/toxiproxy --timeout=60s
tilt --port 10352 wait --for=condition=Ready uiresource/openssh --timeout=60s

Getting a status overview

tilt --port 10352 get uiresource -o json | python3 -c "
import json, sys
data = json.load(sys.stdin)
for item in data.get('items', []):
    name = item['metadata']['name']
    labels = item.get('metadata', {}).get('labels', {})
    runtime = item.get('status', {}).get('runtimeStatus', 'unknown')
    update = item.get('status', {}).get('updateStatus', 'unknown')
    enabled = not item.get('status', {}).get('disableStatus', {}).get('disabled', False)
    state = 'enabled' if enabled else 'disabled'
    print(f'{name}: runtime={runtime} update={update} ({state}) labels={list(labels.keys())}')
"

Enabling resources and waiting for readiness

Ancillary databases are NOT auto-started. Enable them before running tests:

tilt --port 10352 enable postgres clickhouse

Wait for them and their provisioning to complete:

tilt --port 10352 wait --for=condition=Ready uiresource/postgres --timeout=120s
tilt --port 10352 wait --for=condition=Ready uiresource/provision-postgres --timeout=120s

For test infrastructure:

tilt --port 10352 enable toxiproxy openssh
tilt --port 10352 wait --for=condition=Ready uiresource/toxiproxy --timeout=60s
tilt --port 10352 wait --for=condition=Ready uiresource/openssh --timeout=60s

Interpreting arguments

When the user provides $ARGUMENTS:

• If it's a status/overview request (e.g., status, list, overview): show a summary of all resources grouped by label with their health.
• If it's a specific action (e.g., logs e2e_postgres, enable postgres, trigger e2e_mysql-gtid): execute that tilt command.
• If it's a resource name (e.g., e2e_postgres, connector_clickhouse, postgres): show its status, logs, and health.
• If it's a clean up request (e.g., cleanup, destroy): run the cleanup commands to stop containers, teardown Tilt environment, and remove volumes.
• If empty or ambiguous: show the status overview and ask what the user needs.
• if it's a start tilt request** (e.g., start, up): start Tilt according to the instructions above.
• If it's a stop tilt request** (e.g., stop, shutdown): stop Tilt gracefully according to the instructions above.

Cleaning up the environment

To wipe all PeerDB Docker volumes (for a fresh start), first stop containers then remove volumes:

tilt --port 10352 disable --all
docker compose -f docker-compose-dev.yml -f ancillary-docker-compose.yml down
docker volume ls | awk '{print $2}' | grep peerdb | xargs docker volume rm -f

The Tilt UI also has a "Wipe ancillary volumes" button in the nav bar that removes ancillary DB volumes only (without touching core PeerDB volumes).

Important notes

• Always check that required databases are enabled and healthy before triggering tests.
• E2e tests need core PeerDB services (flow-api, flow-worker, catalog) running -- these auto-start with tilt.
• When streaming logs with -f, use a timeout or --since to avoid blocking indefinitely.
• If a test fails with "connection refused" or similar, the most likely cause is that the required database wasn't enabled/provisioned.
• Always one resource per tilt command for actions like trigger. For multiple, run separate commands or use a loop in bash.
• When starting resources, it is not enough with enabling them, you need to trigger them.