Panoramic

/panoramic

panoramic runs correctness cases from test/correctness/cases and integration cases from test/integration/cases. Get the user's approval before running these system-level suites.

Discover the current interface

Check the current Makefile recipes and panoramic --help before choosing a command. Use panoramic run --help for runner options and panoramic list --help to discover selection options. This keeps commands aligned with the working tree as the harness changes.

These human-facing documents provide optional background:

• docs/development/testing.md: test types and CI entry points
• docs/development/testing-patterns.md: conventions for Rust tests
• bin/correctness/README.md: Panoramic and its supporting tools

Choose the suite and runtime

Prefer the Make targets when test dependencies need rebuilding. Confirm their current recipes in the Makefile; the standard entry points are:

make test-correctness
make test-correctness-case CASE=<case>
make test-integration

make test-integration-quick skips image builds. Use it only after verifying that the local images match the working tree.

--runtime scopes integration discovery. On macOS, assume the linux runtime through the local Docker setup unless the user explicitly requests native macOS tests; pass --runtime linux because the CLI defaults to mac there. Follow the current Make recipe for native macOS provisioning. Correctness cases choose their runtime from their case configuration.

Use panoramic list with the intended test directory and runtime to discover eligible case names before selecting them.

Run non-interactively

The Panoramic test targets write artifacts beneath {{saluki}}/target/test-output/panoramic/ by default. This in-tree path is visible to Docker on macOS. Set SALUKI_TEST_OUTPUT_DIR to relocate the shared test-output root or PANORAMIC_LOG_DIR to override Panoramic only. Use the same base for direct runs.

make build-panoramic
target/release/panoramic run \
  -d "$(pwd)/test/integration/cases" \
  --runtime linux \
  -t <case-name> \
  --no-tui -o json -l "$(pwd)/target/test-output/panoramic"

Use comma-separated names with -t and -p 1 when one case needs attributable output. Each run creates a timestamped panoramic-* directory beneath the base. make clean-test-logs deletes the Panoramic directory under the shared test-output root; make clean includes that target. Stdout and run.json identify the directory for a run.

Interpret the result

Exit codes distinguish the run-level result:

Treat code 2 as an environment or harness problem before investigating assertion differences.

A completed run writes run.json; each test directory contains result.json, result.log, and its captured artifacts. Start with the non-passing entries in run.json, follow each entry's log_dir, then use the paths in result.json rather than constructing artifact paths yourself.

jq -r '.tests[] | select(.outcome != "passed") | "\(.name) \(.outcome) \(.log_dir)"' run.json
jq -r '.assertions[] | select(.passed == false) | .kind + ": " + .message' <case-log-dir>/result.json
jq -r '.diagnostic_lines[]? | "\(.source):\(.line_number) \(.text)"' <case-log-dir>/result.json

outcome is the verdict: passed, failed, errored, or timed_out. For a timeout, read the timeout object to identify the deadline, active phase, and configured duration. For other failures, read error, failed assertions, and diagnostic_lines, then open the referenced artifacts for context. Correctness assertion details may point to uncapped files under details/.

run.json includes build_revision: the source revision used to build Panoramic, with -dirty for a modified checkout and unknown when it cannot be determined.

Read the traffic artifacts

A correctness test records what it sent and what each side decoded under <log_dir>/traffic/. Start with traffic/manifest.json: it names each capture, reports its compressed size, and says whether the file is still on disk (file.present). A passing test keeps only the manifest; every other outcome keeps the captures. When input is null, read input_unavailable_reason - the kubernetes_in_docker runtime never produces an input capture.

The captures are zstd-compressed JSON Lines, one record per line, so decompress before reading:

zstd -dc traffic/input.jsonl.zst | head -5
zstd -dc traffic/baseline-decoded.jsonl.zst | jq -c 'select(.kind == "metric") | .value.context'

Input records are in send order, which is not packet order. Decoded records are grouped by kind and indexed within each kind, so index is a position within a kind, not a global receive order.