Testing

Creating Test Cases for Scenescape

Test philosophy

• Always add positive and negative cases (valid path + invalid/edge/boundary).
• Keep tests independent: own setup/teardown; no order dependence.
• Unit tests mock external deps; functional/UI use live services via fixtures.

Runtime verification (mandatory)

After creating or modifying tests, follow .github/skills/test-verification-gate/SKILL.md for target selection, image freshness, execution, and pass/fail reporting. Do not treat lint or syntax checks as verification.

Import path policy (mandatory)

Before adding imports or path setup in a new or modified test file:

1. Check tests/conftest.py and the nearest local conftest.py.
2. Confirm whether modules are already importable via existing fixtures/path setup.
3. Use direct imports (e.g. from controller...) when shared bootstrap already sets paths.
4. Add path manipulation only if no shared bootstrap exists.
5. If required, put setup once in the nearest conftest.py — not per test module.

Prohibited: sys.path.insert(...) in individual test modules when equivalent setup can live in shared conftest.py.

Authoring report: state whether conftest files were checked, where import-path setup lives, and that no unnecessary per-file sys.path.insert was added.

Category routing

Read the matching reference under Additional resources before writing a category.

Hard requirements

• Zephyr ID: every test suite/module needs NEX-T##### (TEST_NAME and/or record_xml_attribute).
• Functional/UI: declare module-level SCENESCAPE_SPEC = FuncTestSpec(...) with the correct ServiceProfile from tests/utils/profiles.py.
• Naming: files test_*.py, functions test_* (pytest norms).
• Markers that matter:
  • @pytest.mark.basic_acceptance — BAT / make run_basic_acceptance_tests
  • @pytest.mark.preserve_db — skip automatic DB restore
  • @pytest.mark.kubernetes_only — skipped on --backend=docker

Authoring checklist

Before marking a test-authoring task complete:

• Zephyr ID present
• Positive and negative (or boundary) cases; at least one negative unless N/A
• Correct category location and markers
• Functional/UI declare SCENESCAPE_SPEC
• Unit tests mock externals; functional/UI use real stack data
• Independent setup/teardown; clear docstrings
• Import-path policy followed (conftest checked; no stray sys.path.insert)
• Runtime verification done per test-verification-gate (or blocker reported)

Additional resources

• Unit tests
• Functional and integration tests (includes multi-controller hierarchy fixtures)
• UI and BAT tests
• Conftest and Zephyr
• Running tests

Automated eval cases: evals/evals.json.