Testing

Testing

Directory Layout

tests/
  unit_tests/          # fast, isolated, no GPU required
  functional_tests/    # GPU-required integration tests
  ci_tests/            # executed in CI pipeline only

Unit tests must be CPU-compatible. Functional tests require at least one GPU. CI test scripts live in tests/ci_tests/ and should not be run locally unless reproducing a CI failure.

Test Tiers

Prefer unit tests over functional tests. CI GPU resources are limited; every functional test slot has a real cost.

Running Tests Locally

Unit Tests (CPU)

pytest tests/unit_tests/ -v

Functional Tests (GPU Required)

pytest tests/functional_tests/ -v

These require at least one GPU. Mark GPU tests with:

@pytest.mark.gpu

or:

@pytest.mark.skipif(not torch.cuda.is_available(), reason="requires GPU")

Adding a Unit Test

1. Place the file under tests/unit_tests/<domain>/test_<name>.py.
2. Keep configs tiny: small hidden dims, 1-2 layers, short sequences.
3. Run locally: pytest tests/unit_tests/<your_test>.py -v

No foreign setattr on config dataclasses in tests. When applying overrides via setattr(config_obj, key, value), always guard first:

if not hasattr(config_obj, key):
    raise ValueError(f"Config has no field '{key}'")
setattr(config_obj, key, value)

Setting a non-existent attribute silently creates a phantom field — the test passes but the recipe fails for a real user.

Adding a Functional Test

1. Place the test under tests/functional_tests/.
2. Functional tests are capped at 2 GPUs in CI.
3. Set CUDA_VISIBLE_DEVICES explicitly for multi-GPU tests.

Tips

• Keep unit test configs tiny: small hidden dims, 1-2 layers, short sequences.
• Set CUDA_VISIBLE_DEVICES explicitly when running multi-GPU tests locally.
• Watch for port conflicts when running multiple torchrun processes. Use --master_port to avoid collisions.