NVIDIA NeMo Switchyard Testing And CI
Read .github/workflows/ci.yml before claiming local CI equivalence. Select tests from the current
diff and symbol references rather than a static subsystem inventory.
Validation Matrix
| Change | Minimum trustworthy validation |
|---|---|
| Markdown or skill Markdown only | Parse skill frontmatter when applicable; git diff --check |
| Python implementation | Focused pytest, uv run ruff check ., uv run mypy switchyard |
| Broad Python or public API | Hermetic full pytest plus lint and mypy |
| One Rust crate | cargo fmt --all --check, workspace clippy, focused crate tests |
| Shared Rust/API/FFI | Rust workspace tests plus affected Python tests |
| Packaging, extras, top-level imports | The slim-install smoke commands from the current CI workflow |
| Live provider behavior | Focused integration test only after explicit authorization |
Standard Gates
Python pre-PR gate:
uv run ruff check .
uv run mypy switchyard
env -u OPENROUTER_API_KEY -u NVIDIA_API_KEY -u OPENAI_API_KEY \
-u ANTHROPIC_API_KEY uv run pytest tests/ -v -m "not integration"
Rust workspace gate:
cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
Use -o addopts= when the repository's pytest defaults would stop failure collection too early.
Run focused tests first, then broaden only when the changed contract or shared ownership warrants it.
Live Tests
Integration tests can spend money, require credentials, and contact real services. Never run them
as an automatic pre-PR step. Provision NVIDIA_API_KEY through the environment or a secret manager
before running the user-authorized path, and state which provider was called:
uv run pytest tests/e2e/<focused_test>.py \
-v -m integration -o addopts=
Skill Validation
Each retained skill must have YAML frontmatter with name and description:
uv run python - <<'PY'
from pathlib import Path
import yaml
for path in Path(".agents/skills").glob("*/SKILL.md"):
parts = path.read_text().split("---", 2)
data = yaml.safe_load(parts[1]) if len(parts) == 3 else None
if not isinstance(data, dict) or not data.get("name") or not data.get("description"):
raise SystemExit(f"{path}: missing name or description")
print("skill frontmatter is valid")
PY
git diff --check
Reporting
Report exact commands, pass/fail status, and whether any live network call occurred. Do not call a focused test run "full CI", and do not treat mypy or clippy warnings as unrelated without checking whether the diff introduced them.