AI-Q Release QA
Use this skill to validate a change in the AI-Q repository before opening or
merging a pull request. The goal is to run the narrowest set of checks that
covers what you touched, then broaden only when a change crosses shared
boundaries — not to run every command every time.
Start Here
- Identify which surfaces the change touches: backend Python (
src/,
sources/, tests/), web UI (frontends/ui/), docs (docs/), or evals
(frontends/benchmarks/).
- Run the scoped checks for those surfaces first (see the matrix below).
- Broaden to the full suite only when the change crosses shared boundaries
(for example editing
src/aiq_agent/common/ or a config many agents load).
- Capture the exact commands and their output —
aiq-prepare-pr requires this
as validation evidence.
- Never paste secrets or
deploy/.env values into command output you share.
Authoritative References
- AGENTS.md: the "Build, test, and validation commands"
section is the source of truth for every command below.
- CONTRIBUTING.md: "Local Validation" — the exact
commands to run and the requirement to include their output in the PR.
pyproject.toml and uv.lock: root Ruff config (line length 120, rule sets
E,F,W,I,PL,UP) and the root dev environment.
mcp/pyproject.toml and mcp/uv.lock: the independent MCP runtime and dev
environment. MCP is not a root dependency group.
frontends/ui/package.json: the real scripts (lint, type-check,
test:ci, build) — use these names, do not invent npm scripts.
For the full per-surface command list and expected results:
- references/validation-matrix.md
Workflow
- List the changed paths (
git status, git diff --name-only) and map them to
surfaces.
- Run the scoped Python, UI, docs, or eval checks for those surfaces.
- If the change touches shared code or config, broaden to the full suite for
that surface.
- Re-run until lint, format, type, and tests all pass; fix failures rather than
skipping checks.
- Record the exact commands and their output for the PR description.
Validation
Pick the block that matches the change. Run the narrowest first.
Backend Python (run from the repo root; the project uses uv):
uv run ruff check <changed paths> # lint
uv run ruff format --check <changed paths> # format check
uv run pytest <scoped test paths> # tests
MCP server (run from the repo root):
uv sync --project mcp --extra dev
uv run ruff check mcp
uv run ruff format --check mcp
uv run --project mcp --extra dev pytest mcp/tests
Broaden to the whole tree when the change crosses shared boundaries:
uv run ruff check .
uv run ruff format --check .
uv run pytest
uv run --project mcp --extra dev pytest mcp/tests
Frontend (from frontends/ui/):
npm run lint
npm run type-check
npm run test:ci
Expected: Ruff reports no lint or format failures, tests pass, and the frontend
lint/type-check/test commands exit cleanly. For docs and eval changes, see the
matrix reference.
Common Mistakes
- Running both complete root and MCP test suites for a one-package change
instead of scoping to the touched paths first — slow, and it buries the
relevant signal.
- Running MCP tests in the root environment; use
uv run --project mcp so the
check consumes the isolated MCP lock and dev extra.
- Skipping
ruff format --check and pushing unformatted code that fails CI.
- Hand-reformatting unrelated code; only the changed code should move.
- Forgetting that
nat eval needs deploy/.env; run evals via
dotenv -f deploy/.env run nat eval ... (see the matrix reference).
- Inventing npm scripts; stick to the canonical checks
lint, type-check,
test:ci, and build defined in frontends/ui/package.json.
Related Skills
aiq-prepare-pr
aiq-add-tool
aiq-add-data-source
1---2name: aiq-release-qa3description: Use when validating an AI-Q change before opening or merging a PR — choosing and running the right Python, frontend, docs, or eval checks for the surfaces you touched instead of one fixed command list.4license: Apache-2.05compatibility: Claude Code, Codex, Cursor, OpenCode, and Agent Skills-compatible tools.6metadata:7 source: https://skillmd.com/skills/nvidia-ai-blueprints-aiq/aiq-release-qa8 version: "0.1.0"9 source-repo: "NVIDIA-AI-Blueprints/aiq"10 tags: "aiq nemo-agent-toolkit validation testing release-qa"11allowed-tools: Read Bash12---1314# AI-Q Release QA1516Use this skill to validate a change in the AI-Q repository before opening or17merging a pull request. The goal is to run the **narrowest set of checks that18covers what you touched**, then broaden only when a change crosses shared19boundaries — not to run every command every time.2021## Start Here2223- Identify which surfaces the change touches: backend Python (`src/`,24 `sources/`, `tests/`), web UI (`frontends/ui/`), docs (`docs/`), or evals25 (`frontends/benchmarks/`).26- Run the scoped checks for those surfaces first (see the matrix below).27- Broaden to the full suite only when the change crosses shared boundaries28 (for example editing `src/aiq_agent/common/` or a config many agents load).29- Capture the exact commands and their output — `aiq-prepare-pr` requires this30 as validation evidence.31- Never paste secrets or `deploy/.env` values into command output you share.3233## Authoritative References3435- [AGENTS.md](../../../AGENTS.md): the "Build, test, and validation commands"36 section is the source of truth for every command below.37- [CONTRIBUTING.md](../../../CONTRIBUTING.md): "Local Validation" — the exact38 commands to run and the requirement to include their output in the PR.39- `pyproject.toml` and `uv.lock`: root Ruff config (line length 120, rule sets40 `E,F,W,I,PL,UP`) and the root dev environment.41- `mcp/pyproject.toml` and `mcp/uv.lock`: the independent MCP runtime and dev42 environment. MCP is not a root dependency group.43- `frontends/ui/package.json`: the real `scripts` (`lint`, `type-check`,44 `test:ci`, `build`) — use these names, do not invent npm scripts.4546For the full per-surface command list and expected results:4748- [references/validation-matrix.md](references/validation-matrix.md)4950## Workflow51521. List the changed paths (`git status`, `git diff --name-only`) and map them to53 surfaces.542. Run the scoped Python, UI, docs, or eval checks for those surfaces.553. If the change touches shared code or config, broaden to the full suite for56 that surface.574. Re-run until lint, format, type, and tests all pass; fix failures rather than58 skipping checks.595. Record the exact commands and their output for the PR description.6061## Validation6263Pick the block that matches the change. Run the narrowest first.6465Backend Python (run from the repo root; the project uses `uv`):6667```bash68uv run ruff check <changed paths> # lint69uv run ruff format --check <changed paths> # format check70uv run pytest <scoped test paths> # tests71```7273MCP server (run from the repo root):7475```bash76uv sync --project mcp --extra dev77uv run ruff check mcp78uv run ruff format --check mcp79uv run --project mcp --extra dev pytest mcp/tests80```8182Broaden to the whole tree when the change crosses shared boundaries:8384```bash85uv run ruff check .86uv run ruff format --check .87uv run pytest88uv run --project mcp --extra dev pytest mcp/tests89```9091Frontend (from `frontends/ui/`):9293```bash94npm run lint95npm run type-check96npm run test:ci97```9899Expected: Ruff reports no lint or format failures, tests pass, and the frontend100lint/type-check/test commands exit cleanly. For docs and eval changes, see the101matrix reference.102103## Common Mistakes104105- Running both complete root and MCP test suites for a one-package change106 instead of scoping to the touched paths first — slow, and it buries the107 relevant signal.108- Running MCP tests in the root environment; use `uv run --project mcp` so the109 check consumes the isolated MCP lock and dev extra.110- Skipping `ruff format --check` and pushing unformatted code that fails CI.111- Hand-reformatting unrelated code; only the changed code should move.112- Forgetting that `nat eval` needs `deploy/.env`; run evals via113 `dotenv -f deploy/.env run nat eval ...` (see the matrix reference).114- Inventing npm scripts; stick to the canonical checks `lint`, `type-check`,115 `test:ci`, and `build` defined in `frontends/ui/package.json`.116117## Related Skills118119- `aiq-prepare-pr`120- `aiq-add-tool`121- `aiq-add-data-source`