rt-claw CI Coverage
Add the narrowest deterministic check that proves the new behavior is wired into
CI, then broaden only when shared code or platform behavior changed.
When Coverage Is Required
Add or update CI coverage when a change introduces any of these:
- a new service, driver, tool, OSAL backend, endpoint backend, or platform port
- a new feature chain that crosses module boundaries
- a new public API, shell command, Meson/Kconfig option, or Makefile target
- a bug fix where the failure can be reproduced without secrets or hardware
- behavior that must not regress across Linux, QEMU, or supported boards
Do not require real credentials, cloud services, physical hardware, or local
operator input for mandatory PR CI. Cover those paths with mocks, stubs,
compile-only checks, Linux-native tests, QEMU smoke tests, or optional online CI.
Coverage Ladder
Choose the first rung that can express the behavior without flakiness:
- Unit test: pure logic, parsers, state transitions, registries, public helpers.
- Linux functional test: shell commands, KV persistence, process lifecycle.
- Linux option build: Meson options, optional sources, platform helpers.
- QEMU smoke test: boot, shell availability, platform integration.
- Hardware or online test: only optional/manual when hardware, network, or
credentials are required.
Prefer one strong narrow test over a broad slow test that only proves the code
compiled.
Placement
| Change |
Preferred coverage |
| Pure helper or parser |
tests/unit/test_<area>.c |
| Service state machine or registry |
tests/unit/ with stubs where needed |
| Shell command or Linux process behavior |
tests/functional/test_linux_*.py |
| Meson option or optional backend |
scripts/test-meson-matrix.sh or a targeted workflow step |
| QEMU-visible platform behavior |
existing test-smoke-* target |
| Agent skill guidance |
make check-agent-skills |
| Documentation-only behavior |
no CI unless examples are executable |
Workflow
- Identify the smallest behavior that must never regress.
- Identify what CI can run without secrets, hardware, or operator input.
- Add the test in the owning area, not in an unrelated subsystem.
- Wire the test into an existing target before adding a workflow.
- Add a workflow job only when no existing target or matrix covers the new path.
- Keep path filters updated if the workflow must run for new files.
- Run the exact CI command locally when practical.
- Report any coverage intentionally left manual or optional.
Existing CI Entry Points
make test-unit-linux: Linux unit tests; preferred for services and helpers.make test-linux: Linux functional tests after make build-linux.make test-smoke-linux: build plus Linux functional tests.scripts/test-meson-matrix.sh: Meson option compile matrix.make test-smoke-esp32c3, make test-smoke-vexpress, make test-smoke-zynq:
QEMU smoke checks.make check-agent-skills: validate .agents/skills/*/SKILL.md metadata.
Feature Chain Guidance
When a feature crosses services, endpoints, tools, platforms, or external
providers, add a chain-specific subsection using this shape:
## <Feature> Chain Guidance
For <feature> work, split mandatory CI from optional integration:
- Mandatory CI should cover <stable local contracts>.
- Do not call real <external providers or hardware> in required PR CI.
- Do not require <feature-specific secrets, devices, permissions, or boards> in
required PR CI.
- Use <unit / Linux functional / option matrix / QEMU target> to cover
<source wiring and deterministic behavior>.
- Use optional/manual tests for <real provider, hardware, browser, or network
verification>.
Use concrete nouns. Avoid vague entries like "test the chain"; name the public
API, event, parser, shell command, option, state transition, or backend callback
that CI can assert.
Voice Chain Example
For voice work, split mandatory CI from optional integration:
- Mandatory CI should cover endpoint attach/detach, endpoint callbacks, event
validation, shell command wiring, option builds, and buffer/state behavior.
- Do not call real STT/TTS providers in required PR CI.
- Do not require USB microphones, speakers, browser microphone permissions, or
Raspberry Pi hardware in required PR CI.
- Use Linux option builds to cover
voice, linux_web_voice, and
linux_local_voice source wiring.
- Use optional/manual tests for real microphone, speaker, browser, and cloud
provider verification.
Rules
- Do not add a new test framework without explicit request.
- Do not silently skip a test that should be mandatory.
- Do not make mandatory CI depend on secrets, network availability, wall-clock
timing, real hardware, or public cloud APIs.
- Do not add production test hooks unless an existing project pattern already
supports them.
- Do not weaken assertions just to stabilize a flaky test; narrow the scope or
move the check to optional CI instead.
1---2name: rt-claw-ci-coverage3description: Use when adding CI coverage for a new rt-claw module, feature chain, endpoint backend, service integration, or regression.4license: MIT5---6
7# rt-claw CI Coverage
8
9Add the narrowest deterministic check that proves the new behavior is wired into
10CI, then broaden only when shared code or platform behavior changed.
11
12## When Coverage Is Required
13
14Add or update CI coverage when a change introduces any of these:
15
16- a new service, driver, tool, OSAL backend, endpoint backend, or platform port
17- a new feature chain that crosses module boundaries
18- a new public API, shell command, Meson/Kconfig option, or Makefile target
19- a bug fix where the failure can be reproduced without secrets or hardware
20- behavior that must not regress across Linux, QEMU, or supported boards
21
22Do not require real credentials, cloud services, physical hardware, or local
23operator input for mandatory PR CI. Cover those paths with mocks, stubs,
24compile-only checks, Linux-native tests, QEMU smoke tests, or optional online CI.
25
26## Coverage Ladder
27
28Choose the first rung that can express the behavior without flakiness:
29
301. Unit test: pure logic, parsers, state transitions, registries, public helpers.
312. Linux functional test: shell commands, KV persistence, process lifecycle.
323. Linux option build: Meson options, optional sources, platform helpers.
334. QEMU smoke test: boot, shell availability, platform integration.
345. Hardware or online test: only optional/manual when hardware, network, or
35 credentials are required.
36
37Prefer one strong narrow test over a broad slow test that only proves the code
38compiled.
39
40## Placement
41
42| Change | Preferred coverage |
43|--------|--------------------|
44| Pure helper or parser | `tests/unit/test_<area>.c` |
45| Service state machine or registry | `tests/unit/` with stubs where needed |
46| Shell command or Linux process behavior | `tests/functional/test_linux_*.py` |
47| Meson option or optional backend | `scripts/test-meson-matrix.sh` or a targeted workflow step |
48| QEMU-visible platform behavior | existing `test-smoke-*` target |
49| Agent skill guidance | `make check-agent-skills` |
50| Documentation-only behavior | no CI unless examples are executable |
51
52## Workflow
53
541. Identify the smallest behavior that must never regress.
552. Identify what CI can run without secrets, hardware, or operator input.
563. Add the test in the owning area, not in an unrelated subsystem.
574. Wire the test into an existing target before adding a workflow.
585. Add a workflow job only when no existing target or matrix covers the new path.
596. Keep path filters updated if the workflow must run for new files.
607. Run the exact CI command locally when practical.
618. Report any coverage intentionally left manual or optional.
62
63## Existing CI Entry Points
64
65- `make test-unit-linux`: Linux unit tests; preferred for services and helpers.
66- `make test-linux`: Linux functional tests after `make build-linux`.
67- `make test-smoke-linux`: build plus Linux functional tests.
68- `scripts/test-meson-matrix.sh`: Meson option compile matrix.
69- `make test-smoke-esp32c3`, `make test-smoke-vexpress`, `make test-smoke-zynq`:
70 QEMU smoke checks.
71- `make check-agent-skills`: validate `.agents/skills/*/SKILL.md` metadata.
72
73## Feature Chain Guidance
74
75When a feature crosses services, endpoints, tools, platforms, or external
76providers, add a chain-specific subsection using this shape:
77
78```text
79## <Feature> Chain Guidance
80
81For <feature> work, split mandatory CI from optional integration:
82
83- Mandatory CI should cover <stable local contracts>.
84- Do not call real <external providers or hardware> in required PR CI.
85- Do not require <feature-specific secrets, devices, permissions, or boards> in
86 required PR CI.
87- Use <unit / Linux functional / option matrix / QEMU target> to cover
88 <source wiring and deterministic behavior>.
89- Use optional/manual tests for <real provider, hardware, browser, or network
90 verification>.
91```
92
93Use concrete nouns. Avoid vague entries like "test the chain"; name the public
94API, event, parser, shell command, option, state transition, or backend callback
95that CI can assert.
96
97### Voice Chain Example
98
99For voice work, split mandatory CI from optional integration:
100
101- Mandatory CI should cover endpoint attach/detach, endpoint callbacks, event
102 validation, shell command wiring, option builds, and buffer/state behavior.
103- Do not call real STT/TTS providers in required PR CI.
104- Do not require USB microphones, speakers, browser microphone permissions, or
105 Raspberry Pi hardware in required PR CI.
106- Use Linux option builds to cover `voice`, `linux_web_voice`, and
107 `linux_local_voice` source wiring.
108- Use optional/manual tests for real microphone, speaker, browser, and cloud
109 provider verification.
110
111## Rules
112
113- Do not add a new test framework without explicit request.
114- Do not silently skip a test that should be mandatory.
115- Do not make mandatory CI depend on secrets, network availability, wall-clock
116 timing, real hardware, or public cloud APIs.
117- Do not add production test hooks unless an existing project pattern already
118 supports them.
119- Do not weaken assertions just to stabilize a flaky test; narrow the scope or
120 move the check to optional CI instead.
zevorn