# Rtl P5s Func Verify Policy

> Internal reference: rtl p5s func verify policy (agent-loaded; do not invoke).

- Skill: `babyworm/rtl-p5s-func-verify-policy` (Agent Skill)
- Install (CLI): `npx skillmds@latest add babyworm/rtl-p5s-func-verify-policy`
- Raw SKILL.md: https://api.skillmd.com/api/skills/babyworm/rtl-p5s-func-verify-policy/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: babyworm (https://skillmd.com/u/babyworm)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/babyworm/rtl-p5s-func-verify-policy

---


# Tier 3 Functional Verification Policy

## Testing Tier Context

```
Tier 1: Smoke Test     — connectivity, R/W, basic ops (rtl-p4-implement Wave 4)
Tier 2: Unit Test      — reference comparison, uarch features (rtl-p4s-unit-test)
Tier 3: Module Regr.   — cocotb multi-seed (THIS POLICY)
Tier 4: Integration    — cross-module, end-to-end (rtl-p5s-integration-test)
```

Tier transition rules:
- Tier 2 PASS (rtl-p4s-unit-test) → Tier 3 eligible
- Tier 3 PASS or PARTIAL_PASS (this skill) → Tier 4 eligible (rtl-p5s-integration-test)
  (PARTIAL_PASS = some Critical/High ac_ids PARTIAL; WARNING at Stage 1, escalated to FAIL at Stage 3)
- Tier 3 FAIL → fix via rtl-p4s-bugfix, re-run Tier 2 then Tier 3

## cocotb Signal Naming Convention

cocotb test files MUST use correct signal names matching RTL port conventions:
- Signal access: `dut.i_data` (NOT `dut.data_i`), `dut.o_valid` (NOT `dut.valid_o`)
- Clock: `dut.clk` (single domain) or `dut.sys_clk` (multiple domains) — NOT `dut.clk_i`
- Reset: `dut.rst_n` (single domain) or `dut.sys_rst_n` (multiple domains) — NOT `dut.rst_ni`
- cocotb clock utility: `cocotb.clock.Clock(dut.sys_clk, 10, units="ns")`
- Reset sequence: drive `dut.sys_rst_n.value = 0`, wait, then `dut.sys_rst_n.value = 1`

## Multi-Seed Strategy

- Default 5 seeds: 1, 42, 123, 1337, 65536 (configurable via `sim/regression/seed_list.txt`)
- Each seed tests different random stimulus ordering
- A module passes multi-seed regression only when ALL seeds pass
- Use `COCOTB_RESOLVE_X=RANDOM` for X-state handling
- Use `RANDOM_SEED={seed}` for reproducibility

## Execution Strategy

- **Local-first runtime**: default to local execution on current host (`--mode local`)
- **Default parallel budget**: use `max(1, nproc-2)` unless user explicitly overrides `--parallel`
- **AWS usage policy**: `aws-batch` is allowed only when the user explicitly asks to use AWS
  and explicit gate/runner wiring exists (`RTL_ALLOW_AWS=1`, `RTL_AWS_BATCH_RUNNER`)
- **Pipelined**: as each module's TB completes → immediately launch sim (don't wait for all TBs)
- **Module-level parallelism**: each module's TB + sim runs as an independent parallel task
- **Multi-seed parallelism**: queue 5 seeds × N modules; local active workers stay within `max(1, nproc-2)`
- **Incremental coverage**: coverage-analyst starts partial analysis on completed modules
- **Early termination**: >5% failure rate across seeds → halt and report immediately

## Coverage Targets

| Metric | Target |
|--------|--------|
| Line coverage | ≥ 90% |
| Toggle coverage | ≥ 80% |
| FSM coverage | ≥ 70% |

Below target: testbench-dev generates additional tests → re-run regression.

## Coverage Collection

```bash
# Verilator compilation with coverage
make -C sim/{module} sim SIM=verilator EXTRA_ARGS="--coverage --trace-fst" TOPLEVEL=dut MODULE=test_dut

# Merge multi-seed coverage data
verilator_coverage --write-info merged.info seed_*/coverage.dat

# Coverage HTML report
genhtml sim/coverage/merged.info -o sim/coverage/html/ --title "Regression Coverage"
```

## Regression Scripts

```bash
# Automated multi-seed regression ({plugin_root} = plugin root resolved from .rat/state/spawn-context.json)
bash {plugin_root}/skills/rtl-p5s-func-verify/scripts/run_regression.sh \
  --mode local --seeds "1 42 123 1337 65536" --sim verilator

# Optional override when user explicitly asks
bash {plugin_root}/skills/rtl-p5s-func-verify/scripts/run_regression.sh \
  --mode local --parallel "$(($(nproc)-2))" --seeds "1 42 123 1337 65536" --sim verilator

# Coverage merge
bash {plugin_root}/skills/rtl-p5s-func-verify/scripts/merge_coverage.sh \
  --format verilator --output sim/coverage/merged.info
```

## Requirement Traceability Matrix Format

Save to `reviews/phase-5-verify/requirement-traceability.md`:

### AC-Level Format (when structured acceptance_criteria with ac_id exist in iron-requirements)

When `iron-requirements.json` contains structured `acceptance_criteria` entries (object arrays with `ac_id`
fields), the RTM uses AC-level granularity:

```markdown
# Phase 5 Review: Requirement Traceability
- Date: YYYY-MM-DD
- Reviewer: func-verifier
- Upper Spec: iron-requirements.json
- Verdict: PASS | PARTIAL_PASS | FAIL

## Feature Coverage Checklist
| REQ ID | AC ID | Description | Test Case | Status |
|--------|-------|-------------|-----------|--------|

## Findings
### [severity] Finding-N: ...

## Verdict
PASS | PARTIAL_PASS | FAIL: [reason]
- PASS: all Critical/High ac_ids VERIFIED or FORMAL
- PARTIAL_PASS: some Critical/High ac_ids PARTIAL (WARNING at Stage 1, escalated to FAIL at Stage 3)
- FAIL: Critical/High ac_ids UNTESTED or tests failing
```

AC-level status values per criterion:
- `VERIFIED` — AC covered by a passing test
- `FORMAL` — AC proved by formal verification
- `PARTIAL` — AC partially covered (some test cases pass, scope limited)
- `UNTESTED` — AC exists but no test covers it
- `NOT_VERIFIABLE` — AC has `verifiable: false` (inspection-only); document in RTM, excluded from
  automated coverage tracking

VERIFIED judgment is made at the individual criterion level when `ac_id` fields are present.

### REQ-Level Format (backward compatible — when no structured AC)

When `acceptance_criteria` is absent or contains a plain string array (P1/P2 format), the RTM uses
the existing REQ-level format:

```markdown
## Feature Coverage Checklist
| REQ ID | Test Name | Result | Status |
|--------|-----------|--------|--------|
```

Verdict rules (Tier 3 module-level, aligns with Stage 1 2-tier model):
- `PASS` — all requirements (or acceptance criteria) verified with passing tests
- `PARTIAL_PASS` — some Critical/High ac_ids are PARTIAL (not yet VERIFIED/FORMAL).
  At Stage 1 module graduation: WARNING (proceed). At Stage 3 final audit: escalated to FAIL.
- `FAIL` — M requirements/criteria UNTESTED, K with failing tests
- Any REQ (or AC) UNTESTED → testbench-dev must generate additional tests
- PARTIAL Critical/High ac_ids → WARNING at this stage, must be resolved before Stage 3

## cocotb Ecosystem Quick Reference

- **cocotb-bus**: Base classes for Driver, Monitor, and Scoreboard
- **cocotbext-axi**: Ready-to-use AXI4/AXI4-Lite/AXI4-Stream masters and slaves
  - Example: `AxiLiteMaster(AxiLiteBus.from_prefix(dut, "s_axi"), dut.sys_clk, dut.sys_rst_n, reset_active_level=False)`
  - Stream: `AxiStreamSource(AxiStreamBus.from_prefix(dut, "s_axis"), dut.sys_clk, dut.sys_rst_n)`
- **cocotb-coverage**: Functional coverage with `@CoverPoint` and `@CoverCross` decorators
  - Example: `@CoverPoint("top.data", bins=[range(0,64), range(64,256)])`
- **TestFactory**: Parameterized test generation
  - Example: `TestFactory(run_test).add_option("width", [8,16,32]).generate_tests()`

See `{plugin_root}/skills/rtl-p5s-func-verify/references/cocotb-ecosystem.md` for complete API reference.

## Tier 2 Baseline Utilization

When Tier 2 unit test results (`sim/{module}/{module}_unit_results.json`) are available
from Phase 4, the CDTG pipeline MUST operate incrementally:

1. **Load baseline**: Read Tier 2 coverage metrics (line_pct, fsm_pct, toggle_pct)
   and already-covered features from unit_results.json
2. **Prioritize gaps**: CDTG Round 1 focuses on uncovered FSM states, untested code
   paths, and features not exercised in Tier 2
3. **Avoid duplication**: Do not regenerate test vectors that duplicate Tier 2 coverage.
   Extend coverage, not repeat it
4. **Graceful degradation**: If Tier 2 results are absent (e.g., module skipped P4 unit
   testing), proceed from zero baseline. Log warning but do not block

Coverage targets remain unchanged: Line ≥ 90%, Toggle ≥ 80%, FSM ≥ 70%.
The baseline only affects CDTG prioritization, not target thresholds.

## Backward Traceability Policy

Test failure reports MUST include requirement impact analysis:
- Every failed test must list its affected req_ids and ac_ids (from `# Covers:` comments)
- Failed tests without coverage comments are flagged as UNTRACEABLE
- Failure Impact Summary table is mandatory in regression reports
- Priority classification: Critical/High → BLOCKING, Medium/Low → WARNING, unmapped → UNTRACEABLE

This policy ensures that test failures are immediately connected to requirements,
enabling prioritized debugging and requirement-level risk assessment.

## Escalation & Stop Conditions

- cocotb not installed → halt and route setup through `/rtl-agent-team:rat-setup`
- Failure persists after 2 RTL fix rounds → escalate to rtl-architect with waveform analysis
- Coverage below 80% after full regression → invoke rtl-p5s-coverage-analyze skill
- cocotb signal name mismatch error → testbench-dev must fix to use `i_`/`o_` convention
- Requirements with NO TEST COVERAGE after additional test generation → escalate to user
- Requirement traceability verdict FAIL with persistent test failures → escalate to rtl-p4-implement

## Final Checklist

- [ ] All cocotb tests use correct signal names (`dut.i_*`, `dut.o_*`, `dut.sys_clk`, `dut.sys_rst_n`)
- [ ] All test vectors run to completion
- [ ] RTL vs ref model comparison done per vector
- [ ] Waveform analysis done for all failures
- [ ] sim/coverage/coverage.xml generated
- [ ] sim/regression/*_result.json written per test
- [ ] Multi-seed regression passed (5 seeds per module: 1, 42, 123, 1337, 65536)
- [ ] Per-module pipelined execution used (TB → sim without waiting for all TBs)
- [ ] Coverage merged across seeds (sim/coverage/merged.info or sim/coverage/coverage.xml)
- [ ] Coverage targets met: line ≥ 90%, toggle ≥ 80%, FSM ≥ 70%
- [ ] sim/regression/seed_{seed}_results.json written per seed
- [ ] Early termination applied if failure rate >5%
- [ ] Requirement Traceability Matrix produced (AC-level when structured acceptance_criteria exist, REQ-level otherwise)
- [ ] Every REQ-NNN in `docs/phase-1-research/iron-requirements.json` (canonical) or legacy `docs/phase-1-research/requirements.json` (fallback) covered by at least one test
- [ ] When structured AC exists: every Critical/High ac_id is VERIFIED, FORMAL, or PARTIAL (PARTIAL = PARTIAL_PASS verdict at Stage 1, escalated to FAIL at Stage 3 final audit). UNTESTED = FAIL.
- [ ] All covered requirements pass their tests (or failures escalated)
- [ ] Traceability verdict is PASS or PARTIAL_PASS (PARTIAL_PASS = WARNING at Stage 1, escalated to FAIL at Stage 3)
- [ ] reviews/phase-5-verify/requirement-traceability.md saved