Test

Test Skill

Generate real tests, run them, and leave reproducible coverage or TDD evidence. Do not stop at a plan unless the requested mode is strategy.

Critical Constraints

• Why: behavior is the contract. Derive tests from acceptance scenarios and public behavior, not implementation details or coverage percentages alone.
• Why: prove new behavior. In TDD mode record a real failing test before the minimal implementation; a test that starts green is not RED evidence.
• Why: avoid false confidence. Assert exact values, error types/messages, and branch outcomes; ban zero-assertion, tautological, and padding tests.
• Why: keep suites trustworthy. Tests must be deterministic, isolated, and independent of timing, ordering, production services, or mutable shared state.
• Why: protect user intent. Report a product bug discovered by a test; do not silently change product behavior or delete existing tests without approval.
• Why: close with proof. Run the narrow test after each edit, then the relevant suite and coverage command before handing work downstream.

Modes

Default to generate. Flags: --mode, --scope, --min-coverage, and --dry-run narrow the workflow but never weaken its evidence requirements.

Oracle-strength hierarchy

Every test asserts through an oracle, and oracles are not equal. Rank them:

exact value > property/invariant > differential (two implementations agree) > smoke (it ran)

Choose the strongest oracle the behavior admits and name the oracle-strength tier when a test uses anything below exact. A smoke assertion where an exact one was available is the oracle downgrade failure mode: the test runs the code but proves almost nothing about it. Stop condition: no acceptance scenario may be covered only by smoke-tier tests when a stronger oracle is practical; if only smoke is practical (e.g. nondeterministic external output), record why in .agents/tests/summary.md so the gap is a visible decision, not an accident.

Mutation-kill proof

A new test earns trust by failing when the behavior it guards is broken. In tdd mode the recorded RED run is that proof. In other mutating modes, prove at least one kill per new behavioral test: mutate the covered logic (flip the branch, break the boundary value, or use the project's mutation tool), confirm the test fails, then restore. A test that stays green through its own mutation is the immortal test failure mode — delete or strengthen it before handoff; never count it as coverage.

Harness health floors

Green is only evidence when the harness can go red. Before trusting or reporting a green suite, confirm these floors:

• The suite runs to completion — a crashed or truncated run is not a pass.
• Zero-assertion test count did not grow with this change.
• Skipped or excluded tests did not silently increase; new skips are named in the summary with a reason.
• At least one deliberate failure (the mutation-kill or RED run above) failed through the same runner and reporting path you are about to trust.

A suite that cannot demonstrate a failure is the dead harness failure mode: its green is decoration. Report a dead harness as a finding; do not build coverage claims on top of it.

Workflow

1. Bind tests to behavior

When a caller-supplied .feature file has scenarios, work forward from each Given/When/Then. Name one covering test after the behavior, and add @covered-by:<test-path>[::<TestName>] above the scenario. Prove the mapping:

bash scripts/check-scenario-coverage.sh skills/<skill>/references/<name>.feature --run

Without scenarios, inventory public behavior, error paths, branches, and edge cases. Rank gaps by risk: high complexity plus low coverage first.

2. Detect the language and baseline

Stop at the first applicable project marker and consult the Standards skill for it:

Write raw coverage to .agents/tests/coverage-raw.txt, a ranked gap inventory to .agents/tests/gaps.md, and language-native machine output where available.

3. Write the smallest valuable tests

Read the target function and its callers before writing tests. Cover every branch and error return with exact expected results. Use descriptive test names and one behavioral focus per table row or parameter set.

Load specialized guidance only when its trigger applies:

• API, CLI, schema, or compatibility contracts: conformance-harnesses.md
• Parsers, serializers, or hostile input: fuzzing.md
• Generated output or snapshots: golden-artifacts.md
• Invariant-heavy behavior: metamorphic-testing.md
• Real databases, queues, APIs, or services: real-service-e2e.md

For golden updates, follow golden-artifact-strategy.md and review the artifact diff; regeneration alone is not acceptance.

4. Run RED, green, and refactor checks

In tdd mode:

1. Write one behavioral test and run it; require a relevant failure.
2. Implement only enough to pass that test.
3. Refactor under green without changing the test contract.
4. Run the focused test and the relevant suite after each cycle.
5. Append the exact commands and outcomes to .agents/tests/tdd-log.md.

In other mutating modes, run each new test immediately, then the owning package or module, then the relevant project suite. A failure caused by a wrong test is fixed in the test; a product defect is reported explicitly rather than masked.

Checkpoint: before coverage measurement, confirm the focused test and the relevant suite are green and the recorded RED evidence names the intended behavior.

5. Measure and hand off

Re-run the baseline coverage command. Summarize before/after coverage, tests added, remaining high-risk gaps, bugs found, and exact validation commands in .agents/tests/summary.md. Supply that evidence to Validate when the test change accompanies a product slice or is ready for acceptance.

Language Rules

• Go: use <source>_test.go, Test<Uppercase>, table-driven cases, and exact output assertions; never cov*_test.go or *_extra_test.go.
• Python: use pytest fixtures and parametrization; type test helpers.
• JS/TS: group describe/it by public behavior and mock external services, not internal implementation.
• Rust: prefer focused unit tests plus integration tests at public boundaries; keep fixtures deterministic.

Strategy Mode

Inventory test files, functions, assertion density, unit/integration/e2e split, fixtures, and CI wiring. Write .agents/tests/strategy.md with prioritized structural gaps and a test architecture; do not generate code in this mode.

Output Specification

• Artifact directory: .agents/tests/ plus test files in the target's language-native locations.
• Filename convention: coverage-raw.txt, coverage-func.txt or coverage.json, gaps.md, summary.md, tdd-log.md, and strategy.md.
• Serialization/schema format: Markdown evidence reports, native coverage text/profile formats, and JSON where the coverage tool supports it.
• Validator command: run the focused test, relevant suite, coverage command, and bash scripts/check-scenario-coverage.sh ... --run when scenarios exist.
• Downstream use: factual evidence that a caller may supply to Validate.

Quality Rubric

• Every acceptance scenario maps to a passing behavioral test.
• New behavior has authentic RED evidence before implementation.
• Assertions are exact and cover happy, edge, and error paths.
• Tests are deterministic, isolated, fast at the unit layer, and maintainable.
• Coverage changes prioritize risk and never substitute for behavioral proof.
• Artifacts name the commands, results, remaining gaps, and discovered defects.

Examples

Generate mode: inspect a parser, baseline coverage, add table-driven happy, malformed, and empty-input cases, run focused plus package tests, then record the coverage delta and remaining gaps.

TDD mode: write TestParseConfig_MissingName, capture its failing output, add the minimum validation, rerun green, refactor, run the full package, and log the cycle in tdd-log.md.

Troubleshooting

References

• test.feature — executable behavior contract
• conformance-harnesses.md
• fuzzing.md
• golden-artifacts.md
• golden-artifact-strategy.md
• metamorphic-testing.md
• real-service-e2e.md

Source: hashgraph-online/awesome-codex-plugins → plugins/boshu2/agentops/skills-codex/test/SKILL.md