Nw Tdd Methodology Paradigm

Test-Writing Paradigm — Property-Based + State-Delta

Trigger: writing a unit or acceptance test — deciding the test-writing paradigm (PBT-by-default, state-delta over single-assert) and whether delta-first applies to a state-mutating test.

Paradigm Mandate — Property-Based + State-Delta (STANDING, 2026-05-05)

Default test-writing paradigm for UNIT + ACCEPTANCE tests — not optional, not "when applicable".

Test-level applicability matrix

Mandate (unit + acceptance levels)

Every unit and acceptance test you write MUST be:

1. Property-based by default — use Hypothesis @given strategies to explore equivalence classes, NOT single-fixture examples. A property test asserting an invariant over N generated inputs replaces N example tests with stronger semantic coverage.

2. State-delta over single-property assertion — capture the FULL observable state surface (universe), declare the expected delta with predicates (prepended_with, set_to, unchanged, containing, idempotent_after, legacy_healed, normalized_to, appended_with), and call assert_state_delta(before, after, universe, expected, strict=True). strict=True forbids hidden mutations on adjacent slots — this is what catches bugs that pinned-fixture asserts miss.

from hypothesis import given, settings, strategies as st
from nwave_ai.state_delta import assert_state_delta, set_to, unchanged

@given(domain_input=domain_specific_strategy())
@settings(max_examples=100, deadline=None)
def test_pbt_invariant(domain_input):
    before = capture_full_state()
    perform_action(domain_input)
    after = capture_full_state()
    assert_state_delta(
        before, after,
        universe={"slot.a", "slot.b", "slot.c", "slot.d"},
        expected={"slot.a": set_to(expected_from(domain_input)), "slot.b": unchanged()},
        strict=True,
    )

3. Acceptance tests express PROPERTIES of the system — Gherkin scenarios should be framed as Property: <invariant statement> with quantified preconditions ("a set of N tasks with arbitrary timestamps") and invariant outcomes ("monotonically descending by timestamp"), instead of single-example Scenario: blocks. Step definitions internally use @given strategies + state-delta assertions.

OLD pattern (banned by default):

Scenario: Operator sees three tasks ordered by recency
  Given tasks A, B, C with timestamps T1 < T2 < T3
  When the operator runs `prism board`
  Then the board shows: B (T2), C (T3), A (T1) — wait, ordered descending: C, B, A

NEW pattern (required default):

Property: Board column order reflects recency
  Given a set of N tasks with arbitrary timestamps
  When the operator runs `prism board`
  Then the column order is monotonically descending by timestamp
  And no task appears twice in the column
  And every input task appears exactly once in the output

Fallback — when property-framing genuinely cannot express the contract (e.g., flow-specific UI tests with puntual outcomes, golden-file diffs, error messages with exact strings):

• Document WHY property-framing failed in a one-line # bypass: comment
• Compensate by adding stronger PBT at unit/integration level

Forbidden bypass paths (insufficient justification):

• "Single-property is enough" — NO. Always declare the universe, even if only one slot is checked. unchanged() predicate covers the rest.
• "Mock-based interaction test" — NO. Mocks are still observable state; their call-recording surface is part of the universe (mock.call_count, mock.last_call_args).

Exempt categories (still apply paradigm where it adds value, but not mandatory):

• Pure-function tests with single output and no side effects
• Schema/AST validators with single output
• Smoke imports

Goals

Empirical efficacy framework — debt-payoff curve (NOT instantaneous hit rate)

Reframing (Ale 2026-05-05): paradigm efficacy is measured via the debt-payoff curve over a surface's lifetime, not via instantaneous hit rate snapshots.

Three stages

Empirical evidence (2026-05-05 pilot, 2 instances)

Reading the data correctly: the master 0% is NOT failure — it's confirmation that Stage 1 already extracted the debt. Stage 2a on a cured surface validates the surface stays healthy. Stage 2b on an uncured surface re-confirms paradigm yield on debt-accumulated code.

Implication for AI delegation (the deeper point)

Humans accumulate test debt (single-property-asserts, missed-universe-keys, post-state-only). Machines applying the paradigm from day zero (Stage 0) do NOT accumulate debt by construction. Therefore: AI-written code under paradigm enforcement has lower debt-rate than human-written code, given equivalent specification quality.

This is why paradigm-as-default for NEW unit tests (the mandate above) matters more than migration: migrations are one-time cleanup; the long-term value is never accumulating debt to migrate.

The paradigm is the environment modification that lets machines build software with lower debt over time (Ale 2026-05-05).

Delta-First Test Paradigm (state-mutating code)

Scope: ~28% of the test suite (419 test files audited). Applies to installer-class, sync-class, and hook-registration code that mutates user-observable state. NOT universal. Pure-function tests, schema validators, and interaction tests retain standard assertion style.

Trigger — apply delta-first when ALL of these hold

1. The test mutates user-observable state in 2 or more distinct slots (e.g. filesystem path + config key + env setting).
2. OR the test implicitly asserts "preserve X" semantics — i.e., the correctness claim is partly about what did NOT change.
3. The code under test is in scripts/install/, scripts/sync/, or any hook-registration path.

Bypass — do NOT apply delta-first when

• Pure-function tests with a single return value (no side effects).
• Single-property assertion (assert result.returncode == 0).
• validate_prerequisites() failure paths (returncode / boolean / exception only).
• Interaction tests (mock.assert_called_with(...)) — no universe to declare.
• AST / schema / YAML validators.
• Adding 3-5× ceremony for zero additional detection gain vs. a direct assert.

Pattern

from nwave_ai.state_delta import (
    assert_state_delta,
    prepended_with,
    unchanged,
    set_to,
    containing,
)

def test_des_plugin_installs_hook(tmp_path):
    before = capture_state(tmp_path)   # snapshot before action

    plugin.install(context_for(tmp_path))

    after = capture_state(tmp_path)    # snapshot after action

    assert_state_delta(
        before,
        after,
        universe={
            "hooks.pre_tool_use",      # every slot that COULD change
            "hooks.post_tool_use",
            "config.rigor",
        },
        expected={
            "hooks.pre_tool_use": prepended_with("des_hook.py"),
            "hooks.post_tool_use": unchanged(),
            "config.rigor": set_to("standard"),
        },
        # implicit-unchanged: any universe slot NOT in expected must be identical
    )

Full API: assert_state_delta(before, after, universe, expected, *, strict=False).

Available predicate factories: prepended_with, appended_with, unchanged, set_to, containing, normalized_to, idempotent_after, legacy_healed.

Import: from nwave_ai.state_delta import assert_state_delta, <predicates>.

Empirical justification

Migration of 7 installer test files: 4/7 (57%) exposed previously-untracked mutations that post-state-property assertions had missed.

Hidden mutations caught:

• attribution.trailer written silently (test_attribution_plugin) — post-state test only checked returncode.
• content.full transitioned None → str (test_opencode_des_plugin) — old assertion never declared content.full in universe.

Reusable helpers that emerged from migration

• _flatten_config(path) — flattens a JSON/YAML config into dotted key paths.
• _skill_filesystem_state(target_dir, track=) — snapshots skills directory into slot dict.
• _*_content_state(target_dir, name) — snapshots a named agent/command content file.

References

• Source: nwave_ai/state_delta/matcher.py
• Canonical migrated example: tests/installer/unit/plugins/test_attribution_plugin.py
• D-12 hard gate examples: tests/state_delta/integration/test_pilot_bug48.py