Standardizing Rust Tests

• /testing determines the assertion type, execution level, and exception path before implementation
• /standardizing-rust is loaded before this reference
• co-located spec tests use <subject>.<evidence>.<level>[.<runner>].rs or the repo-local overlay
• doubles preserve coupling to the real trait, function, protocol, or binary seam
• property assertions use meaningful proptest or quickcheck properties
• compile-time claims use compile-fail evidence
• shared harnesses, generators, and fixtures live in a separate workspace-member crate (test-infrastructure production code), per the product's test-infrastructure PDR
• coverage claims are measured with the repository's real coverage tool or recorded as unavailable

spx/.../tests/<subject>.<evidence>.<level>[.<runner>].rs

Evidence tokens — the assertion type from /testing:

Level tokens — the infrastructure required to run the test:

Optional runner token — appended after the level token when the test requires a specific async executor or test harness (e.g., tokio, actix).

Examples:

spx/55-example.enabler/21-auth.outcome/tests/session_token.scenario.l1.rs
spx/55-example.enabler/21-auth.outcome/tests/registry_fetch.conformance.l2.rs
spx/55-example.enabler/21-auth.outcome/tests/login_flow.scenario.l3.rs

Product specs or decisions may restrict which levels the product supports. Follow the repo-local Rust test overlay when it points to the governing spec or decision.

Level rules:

• Pure computation, parsing, serialization, config loading, command building, and cheap temp-dir filesystem behavior belong at Level 1
• Real workspace binaries, local DBs, local queues, local HTTP services, and containerized collaborators belong at Level 2
• Remote APIs, deployed systems, SaaS collaborators, browser UI, and shared environments belong at Level 3
• Product specs or decisions may disable Level 3 when the suite cannot safely stand up or isolate those collaborators; repo-local overlays only route skills to those declarations

Preferred controlled implementations:

• small hand-written structs implementing the same trait
• recorder structs that capture inputs for later assertions
• deterministic closures passed into function-based seams
• local harness services that exercise the real protocol

Reject by default:

• mockall, faux, or generated mocks as the primary strategy
• spying on the function or method under test instead of exercising it
• replacing the module under test with a fake implementation

If /testing reaches a Stage 5 exception, the double must still preserve coupling to the real interface or protocol. The exception explains why a controlled implementation is needed; it does not justify severing the seam.

Snapshot tests are valid only when the textual or structured output surface is itself the contract. They are weak evidence for business logic that has a stronger structural assertion available.

Every value in a test has exactly one valid origin. Run through this table for each test value before writing it.

THERE ARE NO VALID TEST-OWNED CONSTANTS. A named constant in a test file that duplicates a value the production module should own means the production code needs refactoring.

1. Source-owned values

ALWAYS import command names, rule names, matcher tokens, status values, domain identifiers, and public constants from the owning production module. If the module does not export them yet, refactor it to export them before writing the test.

// ❌ rejected: duplicates a value the production module should own
const PASS_STATUS: &str = "pass";

// ✅ preferred: import from the production module
use product::audit::GateStatus;
let status = GateStatus::Pass;

2. Generator-produced values

Use generators for inputs that vary per run. A generator is a pure function — it emits values, holds no state, and has no side effects.

• Use proptest or quickcheck strategies for randomized inputs
• Write strategy factories for domain-shaped values

// tests/generators/audit.rs

fn valid_gate_statuses() -> impl Strategy<Value = GateStatus> {
    prop_oneof![
        Just(GateStatus::Pass),
        Just(GateStatus::Fail),
        Just(GateStatus::Skipped),
    ]
}

3. Harness-managed

Use harnesses for tests that interact with external systems — filesystems, APIs, binaries, testcontainers. A harness manages setup and teardown; it is not self-contained.

// product-testing/src/harnesses/spec_tree.rs

pub struct TestEnv {
    pub root: tempfile::TempDir,
}

impl TestEnv {
    pub fn new() -> Self {
        TestEnv { root: tempfile::tempdir().expect("temp dir") }
    }
}

Consumers depend on the workspace-member crate via [dev-dependencies] and import as use product_testing::harnesses::spec_tree::TestEnv;.

4. Fixture files

Use fixture files for real-world data the code under test would encounter: a captured JSONL from a chat session, a saved API response, a document the parser must handle. Fixture files live in the product-testing/ workspace-member crate under product-testing/fixtures/ and are read from disk by path — never compiled in or imported as modules. This is the cross-language test-infrastructure rule.

Strings and numbers are never valid fixtures. A string literal representing a domain value belongs in the production module or a generator, not a static file.

5. Test infrastructure layout

Harnesses, generators, and inert fixtures are production code. They live in a separate workspace-member crate (product-testing/ directory at workspace root, Cargo package product-testing, Rust import path product_testing), declared as a [dev-dependencies] entry of consumers:

• product-testing/src/harnesses/<name>.rs — modules that mediate access to external resources.
• product-testing/src/generators/<name>.rs — factories producing valid inputs for proptest/quickcheck/parameterized tests.
• product-testing/src/fixtures/<name>.rs — fixture-loading code that reads inert data files by path.
• product-testing/fixtures/ (data subdirectory) — inert input files.

Co-located helper modules are acceptable only when the helper serves a single test file. Anything shared across two or more test files belongs in product-testing/. Never use tests/support/, crate::test_support, super::tests, or #[cfg(test)] mod patterns as homes for shared test infrastructure — those keep ungoverned utility code inside production crates or under tests/.

• Do not read production source files as test input to prove behavior

Checked-in Rust script or helper binary entrypoints get thin tests:

• argument parsing through the repository's canonical parser
• dispatch into the imported orchestrator
• exit-code mapping and observable terminal output

The orchestrator carries the main behavioral evidence. Entry files stay small and route to tested modules.

Property claims about parsers, serializers, math, ordering, or invariants require property-based tests unless the spec itself narrows the claim to a finite example set.

Compile-time contracts such as trait bounds, derive behavior, or diagnostic guarantees require trybuild or equivalent compile-fail evidence.

Pure function example:

#[test]
fn rejects_empty_url_sets() {
    let result = validate_config(ConfigInput {
        url_sets: BTreeMap::new(),
    });

    assert!(result.is_err());
}

Dependency seam example:

trait CommandRunner {
    fn run(&self, program: &str, args: &[&str]) -> Result<CommandOutput, CommandError>;
}

struct SuccessRunner;

impl CommandRunner for SuccessRunner {
    fn run(&self, _: &str, _: &[&str]) -> Result<CommandOutput, CommandError> {
        Ok(CommandOutput::success("done"))
    }
}

#[test]
fn command_builder_reports_success() {
    let runner = SuccessRunner;
    let result = sync_repo(Path::new("/src"), "origin", &runner).unwrap();

    assert!(result.success);
}

Tempdir example:

#[test]
fn loads_yaml_from_temp_dir() {
    let dir = tempfile::tempdir().unwrap();
    let path = dir.path().join("config.yaml");
    std::fs::write(&path, "site_dir: ./site\n").unwrap();

    let config = load_config(&path).unwrap();

    assert_eq!(config.site_dir, PathBuf::from("./site"));
}

proptest! {
    #[test]
    fn config_roundtrips(input in valid_config_strategy()) {
        let encoded = encode_config(&input).unwrap();
        let decoded = decode_config(&encoded).unwrap();

        prop_assert_eq!(decoded, input);
    }
}

Use trybuild for compile-time guarantees:

#[test]
fn ui_contracts_hold() {
    let cases = trybuild::TestCases::new();

    cases.pass("tests/ui/valid_builder.rs");
    cases.compile_fail("tests/ui/missing_required_field.rs");
}

CLI binary example:

#[test]
fn init_command_writes_project_files() {
    let temp = tempfile::tempdir().unwrap();

    assert_cmd::Command::cargo_bin("herder")
        .unwrap()
        .current_dir(temp.path())
        .args(["init", "demo"])
        .assert()
        .success();

    assert!(temp.path().join("demo/Cargo.toml").exists());
}

Async L2 example:

#[tokio::test]
async fn repository_persists_and_loads_user() {
    let db = test_database().await;
    let repo = UserRepository::new(db.pool());

    repo.save(&user_fixture()).await.unwrap();
    let loaded = repo.find(UserId::new(1)).await.unwrap();

    assert_eq!(loaded.email(), "user@example.com");
}

Remote API example:

#[tokio::test]
#[ignore = "requires credentialed sandbox"]
async fn published_package_is_fetchable_from_registry() {
    let client = registry_client_from_env().unwrap();

    let package = client.fetch_package("example-package").await.unwrap();

    assert_eq!(package.name, "example-package");
}

Browser workflow example:

#[tokio::test]
#[ignore = "requires browser service and deployed app"]
async fn login_flow_reaches_dashboard() {
    let browser = browser_session_from_env().await.unwrap();

    browser.goto("/login").await.unwrap();
    browser.fill("#email", "user@example.com").await.unwrap();
    browser.click("button[type=submit]").await.unwrap();

    assert!(browser.text("main").await.unwrap().contains("Dashboard"));
}

Level 3 tests must declare their isolation boundary, credentials, cleanup behavior, and expected runtime. If the repository has no safe Level 3 lane, stop and surface that product decision rather than hiding the dependency behind a skipped test.

• Prefer cargo llvm-cov for per-file and per-function coverage deltas
• Compare baseline coverage without the test under audit against coverage with the test included
• Report the actual delta; do not infer coverage from reading the file
• If the repository lacks usable coverage tooling, state that limitation explicitly and do not fabricate a coverage pass

Do not require spx validation literal for Rust tests. The literal validator is TypeScript-only. Enforce source-owned values through review and Rust test standards instead.

• levels/level-1.md - pure computation, tempdir, trait seams, and property tests
• levels/level-2.md - CLI binaries, async adapters, local services, and containerized collaborators
• levels/level-3.md - remote systems, browser flows, credentials, isolation, and cleanup

Source: outcomeeng/plugins — distributed by TomeVault.