Skill: Deterministic Simulation & Property Testing (Microsoft Research & AWS Rigor)
Example-based tests verify happy paths; deterministic simulation and property oracles prove mathematical invariants and survive real-world chaos.
Why This Skill Is Needed & Target State
Standard automated tests suffer from the "example-based test trap": developers write 2-3 static, hardcoded scenarios that verify the code works under expected, happy-path conditions. In production, distributed race conditions, packet loss, message reordering, clock skews, corrupted state machines, and untrusted malformed inputs cause catastrophic outages that example tests never anticipate.
The target state is formal engineering rigor inspired by Microsoft Research (Coyote, P, Orleans) and AWS/FoundationDB Hermetic Simulation:
- Every domain invariant is validated across thousands of pseudo-random inputs via Property-Based Testing with automatic shrinking.
- Every distributed interaction, network drop, clock skew, and message reordering is simulated deterministically via Discrete Event Simulation with a virtual clock and a reproducible 64-bit seed.
- Every state machine explicitly defines its transition table and makes impossible states unrepresentable in the type system.
- Every parser, deserializer, and boundary entry point is subjected to structure-aware fuzzing to guarantee crash freedom and bounded resource consumption.
Activation and Boundaries
| Situation |
Behavior |
| Authoring or testing distributed systems, consensus, or replication |
Mandate Deterministic Simulation Testing (DST) with virtual clock and seeded fault injection |
| Implementing business logic with conservation, round-trip, or idempotency rules |
Enforce Property-Based Testing (fast-check, Hypothesis, proptest) with shrinking |
| Designing lifecycles, connection management, or stateful workflows |
Construct an explicit State Transition Table; eliminate impossible states via tagged unions |
| Writing or modifying parsers, deserializers, or untrusted boundary decoders |
Require Structure-Aware Fuzzing asserting crash freedom, depth limits, and byte caps |
| PR relying exclusively on static example tests for critical invariants |
Flag as insufficient; demand property tests or seeded simulation proofs |
| Routine UI styling tweaks, copy typos, or non-consequential local scripts |
Do not activate; use standard Anti-Slop Engineering BUILD/FIX gates |
The Four Foundational Pillars
1. Property-Based Testing (PBT) & Shrinking
Static examples provide zero proof of invariant correctness across the domain space. Agents must write property tests that generate thousands of randomized inputs to stress invariants. When a failure occurs, the framework must automatically shrink the failing input to its minimal reproducible counterexample.
2. Deterministic Simulation Testing (DST)
Real wall-clock timeouts (setTimeout, time.sleep) create slow, flaky tests that fail to surface edge-case race conditions. All simulation must use Discrete Event Simulation (DES) driven by an injected virtual clock and a seeded pseudo-random number generator (PRNG). Network drops, latency jitter, clock skew, and message reordering must be 100% reproducible bit-for-bit from the root seed.
3. State-Machine Formal Modeling & Type Safety
Boolean flag bags (isLoading, isError, isSuccess) allow invalid, corrupted state permutations ($2^N$). State machines must make impossible states unrepresentable using tagged unions or algebraic data types. State transitions must be governed by an explicit transition matrix ($State \times Event \to NextState + Effects$) and validated through generative model-based testing.
4. Structure-Aware Fuzzing on Untrusted Boundaries
Dumb byte-flipping fuzzers fail on structured protocols because inputs are rejected at byte 0. All parsers, deserializers, and schema decoders must be tested using grammar-aware, structure-aware fuzz harnesses that generate valid syntax trees with pathological leaf values, deep nesting, and boundary edge cases.
Invariant Rules
- Ban example-based tests as the sole verification of critical invariants. Code that computes money, mutates critical state, or decodes untrusted input must include property-based or fuzz tests.
- Zero real wall-clock sleeps in simulation code.
setTimeout, sleep(), and Date.now() are forbidden in simulation harnesses; all delays must be scheduled through a DeterministicVirtualClock.
- Explicit seeded PRNG for all simulation & property generation. Unseeded
Math.random(), random.random(), or ambient system entropy are prohibited in test execution paths.
- Mandatory shrinking on property failures. Any property testing setup must support automated shrinking to minimal counterexamples.
- No boolean bags for state representation. Replace independent boolean flags with strict tagged unions, ADTs, or sealed traits.
- Explicit state transition completeness. Every state machine must define an exhaustive transition matrix and reject undefined transitions deterministically.
- Crash-freedom on untrusted parsers. Parsers must never panic, segfault, or throw unhandled exceptions when fed fuzzed inputs.
- Bounded recursion depth and payload byte limits. Every parser must enforce explicit recursion counters ($\le 16$ levels) and byte caps ($\le 10\text{MB}$) to eliminate call-stack and allocation bombs.
- Red-green mutation falsification. A property or simulation suite is unverified until a deliberate invariant mutation fails the test and produces an observable counterexample.
Execution Router
| Task Shape |
Primary Workflow |
Reference Specification |
| Distributed system, actor engine, or consensus testing |
Build Deterministic Simulator |
Discrete Event Simulation |
| Mathematical invariants, ledgers, or round-trip codecs |
Author Property Tests |
Property-Based Testing |
| Lifecycle, workflow, or protocol state machine |
Model State Machine |
State-Machine Modeling |
| Untrusted parser, deserializer, or decoder boundary |
Fuzz Parser Boundaries |
Structure-Aware Fuzzing |
| Persistent simulation audit record generation & verification |
Record Contract |
scripts/simulation_guard.py |
Mandatory Pre-Execution Interrogation Matrix
Before writing or approving tests, subject the suite to this matrix:
- Distrust of Examples: Are we testing only 2 or 3 happy-path examples? What happens when input is empty, negative, max-int, contains Unicode null bytes, or is deeply nested?
- Distrust of Time: Does this test rely on
setTimeout, time.sleep(), or real wall-clock delays? How will this test behave on a heavily loaded CI runner with 100% CPU contention?
- Distrust of Flakiness: If this test fails in CI, can an engineer immediately reproduce the exact failure locally using a single command and seed?
- Distrust of State: Can this entity ever be in an invalid state combination? Why are there 4 boolean flags instead of a single tagged union?
- Distrust of Untrusted Input: What happens if an adversary sends a 5MB payload with 10,000 nested brackets? Will the parser stack overflow, OOM the process, or reject it in 5ms?
Prohibited Anti-Patterns (Instant Rejection List)
- Unseeded Randomness in Tests: Using
Math.random() or random.random() without an explicit seed, creating non-reproducible CI failures.
- Wall-Clock Sleeps as Sync Band-Aids: Using
await sleep(500) or setTimeout to wait for background tasks in distributed tests.
- Boolean State Bags: Defining interfaces with multiple mutable boolean flags (
isLoading, isError, isSuccess) allowing illegal permutations.
- Tautological Property Assertions: Asserting
expect(add(a, b)).toBe(a + b) where add is implemented as return a + b, without testing independent algebraic properties.
- Dumb Byte Fuzzing on Grammar Inputs: Throwing raw random bytes at a JSON parser and claiming high fuzz coverage when $99.9%$ of inputs fail at byte 0.
- Disabling Shrinking: Configuring property tests with
{ shrink: false } or ignoring counterexamples.
Required Output & Review Workflow
Structure all simulation, property testing, and fuzzing deliverables through this 3-phase discipline:
- Formal Invariant & Transition Mapping: Define the mathematical invariants, the explicit state transition table, and the failure modes to simulate.
- Hardened Test Implementation: Emit production-grade test suites using virtual clocks, seeded PRNGs, tagged unions, and structure-aware arbitraries.
- Red-Green Mutation Verification: Prove the suite catches real defects by introducing a deliberate mutation and verifying that the harness fails with a minimal counterexample.
When a durable simulation audit record is required, save a JSON assessment following the Record Contract and audit it with:
npx anti-slop-engineering sim audit <record.json> --json
# or: python3 scripts/simulation_guard.py audit <record.json> --source <test_file.ts> --json
Domain References
- Playbook Suite Overview: Central orchestration index for simulation, property testing, and fuzzing.
- Microsoft Research & AWS DST Playbook: FoundationDB hermetic simulation, Coyote actor testing, virtual clocks, and fault matrix.
- State-Machine & Fuzzing Playbook: Formal state matrices, tagged unions, and grammar-based fuzz targets.
- Simulation Booby-Traps: Common traps: unseeded PRNGs, wall-clock leaks, shallow shrinkers, and tautological properties.
- Discrete Event Simulation Contract: Virtual time priority queues, fault injection, and deterministic replay.
- Property-Based Testing Guide: Invariant categories, shrinking mechanics, and framework comparison.
- State-Machine Modeling Guide: Tagged unions, transition tables, and model-based generative testing.
- Structure-Aware Fuzzing Guide: Untrusted parsers, recursion/payload bounds, and crash freedom.
Failure Taxonomy
| Failure Mode |
Observable Signal |
Remediation |
| Example-based complacency |
Only 2 static tests exist for complex calculation |
Write property test generating 1,000+ random inputs with shrinking |
| Flaky race conditions in CI |
Tests fail intermittently under high CI load |
Replace setTimeout with DeterministicVirtualClock and seeded event queue |
| Non-reproducible failure |
Test failed once in CI but passes locally |
Enforce explicit 64-bit seed logged on every run; replay with identical seed |
| Corrupted entity lifecycle |
Entity has isCompleted=true and isFailed=true |
Refactor to tagged union; make impossible states unrepresentable |
| Call-stack exhaustion / DoS |
Deeply nested payload crashes parser with stack overflow |
Enforce recursion depth counter (depth <= 16) and structure-aware fuzz tests |
Delegated Skills
anti-slop-engineering: Follow core BUILD/FIX gates and zero-dependency discipline.
paranoid-principal-architect: Apply 10M concurrent user scale invariants and zero-trust perimeter defenses.
verification-oracle-architect: Implement contract-verified in-memory fakes and dual-adapter testing parity.
Completion and Limits
The task is complete when mathematical invariants are verified across $\ge 1,000$ iterations with automatic shrinking, distributed race conditions are simulated via a deterministic virtual clock with a reproducible seed, impossible states are eliminated by the type system, and untrusted parsers prove crash freedom and resource bounds under structure-aware fuzzing.
1---2name: deterministic-simulation-and-property-testing3description: Enforces Microsoft Research and AWS rigorous testing discipline. Use when testing distributed consensus, stateful workflows, actor concurrency, codecs/parsers, or untrusted boundaries using Property-Based Testing (fast-check, Hypothesis), Deterministic Simulation Testing (DST with virtual clocks and reproducible seeds), formal State-Machine Modeling with impossible states made unrepresentable, and Structure-Aware Fuzzing — even if they don't explicitly say "deterministic simulation". Do NOT use for trivial UI styling tweaks, simple static text changes, or routine non-consequential scripts.4license: MIT5metadata:6 source: https://skillmd.com/skills/immamdouhaboammar/deterministic-simulation-and-property-testing7 version: "0.1.0"8 parent: anti-slop-engineering9---1011# Skill: Deterministic Simulation & Property Testing (Microsoft Research & AWS Rigor)1213> Example-based tests verify happy paths; deterministic simulation and property oracles prove mathematical invariants and survive real-world chaos.1415## Why This Skill Is Needed & Target State1617Standard automated tests suffer from the "example-based test trap": developers write 2-3 static, hardcoded scenarios that verify the code works under expected, happy-path conditions. In production, distributed race conditions, packet loss, message reordering, clock skews, corrupted state machines, and untrusted malformed inputs cause catastrophic outages that example tests never anticipate.1819The target state is **formal engineering rigor inspired by Microsoft Research (Coyote, P, Orleans) and AWS/FoundationDB Hermetic Simulation**:201. Every domain invariant is validated across thousands of pseudo-random inputs via Property-Based Testing with automatic shrinking.212. Every distributed interaction, network drop, clock skew, and message reordering is simulated deterministically via Discrete Event Simulation with a virtual clock and a reproducible 64-bit seed.223. Every state machine explicitly defines its transition table and makes impossible states unrepresentable in the type system.234. Every parser, deserializer, and boundary entry point is subjected to structure-aware fuzzing to guarantee crash freedom and bounded resource consumption.2425---2627## Activation and Boundaries2829| Situation | Behavior |30| --- | --- |31| Authoring or testing distributed systems, consensus, or replication | Mandate Deterministic Simulation Testing (DST) with virtual clock and seeded fault injection |32| Implementing business logic with conservation, round-trip, or idempotency rules | Enforce Property-Based Testing (fast-check, Hypothesis, proptest) with shrinking |33| Designing lifecycles, connection management, or stateful workflows | Construct an explicit State Transition Table; eliminate impossible states via tagged unions |34| Writing or modifying parsers, deserializers, or untrusted boundary decoders | Require Structure-Aware Fuzzing asserting crash freedom, depth limits, and byte caps |35| PR relying exclusively on static example tests for critical invariants | Flag as insufficient; demand property tests or seeded simulation proofs |36| Routine UI styling tweaks, copy typos, or non-consequential local scripts | Do not activate; use standard Anti-Slop Engineering BUILD/FIX gates |3738---3940## The Four Foundational Pillars4142### 1. Property-Based Testing (PBT) & Shrinking43Static examples provide zero proof of invariant correctness across the domain space. Agents must write property tests that generate thousands of randomized inputs to stress invariants. When a failure occurs, the framework must automatically shrink the failing input to its minimal reproducible counterexample.4445### 2. Deterministic Simulation Testing (DST)46Real wall-clock timeouts (`setTimeout`, `time.sleep`) create slow, flaky tests that fail to surface edge-case race conditions. All simulation must use Discrete Event Simulation (DES) driven by an injected virtual clock and a seeded pseudo-random number generator (PRNG). Network drops, latency jitter, clock skew, and message reordering must be 100% reproducible bit-for-bit from the root seed.4748### 3. State-Machine Formal Modeling & Type Safety49Boolean flag bags (`isLoading`, `isError`, `isSuccess`) allow invalid, corrupted state permutations ($2^N$). State machines must make impossible states unrepresentable using tagged unions or algebraic data types. State transitions must be governed by an explicit transition matrix ($State \times Event \to NextState + Effects$) and validated through generative model-based testing.5051### 4. Structure-Aware Fuzzing on Untrusted Boundaries52Dumb byte-flipping fuzzers fail on structured protocols because inputs are rejected at byte 0. All parsers, deserializers, and schema decoders must be tested using grammar-aware, structure-aware fuzz harnesses that generate valid syntax trees with pathological leaf values, deep nesting, and boundary edge cases.5354---5556## Invariant Rules57581. **Ban example-based tests as the sole verification of critical invariants.** Code that computes money, mutates critical state, or decodes untrusted input must include property-based or fuzz tests.592. **Zero real wall-clock sleeps in simulation code.** `setTimeout`, `sleep()`, and `Date.now()` are forbidden in simulation harnesses; all delays must be scheduled through a `DeterministicVirtualClock`.603. **Explicit seeded PRNG for all simulation & property generation.** Unseeded `Math.random()`, `random.random()`, or ambient system entropy are prohibited in test execution paths.614. **Mandatory shrinking on property failures.** Any property testing setup must support automated shrinking to minimal counterexamples.625. **No boolean bags for state representation.** Replace independent boolean flags with strict tagged unions, ADTs, or sealed traits.636. **Explicit state transition completeness.** Every state machine must define an exhaustive transition matrix and reject undefined transitions deterministically.647. **Crash-freedom on untrusted parsers.** Parsers must never panic, segfault, or throw unhandled exceptions when fed fuzzed inputs.658. **Bounded recursion depth and payload byte limits.** Every parser must enforce explicit recursion counters ($\le 16$ levels) and byte caps ($\le 10\text{MB}$) to eliminate call-stack and allocation bombs.669. **Red-green mutation falsification.** A property or simulation suite is unverified until a deliberate invariant mutation fails the test and produces an observable counterexample.6768---6970## Execution Router7172| Task Shape | Primary Workflow | Reference Specification |73| --- | --- | --- |74| Distributed system, actor engine, or consensus testing | [Build Deterministic Simulator](usecases/build-deterministic-simulator.md) | [Discrete Event Simulation](references/discrete-event-simulation-and-fault-injection.md) |75| Mathematical invariants, ledgers, or round-trip codecs | [Author Property Tests](usecases/author-property-tests.md) | [Property-Based Testing](references/property-based-testing-and-shrinking.md) |76| Lifecycle, workflow, or protocol state machine | [Model State Machine](usecases/model-state-machine.md) | [State-Machine Modeling](references/state-machine-modeling-and-type-invariants.md) |77| Untrusted parser, deserializer, or decoder boundary | [Fuzz Parser Boundaries](usecases/fuzz-parser-boundaries.md) | [Structure-Aware Fuzzing](references/structure-aware-fuzzing-and-parsers.md) |78| Persistent simulation audit record generation & verification | [Record Contract](references/simulation-record-contract.md) | `scripts/simulation_guard.py` |7980---8182## Mandatory Pre-Execution Interrogation Matrix8384Before writing or approving tests, subject the suite to this matrix:8586- **Distrust of Examples:** Are we testing only 2 or 3 happy-path examples? What happens when input is empty, negative, max-int, contains Unicode null bytes, or is deeply nested?87- **Distrust of Time:** Does this test rely on `setTimeout`, `time.sleep()`, or real wall-clock delays? How will this test behave on a heavily loaded CI runner with 100% CPU contention?88- **Distrust of Flakiness:** If this test fails in CI, can an engineer immediately reproduce the exact failure locally using a single command and seed?89- **Distrust of State:** Can this entity ever be in an invalid state combination? Why are there 4 boolean flags instead of a single tagged union?90- **Distrust of Untrusted Input:** What happens if an adversary sends a 5MB payload with 10,000 nested brackets? Will the parser stack overflow, OOM the process, or reject it in 5ms?9192---9394## Prohibited Anti-Patterns (Instant Rejection List)95961. **Unseeded Randomness in Tests:** Using `Math.random()` or `random.random()` without an explicit seed, creating non-reproducible CI failures.972. **Wall-Clock Sleeps as Sync Band-Aids:** Using `await sleep(500)` or `setTimeout` to wait for background tasks in distributed tests.983. **Boolean State Bags:** Defining interfaces with multiple mutable boolean flags (`isLoading`, `isError`, `isSuccess`) allowing illegal permutations.994. **Tautological Property Assertions:** Asserting `expect(add(a, b)).toBe(a + b)` where `add` is implemented as `return a + b`, without testing independent algebraic properties.1005. **Dumb Byte Fuzzing on Grammar Inputs:** Throwing raw random bytes at a JSON parser and claiming high fuzz coverage when $99.9\%$ of inputs fail at byte 0.1016. **Disabling Shrinking:** Configuring property tests with `{ shrink: false }` or ignoring counterexamples.102103---104105## Required Output & Review Workflow106107Structure all simulation, property testing, and fuzzing deliverables through this 3-phase discipline:1081091. **Formal Invariant & Transition Mapping:** Define the mathematical invariants, the explicit state transition table, and the failure modes to simulate.1102. **Hardened Test Implementation:** Emit production-grade test suites using virtual clocks, seeded PRNGs, tagged unions, and structure-aware arbitraries.1113. **Red-Green Mutation Verification:** Prove the suite catches real defects by introducing a deliberate mutation and verifying that the harness fails with a minimal counterexample.112113When a durable simulation audit record is required, save a JSON assessment following the [Record Contract](references/simulation-record-contract.md) and audit it with:114```bash115npx anti-slop-engineering sim audit <record.json> --json116# or: python3 scripts/simulation_guard.py audit <record.json> --source <test_file.ts> --json117```118119---120121## Domain References122123- **[Playbook Suite Overview](playbooks/README.md)**: Central orchestration index for simulation, property testing, and fuzzing.124- **[Microsoft Research & AWS DST Playbook](playbooks/microsoft-aws-dst-playbook.md)**: FoundationDB hermetic simulation, Coyote actor testing, virtual clocks, and fault matrix.125- **[State-Machine & Fuzzing Playbook](playbooks/state-machine-and-fuzzing-playbook.md)**: Formal state matrices, tagged unions, and grammar-based fuzz targets.126- **[Simulation Booby-Traps](playbooks/references/simulation-booby-traps.md)**: Common traps: unseeded PRNGs, wall-clock leaks, shallow shrinkers, and tautological properties.127- **[Discrete Event Simulation Contract](references/discrete-event-simulation-and-fault-injection.md)**: Virtual time priority queues, fault injection, and deterministic replay.128- **[Property-Based Testing Guide](references/property-based-testing-and-shrinking.md)**: Invariant categories, shrinking mechanics, and framework comparison.129- **[State-Machine Modeling Guide](references/state-machine-modeling-and-type-invariants.md)**: Tagged unions, transition tables, and model-based generative testing.130- **[Structure-Aware Fuzzing Guide](references/structure-aware-fuzzing-and-parsers.md)**: Untrusted parsers, recursion/payload bounds, and crash freedom.131132---133134## Failure Taxonomy135136| Failure Mode | Observable Signal | Remediation |137| --- | --- | --- |138| Example-based complacency | Only 2 static tests exist for complex calculation | Write property test generating 1,000+ random inputs with shrinking |139| Flaky race conditions in CI | Tests fail intermittently under high CI load | Replace `setTimeout` with `DeterministicVirtualClock` and seeded event queue |140| Non-reproducible failure | Test failed once in CI but passes locally | Enforce explicit 64-bit seed logged on every run; replay with identical seed |141| Corrupted entity lifecycle | Entity has `isCompleted=true` and `isFailed=true` | Refactor to tagged union; make impossible states unrepresentable |142| Call-stack exhaustion / DoS | Deeply nested payload crashes parser with stack overflow | Enforce recursion depth counter (`depth <= 16`) and structure-aware fuzz tests |143144---145146## Delegated Skills147148- `anti-slop-engineering`: Follow core BUILD/FIX gates and zero-dependency discipline.149- `paranoid-principal-architect`: Apply 10M concurrent user scale invariants and zero-trust perimeter defenses.150- `verification-oracle-architect`: Implement contract-verified in-memory fakes and dual-adapter testing parity.151152---153154## Completion and Limits155156The task is complete when mathematical invariants are verified across $\ge 1,000$ iterations with automatic shrinking, distributed race conditions are simulated via a deterministic virtual clock with a reproducible seed, impossible states are eliminated by the type system, and untrusted parsers prove crash freedom and resource bounds under structure-aware fuzzing.