Property-Based Tester
Implements property-based testing frameworks that generate random test inputs and verify properties hold, finding edge cases that manual testing misses.
When to Use This Skill
- Testing algebraic laws (monoid, functor, monad)
- Verifying compiler transformations preserve semantics
- Finding bugs in data structures
- Fuzzing APIs with type-aware generation
What This Skill Does
- Generators: Builds random generators for types via combinators
- Property Definition: Defines properties as functions returning Boolean
- Shrinking: Implements minimal counterexample finding
- Statistics: Reports distribution of test inputs
Key Concepts
| Concept | Description |
|---|---|
| Generator | Produces random values of a type |
| Shrinker | Reduces failing inputs to minimal examples |
| Property | Invariant that should hold for all inputs |
| Coverage | Distribution of generated inputs |
Tips
- Write properties that should always hold, not just examples
- Use shrinking to get minimal failing cases
- Cover edge cases: empty, single element, large inputs
- Combine with type-based generation for better coverage
Related Skills
smt-solver-interface- Symbolic verificationfuzzer-generator- Random testingtype-inference-engine- Type-aware generation
Canonical References
| Reference | Why It Matters |
|---|---|
| Claessen & Hughes, "QuickCheck: Lightweight Automatic Testing" (ICFP 2000) | Original Haskell property-based testing library |
| MacIver, "Hypothesis: Property-Based Testing in Python" (2015+) | Python ecosystem standard with sophisticated coverage guidance |
| Claessen & Hughes, "Testing the Hard Stuff and Staying Sane" (ICFP 2002) | Advanced random testing techniques for critical software |
Tradeoffs and Limitations
| Approach | Pros | Cons |
|---|---|---|
| Random generation | Finds edge cases | May miss specific bugs |
| Enumeration | Complete coverage | State space explosion |
| Symbolic execution | Targeted testing | Slower, complex |
Assessment Criteria
| Criterion | What to Look For |
|---|---|
| Generator coverage | All interesting cases covered |
| Shrinking effectiveness | Minimal failing examples |
| Property correctness | Properties capture true invariants |
Quality Indicators
✅ Good: Good coverage, effective shrinking, correct properties ⚠️ Warning: Limited coverage, slow shrinking ❌ Bad: Wrong properties, no shrinking
Research Tools & Artifacts
Property-based testing frameworks:
| Tool | Language | What to Learn |
|---|---|---|
| QuickCheck | Haskell | Original PBT |
| Hypothesis | Python | Python standard |
| Fast-Check | TypeScript | Composable |
| ScalaCheck | Scala | Integration |
| Erlang QuickCheck | Erlang | Industrial |
Research Frontiers
1. Stateful Testing
- Goal: Test stateful systems
- Papers: "QuickCheck Testing for Stateful Software"
- Tools: StateMachines, sequences
Implementation Pitfalls
| Pitfall | Real Consequence | Solution |
|---|---|---|
| Weak properties | False positives | Strengthen invariants |
Converted and distributed by TomeVault — claim your Tome and manage your conversions.