Property Based Testing

Property-Based Testing

Testing that properties hold for arbitrary inputs generated by the framework.

Context

You are designing property-based tests. Describe a property (invariant) and let the framework generate hundreds of test cases.

Domain Context

• Properties: "Sorting returns elements in ascending order" (invariant)
• Generators: Framework generates random valid inputs
• Shrinking: When a property fails, framework finds minimal failing case
• Coverage: Catches edge cases humans wouldn't think of
• Complementary: Use with unit tests, not replacement

Instructions

1. Identify Invariants: What property must always be true? Idempotence? Commutativity?
2. Choose Generators: What valid inputs should be tested? Numbers, strings, lists?
3. Write Property: Assert the invariant; let framework vary inputs
4. Run Generator: Property-based test runner generates cases, looks for failures
5. Shrink Failures: When property fails, framework finds minimal case
6. Document Examples: Add unit tests for important examples alongside property tests

Anti-Patterns

• Testing trivial properties (identity, basic I/O); waste of PBT
• Generators that are too narrow; should cover wide range of inputs
• Ignoring shrunk failures; use them to understand the bug
• Combining PBT with side effects; test pure functions
• No complementary unit tests; properties are great but unit tests provide examples

Further Reading

• John Hughes, QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs
• Hypothesis documentation (Python library for PBT)