diegosouzapw Scientific Test-Driven Development
Overview
Pragmatic test-driven development for scientific code: write tests first for new features and complex changes, verify with tests for simple bug fixes.
Core principle: Tests before implementation for new behavior, tests verify implementation for known bugs.
Announce at start: "I'm using the scientific-tdd skill to implement this feature."
When to Use This Skill
MUST use for:
- New features or algorithms
- Complex modifications to existing code
- Adding new mathematical models
- Implementing new likelihood functions or state transitions
Can skip test-first for:
- Simple bug fixes where existing tests already cover the behavior
- Documentation changes
- Refactoring with existing comprehensive tests (use safe-refactoring instead)
Process Checklist
Copy to TodoWrite:
Scientific TDD Progress:
- [ ] Understand existing behavior (read code and tests)
- [ ] Write test capturing desired new behavior
- [ ] Run test to confirm RED (fails as expected)
- [ ] Implement minimal code to pass test
- [ ] Run test to confirm GREEN (passes)
- [ ] Run full test suite (check for regressions)
- [ ] Run numerical validation if mathematical code changed
- [ ] Run code-reviewer agent (and/or ux-reviewer when appropriate)
- [ ] Refactor if needed (keep tests green)
- [ ] Commit with descriptive message
Detailed Steps
Step 1: Understand Existing Behavior
Before writing new tests, understand current state:
- Read relevant source files
- Read existing tests for similar functionality
- Run existing tests to see current behavior
- Identify what needs to change
Commands:
# Find relevant tests
pytest --collect-only -q | grep <relevant_term>
# Run specific test file
/Users/edeno/miniconda3/envs/non_local_detector/bin/pytest src/non_local_detector/tests/<test_file>.py -v
Step 2: Write Failing Test (RED)
Write test that captures desired behavior:
Test Structure:
def test_descriptive_name_of_behavior():
"""Test that [specific behavior] works correctly.
This test verifies that [explain what you're testing] when [condition].
"""
# Arrange: Set up test data
input_data = create_test_input()
# Act: Call the function/method
result = function_under_test(input_data)
# Assert: Verify behavior
assert result.shape == expected_shape
assert np.allclose(result.sum(), 1.0, atol=1e-10) # Probabilities sum to 1
For mathematical code, verify:
- Correct output shapes
- Mathematical invariants (probabilities sum to 1, matrices are stochastic)
- Expected numerical values (with appropriate tolerances)
- Edge cases (empty inputs, single element, boundary conditions)
Step 3: Run Test - Confirm RED
CRITICAL: Test MUST fail before implementing:
/Users/edeno/miniconda3/envs/non_local_detector/bin/pytest src/non_local_detector/tests/<test_file>.py::test_name -v
Expected output: Test fails with clear error (function not defined, wrong output, etc.)
If test passes: The test isn't testing new behavior - reconsider what you're testing.
Step 4: Implement Minimal Code
Write simplest code that makes test pass:
- Don't over-engineer
- Don't add features not tested
- Follow YAGNI (You Aren't Gonna Need It)
- Use existing patterns from codebase
For scientific code:
- Maintain numerical stability
- Use JAX operations where appropriate
- Follow existing conventions for shapes and broadcasting
Step 5: Run Test - Confirm GREEN
/Users/edeno/miniconda3/envs/non_local_detector/bin/pytest src/non_local_detector/tests/<test_file>.py::test_name -v
Expected output: Test passes
If test fails: Debug until it passes, then verify you're testing the right thing.
Step 6: Run Full Test Suite
Check for regressions:
/Users/edeno/miniconda3/envs/non_local_detector/bin/pytest -v
Expected: All tests pass (same count as before)
If new failures: Your change broke something - fix before proceeding.
Step 7: Numerical Validation (if applicable)
If you modified mathematical/algorithmic code:
Use numerical-validation skill:
@numerical-validation
This verifies:
- Mathematical invariants still hold
- Property-based tests pass
- Golden regression tests pass
- No unexpected numerical differences
Step 8: Refactor (optional)
If code can be improved while keeping tests green:
- Improve readability
- Extract reusable functions
- Optimize performance (but verify numerics don't change)
After each refactor:
/Users/edeno/miniconda3/envs/non_local_detector/bin/pytest -v
Step 9: Commit
git add <test_file> <implementation_file>
git commit -m "feat: add <feature description>
- Add test for <specific behavior>
- Implement <what you implemented>
- All tests passing (<N> tests)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>"
Example Workflow
Task: Add new random walk transition with custom variance
1. Read: src/non_local_detector/continuous_state_transitions.py
2. Read: src/non_local_detector/tests/transitions/test_continuous_transitions.py
3. Write test: test_random_walk_custom_variance()
4. Run test: FAIL - "NotImplementedError: custom variance not supported"
5. Implement: Add variance parameter to RandomWalk class
6. Run test: PASS
7. Run full suite: 427 tests passed
8. Run numerical validation: All invariants hold
9. Commit: "feat: add custom variance support to RandomWalk"
Integration with Other Skills
- Before using this skill: Often preceded by brainstorming or design discussion
- Use with numerical-validation: For mathematical code changes
- After this skill: May use safe-refactoring for cleanup
- Alternative to this skill: Use safe-refactoring if changing structure, not behavior
Red Flags
Don't:
- Write implementation before test (except for documented bug fixes)
- Skip running test to see it fail
- Add untested code "for future use"
- Skip full test suite after implementation
- Commit failing tests
- Skip numerical validation for mathematical code
Do:
- Write descriptive test names
- Test one behavior per test
- Use appropriate numerical tolerances (1e-10 for probabilities)
- Run tests frequently
- Commit small, working increments
- Ask if unsure whether to use TDD for a specific change