Stepwise Testing

Stepwise Testing

Exhaustive verification approach that validates every step, line of execution, and component - not just end-to-end outcomes. When debugging, instrument each step; when testing, verify every intermediate state; never assume, always verify.

When to use me

Use stepwise testing when:

• Debugging complex failures and need to trace exactly where things break
• Implementing multi-step workflows where each step must be verified
• Testing critical path scenarios in production code
• Investigating intermittent bugs that require exhaustive tracing
• Building complex features where components interact
• Establishing test coverage for new codebases

What I do

1. Debug Instrumentation

• Add print/log statements at every decision point, function entry/exit, and state change
• Trace data flow through the entire call stack
• Log variable states before/after each transformation
• Capture intermediate results for comparison

2. Stepwise Verification

• Test each function individually with edge cases and boundary conditions
• Verify data transformations at each step, not just final output
• Ensure state transitions only occur under valid conditions
• Validate side effects (file IO, network calls, memory operations)

3. Component Isolation

• Break complex flows into testable units
• Mock dependencies to verify interface contracts
• Test error paths and failure scenarios for each component
• Verify resource lifecycle (cleanup, leaks, handles)

4. Exhaustive Coverage

• Don't skip steps just because they "should" work
• Verify assumptions about third-party libraries/APIs
• Test concurrency and race conditions where applicable
• Validate error handling and exception recovery

Examples

Debugging with Stepwise Tracing

Before (missing details):

def process_user(data):
    validated = validate(data)
    result = transform(validated)
    save(result)
    return result

After (stepwise instrumentation):

def process_user(data):
    print(f"[ENTER] process_user with data: {data}")
    
    # Step 1: Validation
    print("[STEP] Starting validation...")
    validated = validate(data)
    print(f"[AFTER] validate returned: {validated}, keys: {validated.keys()}")
    
    # Step 2: Transformation
    print("[STEP] Starting transformation...")
    result = transform(validated)
    print(f"[AFTER] transform returned: {result}")
    
    # Step 3: Persistence
    print("[STEP] About to save result...")
    save(result)
    print(f"[SUCCESS] Saved result")
    
    print(f"[EXIT] process_user returning: {result}")
    return result

Testing Each Step Individually

Don't just test the final output:

# Bad - only tests final result
def test_process_user():
    result = process_user({"name": "test"})
    assert result["status"] == "success"

Test each intermediate step:

# Good - tests each transformation
def test_validate_step():
    invalid_data = {"name": ""}
    result = validate(invalid_data)
    assert result is None or get_error(result) == "name_required"
    
def test_transform_step():
    validated = {"name": "test", "email": "test@example.com"}
    result = transform(validated)
    assert result is not None
    assert "status" in result
    assert "processed_at" in result  # Timestamp added
    
def test_save_step():
    result = {"name": "test", "email": "test@example.com"}
    saved_id = save(result)
    assert isinstance(saved_id, str)
    # Verify actually saved
    retrieved = retrieve(saved_id)
    assert retrieved == result

Verifying Complex Multi-Step Workflows

# Example: Payment processing flow
# Step 1: Validate payment method
print("[1/6] Validating payment method...")
payment_method = validate_payment_method(card_data)
print(f"    Result: {payment_method.get('status', 'invalid')}")
assert payment_method["status"] == "valid", f"Invalid: {payment_method}"

# Step 2: Check balance/funds
print("[2/6] Checking available balance...")
balance = check_balance(payment_method["account_id"])
print(f"    Balance: {balance}/ Required: {amount}")
assert balance >= amount, f"Insufficient funds"

# Step 3: Hold funds
print("[3/6] Placing funds on hold...")
hold_id = place_hold(payment_method["account_id"], amount)
print(f"    Hold ID: {hold_id}")
assert hold_id is not None

# Step 4: Execute transfer
print("[4/6] Executing transfer...")
transfer = execute_transfer(hold_id, recipient_account)
print(f"    Transfer ID: {transfer['id']}")
assert transfer["status"] == "pending"

# Step 5: Verify receipt
print("[5/6] Verifying recipient received funds...")
recipient_check = check_transaction(transfer["id"])
print(f"    Recipient status: {recipient_check['status']}")
assert recipient_check["status"] == "completed"

# Step 6: Finalize
print("[6/6] Finalizing and releasing hold...")
finalize(transfer["id"])
print("    ✅ Complete")

Network Request Verification

Don't assume a library works:

# Don't just use requests library
response = requests.get(url)
assert response.ok

# Verify each step:
print("[1/3] Testing DNS resolution...")
import socket
try:
    ip = socket.gethostbyname(urlparse(url).hostname)
    print(f"    Resolved to: {ip}")
except socket.gaierror as e:
    print(f"    ❌ DNS failed: {e}")
    raise

print("[2/3] Testing TCP connection...")
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(5)
try:
    sock.connect((ip, 443))
    print(f"    ✅ Connected")
except Exception as e:
    print(f"    ❌ Connection failed: {e}")
    raise
finally:
    sock.close()

print("[3/3] Testing HTTP handshake...")
response = requests.get(url, timeout=10)
print(f"    Status: {response.status_code}")
print(f"    Headers: {dict(response.headers)}")
print(f"    Body length: {len(response.text)}")
assert response.ok, f"HTTP error: {response.status_code}"

Output format

Stepwise testing produces detailed, line-by-line output:

[STEP 1/5] Validating input parameters...
    Input: {'user_id': 123, 'amount': 50.0}
    ✅ All required fields present
    ✅ Data types valid

[STEP 2/5] Checking user permissions...
    User role: 'standard'
    Permission check: 'can_withdraw' -> False
    ❌ Permission denied
    [FAIL] User 123 cannot withdraw permissions

Debugging Patterns

Pattern 1: Function Entry/Exit

def function_name(arg1, arg2):
    print(f"[ENTER] function_name: arg1={arg1}, arg2={arg2}")
    try:
        result = # ... work ...
        print(f"[EXIT] function_name -> {result}")
        return result
    except Exception as e:
        print(f"[ERROR] function_name: {e}")
        raise

Pattern 2: Loop Instrumentation

for i, item in enumerate(items):
    print(f"[LOOP] Iteration {i}/{len(items)}: item={item}")
    result = process(item)
    print(f"[LOOP] Result: {result}")

Pattern 3: State Changes

print(f"[STATE] Before: counter={counter}")
counter += 1
print(f"[STATE] After: counter={counter}")

Pattern 4: Conditional Branches

if condition:
    print(f"[BRANCH] Taking 'if' path: condition={condition}")
    # ... work ...
else:
    print(f"[BRANCH] Taking 'else' path: condition={condition}")
    # ... work ...

Testing Patterns

Pattern 1: Unit Test Each Function

def test_validate():
    # Valid input
    assert validate({"name": "test"}) is not None
    # Missing field
    assert validate({"missing": True}) is None
    # Invalid type
    assert validate({"name": []}) is None

def test_transform():
    # Normal case
    assert transform({"name": "test"}) is not None
    # Test data transformation
    result = transform({"name": "test"})
    assert "processed_at" in result

Pattern 2: Verify Assumptions

def test_third_party_behavior():
    """Don't assume the library behaves as documented"""
    response = library.action(input)
    # Verify the actual behavior
    assert response.structure == expected
    assert response.timing < threshold

Pattern 3: Edge Cases

def test_edge_cases():
    # Empty input
    assert transform({}) == expected_empty
    # Very large input
    assert transform(large_data) == expected_large
    # Unicode/special chars
    assert transform({"name": "🚀"}) == expected_unicode

Implementation Guidelines

1. Add Instrumentation First

Before adding complex logic, instrument the simple version:

def simple():
    print("[START]")
    print("[STEP 1] ...")
    print("[STEP 2] ...")
    print("[DONE]")

2. Test Each Component

Never combine untested components. Test separately first:

# Test A alone
result_a = component_a()
# Test B alone
result_b = component_b(result_a)
# Only then integrate
integrated = integrate(result_b)

3. Verify External Dependencies

Don't trust third-party code:

# Verify it actually works
test_result = external_library.call(test_input)
assert test_result == expected_external_behavior

4. Capture Evidence

Save intermediate states for inspection:

debug_states = []
for step in workflow:
    state_before = copy.deepcopy(current_state)
    result = execute(step)
    state_after = copy.deepcopy(current_state)
    debug_states.append({
        "step": step,
        "before": state_before,
        "result": result,
        "after": state_after
    })

Notes

• Stepwise testing takes time but saves debugging time later
• Instrument everywhere - you can remove it after debugging
• Test assumptions about libraries/frameworks - don't assume documentation is accurate
• Log both success and failure - knowing where it succeeded is as important as where it failed
• Keep it granular - one assertion/verification per line where practical
• Version control the instrumentation - you can comment it out but keep it for future debugging
• Use debug levels - ERROR, WARN, INFO, DEBUG, TRACE for different detail levels