Static Source Bug Class Coverage Test

static-source-bug-class-coverage-test

✅ PROMOTED: pattern from a query-fix session. TDD pressure-test passed (RED: relative path + fragile f-string detection; GREEN: Path(__file__).resolve() + triple-quote regex + clean whitelist skip). Polish incorporated: hint about f-string-concatenated SQL.

Pattern (short form)

1. Identify bug class as source pattern: SQL block with X AND without Y, async function without Z, etc.
2. Test loads file via Read (Path(__file__).resolve().parents[N] / "path/to/file.py") — NO relative path
3. Extract pattern candidates via regex over triple-quoted strings or code blocks
4. For each candidate: check fix-marker pattern is contained
5. IMPORTANT — whitelist skip: patterns that get the fix via f-string substitution ({where_clause} templates) skip, otherwise false-positives for already-fixed code
6. Violations list: on mismatch → assert with full snippet list in AssertionError

Limitation: this test only covers triple-quoted SQL strings ("""..."""). SQL strings via f-string concatenation (f"SELECT ... WHERE " + condition) are not captured — for such patterns an integration test against real DB is the more reliable solution.

Concrete example

def test_status_module_no_unfiltered_win_aggregations():
    """Static check: every sum(CASE WHEN win) aggregation in status.py
    must be in a query block whose WHERE contains win IS NOT NULL."""
    from pathlib import Path
    import re

    src_path = (
        Path(__file__).resolve().parents[2]
        / "api/routes/dashboard/modules/status.py"
    )
    src = src_path.read_text(encoding="utf-8")

    # Extract triple-quoted SQL blocks
    sql_blocks = re.findall(r'"""(.*?)"""', src, re.DOTALL)

    violations: list[str] = []
    for block in sql_blocks:
        block_lower = block.lower()
        if "virtual_trades" not in block_lower:
            continue
        if not ("sum(case when" in block_lower and "win" in block_lower):
            continue
        # WHITELIST: skip f-string blocks with externalized where_clause —
        # those are guarded separately by the 4 endpoint mock tests
        if "{where_clause}" in block:
            continue
        if "win is not null" not in block_lower:
            violations.append(block.strip()[:200])

    assert not violations, (
        "Unfiltered win-aggregation(s) found in status.py — bug class "
        "(cockpit distortion) reintroduced:\n\n"
        + "\n---\n".join(violations)
    )

Anti-Patterns

Quick reference: when static-source vs endpoint-mock vs integration test

Real-world impact

Query-fix session: 4 aggregation spots, 1 static-source test covers all plus regression guard for future aggregations. Test found 4 bug spots naively (2 of them false-positive due to f-string). Refinement to 2 real with whitelist skip was ironically a 2-iteration cycle, the skill does that in 1.

Background: TDD progression (Bulletproofing log)

Cycle 1 — PASS

• RED subagent (without skill): also chose static inspection (no endpoint mock) — surprisingly smart. BUT: used relative path Path("api/routes/analytics/metrics.py") (breaks at different cwd), and complex heuristic f-string-variable-definition regex (searches var definition in whole module — fragile with multi-module imports). Honesty caveat self-named.

• GREEN subagent (with skill): Path(__file__).resolve().parents[2] correct, triple-quote regex clean, whitelist skip with explanatory comment. Self-reflection identified skill gap: f-string-concatenated SQL not covered → polish item incorporated.

• Refactor: no R1-R3 needed. Polish item (f-string concatenation limitation note) directly incorporated.

Cycle-2-Backlog (Polish, non-blocking)

1. Glob pattern example for "source pattern across multiple files" case (e.g. glob("api/**/*.py") + iterate)
2. --no-header / format-string adjustment for violations message when snippet is very large