Debug Protocol

Debug Protocol — Evidence-First Reproduction

A systematic approach to debugging when the expected instrumentation (logs, telemetry, debug servers) fails or is unavailable. This protocol prioritizes using directly observable evidence early and pivoting quickly when a collection strategy stalls.

Core Principles

1. Available evidence is ground truth from round one. Do not delay with parallel collection mechanisms (e.g., trying to set up a debug server, waiting for telemetry) when you already have direct evidence (screenshots, console output, error messages, stack traces). The direct evidence has higher signal-to-noise than speculative instrumentation.

2. Pivot after two failed collection rounds. If a logging mechanism, telemetry pipeline, or debug-server strategy fails to produce output twice in a row, stop and switch to an alternative evidence source immediately. Do not attempt the same collection mechanism a third time. The cost of a third failed round (time spent, additional system state changes) outweighs the benefit of "one more attempt."

3. Each reproduction round is a scarce resource. Running the system again to trigger the bug consumes real time and may change system state. Minimize reproduction rounds by extracting maximum information per run. Plan observation points before each round; don't go in blind hoping to see something useful.

4. Name the failure mode. When a collection strategy fails, don't use vague language ("the logs aren't working"). Name the specific failure: "telemetry batching delay blocking runtime output," "log file permissions preventing write," "debug server timeout on initialization." Naming makes clear what the next alternative should address.

Reproduction Workflow

Round 1: Establish baseline with available evidence

• Identify what you can observe directly without new instrumentation: stdout/stderr, console logs already in memory, screenshots, error dialog text, HTTP response bodies, database query results, file system state, process list, network traffic already captured.
• Extract all ground-truth information from these sources. Document the failure mode and expected vs. actual state in detail.
• If the error is reproducible and you have good signal-to-noise evidence, consider whether additional instrumentation is even necessary before proceeding to round 2.

Round 2: Targeted instrumentation if needed

• Based on round 1, add targeted logging, breakpoints, or telemetry to answer your highest-confidence hypothesis about the cause. Do not add broad instrumentation.
• Run the reproduction. Collect the new evidence.
• If the instrumentation failed (e.g., log file not written, debugger did not break, telemetry endpoint unreachable): evaluate why. Is this a blocker, or can you gather signal from the absence of data?

Pivot decision (critical gate)

• If your primary evidence-collection strategy failed once, attempt it one more time with a fix (different log level, different telemetry endpoint, check file permissions). This is round 2.
• If it failed twice, stop. Do not attempt the same mechanism again. Move to alternative evidence immediately.
• Document what failed and why: "log writes blocked by permissions," "telemetry endpoint timing out," "debugger unable to attach," etc.

Round 3+: Alternative evidence source

• Identify an alternative that answers the same question differently: if logs failed, try trace files or structured output; if telemetry failed, try capturing network traffic or reading memory state; if a debugger failed, try instrumented output to stdout.
• Plan the round with the same rigor as round 2—decide what you're observing and where before running.
• Execute and extract evidence.

Failure Modes and Alternatives

Anti-Patterns

• Re-attempting the same failed collection three times. This is not persistence; it's wasted rounds. After two failures, assume the mechanism is blocked and switch.
• Creating parallel instrumentation while the primary is still running. Parallel mechanisms create noise and compete for resources. Pick one, try it (two rounds max), then switch.
• Waiting for a log aggregation service to flush. If you're blocked on telemetry delays, use real-time evidence instead (stdout, memory dumps, network captures).
• Treating "no log output" as equivalent to "no error." Silence is a separate failure mode—log strategy failed, not the system succeeded. Pivot to alternative evidence.
• Spending more time setting up instrumentation than actual reproduction. If setup costs more than two runs, the evidence you gather is not worth it—use what you can observe directly.

Example Flow

Scenario: Web service crashes under load; the error is intermittent.

1. Round 1 (direct evidence): Capture a crash: screenshot of error dialog, stdout showing stack trace, exit code 1. Note the timing: "crash happens ~30s after load test starts."

2. Round 2 (targeted instrumentation): Add timestamp logging to resource-allocation code. Run load test again. Check log file. Log file not written (maybe due to async buffering or permissions).

3. Pivot decision: That collection strategy failed. Do not re-attempt logging setup. Instead:

4. Round 3 (alternative): Run load test with stdout capturing: script (Unix) or Tee-Object (PowerShell). Pipe real-time output to a file. In-process writes flush immediately. Re-run test. Capture stack trace and resource state at time of crash.

Result: obtained the evidence (resource exhaustion details) in round 3 instead of stalling on logging in round 2.

Documenting the Debug Session

Record for the next person (or your future self):

• What we observed: ground-truth evidence from each round (screenshot, log line, command output).
• What failed: "telemetry endpoint timeout after 5s," not just "logs didn't work."
• Why we pivoted: the explicit gate (two failures, then switch).
• What we learned: "crashes correlate with memory > 80% utilization," not just the bug itself but the signal pattern.
• What we didn't check: boundary conditions, race conditions, other code paths. This helps prioritize the next round if the bug recurs.

Trigger phrases: "debug," "troubleshoot," "bug reproduction," "reproduce the error," "logs not working," "telemetry broken," "intermittent failure," "can't see what's happening."

Related skills: systematic-debugging (deeper adversarial multi-round framework), test-driven-development (preventing bugs before they reach production).