Root Cause Tracing

Root-Cause Tracing

Companion to debugging-and-error-recovery. Use when the symptom is deep, a failure surfaces 10+ layers away from its cause. Linear debugging handles shallow failures; this is the loop-back version.

Core Principle

Trace backward, not forward. Start at the symptom and walk up: what input reached this function? What called it? What data did THAT receive? Log at the boundary, not the middle; one hypothesis per probe; and fix the upstream cause, never the symptom layer.

Iron Laws

When to Use

Failure surfaces 10+ layers from the cause; the linear debugging-and-error-recovery workflow didn't find it; the error message is misleading or generic; the same fix keeps coming back; the failure appears only in production-like environments.

When NOT to Use

The cause is already known (use debugging-and-error-recovery or direct source-first work); the failure is at the entry point (no tracing needed); you have a stack trace pointing to the line (just read the code).

Workflow

1. Start at the symptom layer where the failure surfaces.
2. Walk up one boundary at a time: log the input, log the output, confirm the boundary (see The Backward Trace).
3. Add probes between suspect layers, one hypothesis per probe (Instrumentation Strategy).
4. At the root layer: write a regression test that would have caught the original symptom, fix the upstream invariant, and re-run the trace from symptom to root (When You Find the Root).

The Backward Trace

[Symptom layer: where the failure surfaces]
  ↑ "what input did this function receive?"
  ↑ "what called this function?"
  ↑ "what data did THAT receive?"
  ↑ "where did THAT data come from?"
  ↑ [Root layer: where invalid state originated]

Each ↑ is a step. At each step: log the input, log the output, confirm the boundary.

Instrumentation Strategy

// BAD: log in the middle
function processUser(user) {
  console.log("processing user", user) // noise
  // ...
}

// GOOD: log at the boundary
function processUser(user: User): Result {
  logger.debug("processUser.input", { userId: user.id, ...user })
  // ...
  logger.debug("processUser.output", { result })
  return result
}

Log the boundary, not the body. Body logging creates noise; boundary logging creates a chain of evidence.

When You Find the Root

• Write a regression test that would have caught the original symptom.
• Fix the upstream invariant (a type, a guard, a parse), not the symptom.
• Verify the fix by re-running the trace from symptom to root. No new issues.

Common Mistakes

Tracing forward; adding 10 log lines at once; logging without a hypothesis; fixing the symptom layer; "the stack trace points to X" without verifying X is the cause; tracing 30 minutes without writing the chain; no regression test after.

Red Flags

Hypothesis-free logging; log lines without structure (strings, not objects); tracing forward; "the bug is in X" without evidence; same fix looped; regression test skipped; trace stops at first plausible cause (might be a layer, not the root).

Anti-Patterns

Trace forward; log without hypothesis; fix the symptom layer; "I think X" without distinguishing; no regression test; log strings, not objects; infinite trace without cap.

Verification

• Re-run the trace from symptom to root after the fix: no new issues appear.
• A regression test that would have caught the original symptom exists and passes.
• The chain of boundary logs shows the upstream invariant was the cause, not a plausible intermediate layer.

References

N/A, no reference files; the backward-trace loop is fully specified in this file.