# Systematic Debugging > Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes - Skill: `tomevault-io/systematic-debugging-7` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add tomevault-io/systematic-debugging-7` - Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/systematic-debugging-7/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: tomevault-io (https://skillmd.com/u/tomevault-io) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/tomevault-io/systematic-debugging-7 --- # Systematic Debugging Random fixes waste time and create new bugs. Quick patches mask underlying issues. **Core principle:** ALWAYS find root cause before attempting fixes. Symptom fixes are failure. ## The Iron Law ``` NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST ``` If you haven't completed Phase 1, you cannot propose fixes. ## When to Use Use for ANY technical issue: test failures, bugs, unexpected behavior, performance problems, build failures, integration issues. **Use this ESPECIALLY when:** - Under time pressure (emergencies make guessing tempting) - "Just one quick fix" seems obvious - You've already tried multiple fixes - Previous fix didn't work - You don't fully understand the issue ## The Four Phases You MUST complete each phase before proceeding to the next. ### Phase 1: Root Cause Investigation **BEFORE attempting ANY fix:** 1. **Read Error Messages Carefully** - Don't skip past errors or warnings - Read stack traces completely - Note line numbers, file paths, error codes 2. **Reproduce Consistently** - Can you trigger it reliably? - What are the exact steps? - If not reproducible, gather more data -- don't guess 3. **Check Recent Changes** - What changed that could cause this? - Git diff, recent commits - New dependencies, config changes, environmental differences 4. **Gather Evidence in Multi-Component Systems** **WHEN system has multiple components (CI -> build -> signing, API -> service -> database):** **BEFORE proposing fixes, add diagnostic instrumentation:** ``` For EACH component boundary: - Log what data enters component - Log what data exits component - Verify environment/config propagation - Check state at each layer Run once to gather evidence showing WHERE it breaks THEN analyze evidence to identify failing component THEN investigate that specific component ``` 5. **Trace Data Flow** See `root-cause-tracing.md` in this directory for the complete backward tracing technique. **Quick version:** - Where does bad value originate? - What called this with bad value? - Keep tracing up until you find the source - Fix at source, not at symptom ### Phase 2: Pattern Analysis **Find the pattern before fixing:** 1. **Find Working Examples** - Locate similar working code in same codebase 2. **Compare Against References** - Read reference implementation COMPLETELY, don't skim 3. **Identify Differences** - List every difference between working and broken, however small 4. **Understand Dependencies** - Other components, settings, config, environment, assumptions ### Phase 3: Hypothesis and Testing **Scientific method:** 1. **Form Single Hypothesis** - State clearly: "I think X is the root cause because Y" - Be specific, not vague 2. **Test Minimally** - Make the SMALLEST possible change to test hypothesis - One variable at a time - Don't fix multiple things at once 3. **Verify Before Continuing** - Did it work? Yes -> Phase 4 - Didn't work? Form NEW hypothesis - DON'T add more fixes on top 4. **When You Don't Know** - Say so. Don't pretend. Research more. ### Phase 4: Implementation **Fix the root cause, not the symptom:** 1. **Create Failing Test Case** - Simplest possible reproduction - MUST have before fixing - Use the `tdd-workflow` skill for writing proper failing tests 2. **Implement Single Fix** - Address the root cause identified - ONE change at a time - No "while I'm here" improvements 3. **Verify Fix** - Test passes now? No other tests broken? - Run `task test` and `task test-race` to confirm 4. **If Fix Doesn't Work** - STOP - If < 3 attempts: Return to Phase 1, re-analyze with new information - **If 3+ attempts: STOP and question the architecture (step 5)** 5. **If 3+ Fixes Failed: Question Architecture** **Pattern indicating architectural problem:** - Each fix reveals new shared state/coupling in different place - Fixes require "massive refactoring" to implement - Each fix creates new symptoms elsewhere **STOP and question fundamentals:** - Is this pattern fundamentally sound? - Should we refactor architecture vs. continue fixing symptoms? - **Discuss with your human partner before attempting more fixes** ## Red Flags - STOP and Follow Process If you catch yourself thinking: - "Quick fix for now, investigate later" - "Just try changing X and see if it works" - "Add multiple changes, run tests" - "It's probably X, let me fix that" - "I don't fully understand but this might work" - "Here are the main problems: [lists fixes without investigation]" - Proposing solutions before tracing data flow - **"One more fix attempt" (when already tried 2+)** **ALL of these mean: STOP. Return to Phase 1.** ## Your Human Partner's Signals You're Doing It Wrong **Watch for these redirections:** - "Is that not happening?" - You assumed without verifying - "Will it show us...?" - You should have added evidence gathering - "Stop guessing" - You're proposing fixes without understanding - "Think deeper" - Question fundamentals, not just symptoms **When you see these:** STOP. Return to Phase 1. ## Quick Reference | Phase | Key Activities | Success Criteria | |-------|---------------|------------------| | **1. Root Cause** | Read errors, reproduce, check changes, gather evidence | Understand WHAT and WHY | | **2. Pattern** | Find working examples, compare | Identify differences | | **3. Hypothesis** | Form theory, test minimally | Confirmed or new hypothesis | | **4. Implementation** | Create test, fix, verify (`task test`, `task test-race`) | Bug resolved, tests pass | ## When Process Reveals "No Root Cause" If systematic investigation reveals issue is truly environmental, timing-dependent, or external: 1. You've completed the process 2. Document what you investigated 3. Implement appropriate handling (retry, timeout, error message) 4. Add monitoring/logging for future investigation **But:** 95% of "no root cause" cases are incomplete investigation. ## Supporting Techniques Available in this directory: - **`root-cause-tracing.md`** - Trace bugs backward through call stack to find original trigger - **`defense-in-depth.md`** - Add validation at multiple layers after finding root cause - **`condition-based-waiting.md`** - Replace arbitrary timeouts with condition polling **Related skills:** - **`tdd-workflow`** - For creating failing test case (Phase 4, Step 1) - **`verification-before-completion`** - Verify fix worked before claiming success --- > Converted and distributed by [TomeVault](https://tomevault.io/claim/riddopic) — claim your Tome and manage your conversions.