Issue Debugging

Issue Debugging

Systematic methodology for debugging that prevents the "first error fix and declare done" anti-pattern. Ensures root cause analysis, impact assessment, and verified fixes.

When to Use This Skill

Load this skill when the user request matches ANY of these patterns:

DO NOT use for:

• New feature implementation (no existing bug)
• Refactoring without reported issues
• General code questions

Overview

The Problem

AI (and developers) often fix the first visible error and declare victory without verifying end-user functionality works. This leads to:

• Symptom fixes instead of root cause fixes
• Regressions in seemingly unrelated areas
• False confidence in fix completeness

The Solution

This skill provides:

1. Root Cause Analysis - Find the actual cause, not just symptoms
2. Impact Analysis - Map affected areas and dependencies
3. Validation Planning - Tiered test priorities (P1/P2/P3)
4. Confidence Assessment - Objective fix reliability rating
5. Escalation Triggers - When manual testing is required

Root Cause Analysis

The 5 Whys Method

Iteratively ask "Why?" to drill past symptoms to root cause:

Problem: Login fails with 500 error
├─ Why? → API returns null for user profile
├─ Why? → Database query found no records
├─ Why? → Migration didn't run on deploy
├─ Why? → Deployment script skipped migration step
└─ Why? → Environment variable missing

Root Cause: Missing environment variable (not the 500 error)

Key Principle: Focus on HOW and WHY the defect occurred, not just WHERE.

Hypothesis-Driven Debugging

Apply scientific method:

Binary Search / Bisection

For isolating bugs in time or space:

1. Place checkpoint at midpoint
2. Determine if bug is before or after
3. Repeat on remaining half
4. Continue until exact location found

Efficiency: O(log n) - approximately 7 tests for 100 commits.

Impact / Dependency Analysis

Mapping Procedure

For each affected file, identify:

Affected Code: src/auth/login.ts

Upstream (What calls this?):
├─ src/api/auth-routes.ts → POST /login
├─ src/middleware/session.ts → validateSession()
└─ src/pages/login.tsx → handleSubmit()

Downstream (What does this affect?):
├─ User dashboard → fetches profile on load
├─ Settings page → assumes profile exists
└─ API responses → include user context

Risk Scope Assessment

Debug Report Format

Location

Debug reports are written to: logs/debug-reports/{issue-id}-{timestamp}.yaml

This location is separate from standard agent logs to:

• Enable easy reference by downstream pipeline stages
• Keep debug context grouped by issue
• Allow FixValidator to read validation plans

Schema Overview

debug_report:
  metadata:
    issue_id: "{id}"
    timestamp: "{ISO-8601}"
    analyzer: "bulwark-issue-analyzer"
    complexity: low | medium | high

  analysis:
    symptom: "{user-visible problem}"
    root_cause: "{underlying reason}"
    fix_approach: "{recommended approach}"

  impact_analysis:
    affected_files: [...]
    upstream_dependencies: [...]
    downstream_effects: [...]
    risk_scope: isolated | medium | broad

  validation_plan:
    tests_to_execute:
      - path: "{test file}"
        reason: "{why this test}"
        priority: 1  # P1=must, P2=should, P3=nice-to-have
    functionalities_to_validate: [...]

  confidence_criteria:
    high: [...]
    medium: [...]
    low: [...]

  debug_journey:  # Required for medium/high complexity
    hypotheses_tested: [...]

Full schema: See references/debug-report-schema.md

Validation Plan

Tiered Test Priorities

Selection Criteria

P1 Tests (always run):

• Tests in the same file/module as the fix
• Tests named after the affected function
• Tests that were failing before the fix

P2 Tests (run if possible):

• Tests for upstream callers
• Tests for downstream consumers
• Tests for related functionality

P3 Tests (run if available):

• E2E tests that include the affected flow
• Performance tests
• Edge case tests

Functionality Validation

Beyond tests, list user-level verifications:

• "User can log in successfully"
• "Dashboard displays correct data after login"
• "Session persists across page refresh"

Confidence Rubric

Assessment Criteria

Determining Complexity

Escalation Triggers

Manual testing is required when ANY of:

• Confidence level is low
• Risk scope is broad AND confidence is not high
• Any functionality cannot be validated via automated tests
• Security-related fix without security test coverage

Escalation Message Format

When escalation is triggered, the Orchestrator must inform the user:

Manual testing required for:
- [functionality 1]
- [functionality 2]

Reason: [why automated validation insufficient]

Recommended manual test steps:
1. [step]
2. [step]

Debug Journey Logging

When Required

Format

debug_journey:
  started_at: "{timestamp}"
  completed_at: "{timestamp}"

  hypotheses_tested:
    - hypothesis: "Database connection timeout causing failure"
      tested_at: "{timestamp}"
      method: "Checked DB logs and connection pool stats"
      result: rejected
      evidence: "DB logs show successful queries, pool not exhausted"

    - hypothesis: "Null profile object when user has no data"
      tested_at: "{timestamp}"
      method: "Added logging at profile access point"
      result: confirmed
      evidence: "Stack trace shows NPE at profile.name access"

  investigation_path:
    - "Started with error log analysis"
    - "Ruled out infrastructure issues"
    - "Narrowed to application code"
    - "Identified profile access as failure point"

Anti-Patterns

What NOT to Do

Full checklist: See references/anti-patterns.md

Quick Reference

Before Fixing

• Symptom identified and documented
• Root cause analysis completed (not just symptom)
• Impact analysis completed (upstream/downstream)
• Risk scope assessed (isolated/medium/broad)
• Complexity determined (low/medium/high)

After Fixing

• Fix implemented
• P1 tests pass
• P2 tests pass (or documented why skipped)
• Confidence level assessed
• Manual testing escalated if required
• Debug journey documented (if medium/high complexity)
• Debug report written to logs/debug-reports/

Complexity → Actions

Integration

Pipeline Integration

This skill is used by the Fix Validation pipeline:

IssueAnalyzer (loads this skill, produces debug report)
|> FixWriter (reads debug report for context)
|> TestWriter (reads validation plan)
|> FixValidator (executes validation plan, assesses confidence)
|> CodeReviewer (reviews all, makes approval decision)

Agent Integration

Artifact Flow

IssueAnalyzer
    └─> logs/debug-reports/{issue-id}.yaml
              │
              ├─> FixWriter (reads root cause, fix approach)
              ├─> TestWriter (reads validation plan)
              ├─> FixValidator (executes tests, assesses confidence)
              └─> CodeReviewer (reviews everything)

Related Skills

• subagent-prompting (P0.1) - 4-part template for agent invocation
• subagent-output-templating (P0.2) - Output format for logs
• pipeline-templates (P0.3) - Fix Validation pipeline definition