Diagnosis

Diagnosis

Concept of the skill

Use when facing an unknown software failure, when symptoms point to different root causes, or when an initial debugging attempt has not converged.

Coverage

The triage-first framework that classifies a software failure into a problem class and routes it to the right diagnostic technique before root-cause investigation begins. Names nine symptom classes — Logic Error, Runtime Crash, Data Integrity, Timing / Race, Performance, Configuration, Security, Integration, Tooling / Build / Script-path — and provides a classification decision tree that walks from "is there a stack trace?" to a single class. Specifies a universal evidence-collection protocol (exact error message, reproduction steps, last-known-good state, environment facts) and class-specific evidence checklists. Lays out the technique-selection matrix — stack-trace reading, data-flow tracing, git bisect, differential comparison, instrumentation, MRE isolation, profiling, boundary probing — with each technique's time cost, best-case class, and evidence prerequisite. Defines the diagnostic confidence ladder (level 0 Symptom → 1 Classified → 2 Localized → 3 Root Cause → 4 Verified Fix) with explicit "you can say / you cannot say" boundaries at each level and stuck-state checkpoints (5-min, 10-min, 15-min, oscillation). Names escalation criteria for switching approach, switching class, or escalating to a human. Covers three cross-domain patterns where multiple classes apply simultaneously: the Cascade (one root cause, many symptoms), the Coincidence (two unrelated bugs that look like one), the Environment Ghost (works in one environment, fails in another). Catalogues diagnostic anti-patterns and ships a structured diagnostic-session template.

Philosophy of the skill

Debugging fails most often not because the engineer lacks skill, but because the wrong methodology is applied to the problem class. A timing bug needs different tools than a data-integrity bug. A scope leak needs different thinking than a rendering glitch. The most expensive debugging mistake is spending 30 minutes applying scientific-method debugging to what is actually a configuration error discoverable in 2 minutes.

This skill is the triage nurse, not the surgeon. A nurse does not treat the patient — they take vital signs, route to cardiology or neurology, and escalate to the attending physician when criteria are met. Software diagnosis works the same way: collect evidence, classify the symptom, route to the right specialist technique, and pivot when convergence stalls. The small cost of triage is almost always smaller than the cost of chasing a plausible but wrong cause. Skipping triage because "the cause is obvious" is a confirmation-bias trap; even seasoned engineers benefit from making the classification step explicit.

1. The Diagnostic Triage Protocol

Before debugging, diagnose which kind of problem you have. The class determines the technique and the technique determines the time-to-fix.

1. Collect baseline evidence (Section 3)
2. Classify the symptom            (Section 2)
3. Select the diagnostic technique (Section 4)
4. Execute using the routed technique
5. If not converging after 3 attempts, escalate (Section 6)

Rule: never start fixing before completing steps 1–3. The cost of misclassification often exceeds the cost of a short triage pass, and the written classification gives the next person something concrete to challenge.

Diagnosis vs debugging handoff

Treat the handoff as a contract, not a vague recommendation. Diagnosis does not fix the bug; it decides which investigation path is justified by evidence.

2. Symptom-Classification Taxonomy

Every failure falls into one of nine classes. Each class has a primary diagnostic technique.

Classification decision tree

Is there a stack trace or error message?
  YES → Does it point to a specific line?
          YES → Runtime Crash (read the line; check preconditions)
          NO  → Is it a timeout or OOM?
                  YES → Performance
                  NO  → Logic Error (the error is a symptom of wrong state)
  NO  → Is the output wrong but no error thrown?
          YES → Is the wrongness in calculated numbers or records?
                  YES → Data Integrity
                  NO  → Logic Error
          NO  → Is it intermittent?
                  YES → Timing / Race
                  NO  → Does it depend on environment?
                          YES → Configuration
                          NO  → Does the error message contain a file/module path?
                                  YES → Tooling / Build / Script-path
                                  NO  → Does it involve external services?
                                          YES → Integration
                                          NO  → Are there security signals
                                                (auth failure, permission error,
                                                unexpected data exposure, HMAC failure,
                                                access-control bypass)?
                                                  YES → Security
                                                  NO  → Unknown / Unclassified
                                                          → restart evidence collection;
                                                            run a fresh investigative sweep

3. Evidence-Collection Protocol

Before forming any hypothesis, collect baseline evidence. The class determines the additional evidence needed beyond the universal set.

Evidence safety rule

Diagnostic notes, logs, screenshots, and repro snippets often contain more sensitive information than the final fix. Collect enough evidence to classify the failure, but redact or replace personal data, credentials, session tokens, raw request bodies, and secret-bearing headers before copying evidence into a shared note, issue, audit artifact, or skill. Prefer internal opaque IDs, hashes, synthetic examples, and minimal reproductions over real payload dumps.

Universal evidence (always collect)

Class-specific evidence

Rule: if you cannot fill the universal evidence table, you are not ready to hypothesize. Collect first, think second.

Evidence ledger

Use an evidence ledger when the investigation has more than one plausible class. This keeps assumptions separate from observations and prevents confidence inflation.

If an observation changes the likely class, update the class explicitly. Silent reclassification is how investigations drift into mythology.

4. Technique-Selection Matrix

Once the symptom is classified, pick the cheapest technique that could resolve the class.

Technique-ordering principle

Always start with the cheapest technique that could resolve the class:

1. Read the error (~30 s) — cheapest first pass for runtime crashes
2. Check the environment (~1 min) — cheapest first pass for configuration issues
3. Trace the data flow (~5 min) — cheapest first pass for logic / data errors
4. Isolate with MRE (~10 min) — useful when too many variables remain in play
5. Instrument and observe (~10+ min) — necessary when timing / intermittent failures cannot be reproduced directly

The percentages are intentionally absent. This skill is a routing framework, not a benchmark claim. Use local incident history or an actual eval corpus before making quantified success-rate claims.

5. The Diagnostic Confidence Ladder

As evidence accumulates, confidence in the diagnosis should increase monotonically. If it doesn't, the symptom has been misclassified.

Stuck-state checkpoints

• Stuck at level 0 for > 5 min → you need more evidence; restart Section 3
• Stuck at level 1 for > 10 min → likely misclassification; re-run the classification tree
• Stuck at level 2 for > 15 min → the problem may be cross-domain; check whether multiple classes apply
• Oscillating between levels → stop. Write down what you know vs what you're assuming. The assumption is wrong.

Reclassification rule

Classification is provisional until the evidence keeps moving the confidence ladder upward. Re-run the classification tree when any of these happens:

6. Escalation Criteria

Switch diagnostic approach when

Escalate to human when

7. Cross-Domain Patterns

Some failures span multiple classes simultaneously. These compound failures are the hardest to diagnose.

Pattern: the Cascade

A single root cause triggers symptoms across multiple classes.

Root cause: missing null-check in a data transform
  → Data Integrity symptom: wrong totals
  → Logic Error symptom:    UI shows negative values
  → Integration symptom:    webhook payload rejected by partner

Diagnostic approach: find the earliest symptom in the data flow. That's closest to the root cause.

Pattern: the Coincidence

Two unrelated bugs appear simultaneously, creating a misleading compound symptom.

Bug A: CSS regression from a recent deploy        (Logic Error)
Bug B: slow API from an unrelated query change    (Performance)
Combined symptom: "the page is broken and slow"

Diagnostic approach: separate the symptoms. Test each independently. If fixing one doesn't affect the other, they're independent bugs.

Pattern: the Environment Ghost

Works in one environment, fails in another, with no code difference.

Local:    works   (runtime 20.11, .env.local, fresh DB)
Staging:  fails   (runtime 20.9,  CI env vars, migrated DB)

Diagnostic approach: diff everything — runtime versions, env vars, DB state, feature flags, DNS, SSL, headers. The first difference you find is usually the cause.

8. Anti-Patterns

9. Diagnostic-Session Template

Use this template to structure a diagnostic session. It prevents skipping steps.

## Diagnostic Session: [Brief description]

### 1. Symptom

- What: [exact error or wrong behavior]
- Where: [route / component / job]
- When: [always / intermittent / environment-specific]
- Since: [commit / deploy / data change]

### 2. Classification

- Primary class: [from taxonomy]
- Confidence: [0–4 level]
- Technique: [from technique matrix]

### 3. Evidence Collected

- [ ] Error message / wrong output (exact)
- [ ] Reproduction steps (minimal)
- [ ] Last-known-good state
- [ ] Environment facts
- [ ] Sensitive evidence redacted or replaced with safe identifiers
- [ ] Class-specific evidence: [list]

### 4. Evidence Ledger

| Observation | Source | Class signal | Contradiction | Next test |
| ----------- | ------ | ------------ | ------------- | --------- |
|             |        |              |               |           |

### 5. Hypotheses Tested

| #   | Hypothesis | Test | Result | Confidence after |
| --- | ---------- | ---- | ------ | ---------------- |
| 1   |            |      |        |                  |

### 6. Resolution

- Root cause: [one sentence]
- Fix: [what was changed]
- Prevention: [test / guard / doc added]

Grounding and Evaluation State

This skill is grounded in public troubleshooting and diagnostic-practice references: Google SRE troubleshooting guidance, git bisect documentation for regression bisection, Stack Overflow MRE guidance for isolation, Chrome DevTools and PostgreSQL EXPLAIN docs for measurement/profiling examples, OWASP logging guidance for diagnostic event capture, and OpenTelemetry sensitive-data guidance for safe telemetry handling.

The current eval metadata remains intentionally conservative: eval_artifacts: planned, eval_state: unverified, and routing_eval: absent. Do not mark this skill verified or routing-present until a real comprehension eval and routing eval include diagnosis and pass in the same change.

Verification

• The symptom was classified before any debugging technique was chosen
• Baseline evidence was collected before any hypothesis was formed
• Sensitive or secret-bearing evidence was redacted, synthesized, hashed, or replaced with opaque IDs before sharing
• The cheapest technique that could resolve this class was tried first
• Confidence increased monotonically — or the symptom was re-classified the moment it didn't
• If the approach was changed, the reason was documented (which signal triggered the switch)
• The time-based stuck-state checkpoints were respected (5-min / 10-min / 15-min triggers)
• If the failure spanned multiple classes, the cross-domain pattern (Cascade / Coincidence / Environment Ghost) was named explicitly

Do NOT Use When

Skill Graph context

Classification

• Subject: software-engineering-method
• Public: true
• Domain: engineering/debugging
• Scope: Use when facing an unknown software failure, when symptoms point to different root causes, or when an initial debugging attempt has not converged. Provides a triage-first diagnostic routing framework: classify the failure, collect the right evidence, choose a technique, track confidence, and escalate when stuck. Do NOT use for executing scientific debugging after triage (use debugging), code-quality review (use code-review), or proactive observability setup.

When to use

• the agent has been chasing this bug for 30 minutes — what's the structural fix?
• the symptoms span data integrity and UI rendering — which is the root cause?
• the build fails locally but passes in CI — how do I diagnose that class first?
• I have a stack trace and an unhandled exception — what's the cheapest technique?
• intermittent failure that doesn't reproduce on retry — which class is this?
• we ran profiling, instrumentation, and bisect — none converge. What did we misclassify?
• two engineers disagree on whether this is a config issue or a logic error — what evidence settles it?

Not for

• actually execute scientific-method debugging on this stack trace
• review this AI-generated PR for correctness
• scan this repo for OWASP top 10 vulnerabilities
• design observability instrumentation for this service
• decide which agent should pick up this ticket
• what's the right test pyramid for this feature

Related skills

• Verify with: debugging, a11y
• Related: code-review, error-tracking, owasp-security, testing-strategy, debugging

Grounding

• Mode: universal
• Truth sources: https://sre.google/sre-book/effective-troubleshooting/, https://git-scm.com/docs/git-bisect, https://stackoverflow.com/help/minimal-reproducible-example, https://developer.chrome.com/docs/devtools/performance/overview, https://www.postgresql.org/docs/current/sql-explain.html, https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html, https://opentelemetry.io/docs/security/handling-sensitive-data/

Keywords

• diagnostic triage software failure, symptom classification taxonomy, what kind of bug is this, which debugging approach, diagnostic routing framework, evidence collection before hypothesis, confidence ladder debugging, escalation criteria debugging, cascade vs coincidence failure, environment ghost