Abductive Mode

Abductive Mode

Generate best explanation from observation through competing hypotheses.

When to Use

• Single anomaly or unexpected outcome occurred
• Need to diagnose root cause
• Something surprising happened
• Question is "Why did this happen?"

Flow

Observation → Hypotheses (≥5) → Evidence Debate → Best Explanation

Stage 1: Observation

Quantify the anomaly before explaining it.

Required elements:

• Deviation: Numeric change (absolute and relative)
• Timeline: When started, duration, persistence
• Scope: Which segments, systems, users affected
• Baseline: Where the expected value comes from

Example:

Observation: Enterprise conversion dropped from 15% to 9% (40% relative decline).

Timeline: Started week 5 of Q3, accelerated through Q4.

Scope: Enterprise segment only; SMB conversion stable at 22%.

Baseline: 15% was trailing 6-month average before Q3.

Gate: Anomaly must be quantified, not just described. "Conversion dropped" is insufficient. "Conversion dropped 40% (15% → 9%)" is sufficient.

Stage 2: Hypothesis Generation

Generate ≥5 hypotheses across ≥3 categories before evaluating any.

Standard Categories

AI System Categories

When diagnosing AI/agent failures, add these:

By Intelligence Layer:

By System Layer:

By Agent Failure Mode:

Example:

H1 (Operational): AM departures caused relationship gaps.
Mechanism: Key AMs left → rapport lost → deals stalled.

H2 (External): Economic uncertainty extending cycles.
Mechanism: CFO involvement up → longer approval chains.

H3 (Market): Competitor launched enterprise offering.
Mechanism: Prospects evaluating alternatives before committing.

H4 (Product): Missing enterprise features blocking purchases.
Mechanism: SSO/audit gaps appearing in requirements.

H5 (Operational): Not a real drop—just elongated cycles.
Mechanism: Pipeline value unchanged, timing shifted.

Gate: Must have ≥5 hypotheses across ≥3 categories before proceeding.

Stage 3: Evidence Debate

Test each hypothesis against available evidence.

For each hypothesis, ask:

1. What would we expect to see if this is true?
2. What would contradict this?
3. What evidence do we actually have?

Scoring dimensions:

Challenge techniques:

• "This doesn't explain [aspect of the anomaly]..."
• "If this were true, we'd also see [X], but we don't..."
• "This assumes [Y], but is that valid?"

Example:

Testing H1 (AM departures):

• Supporting: 60% of drop concentrated in departed AMs' accounts ✓
• Supporting: Timeline matches—departures Q3, acceleration Q4 ✓
• Challenge: Doesn't explain why NEW accounts also slowed (15% of drop)
• Verdict: Explains 85% of effect, not 100%

Testing H3 (Competitor):

• Check: Win/loss data shows only 3 of 47 losses mentioned competitor
• That's 6%—noise, not signal
• Verdict: Ruled out as primary cause

Testing H4 (Features):

• Check: Zero mentions of SSO/audit in loss reasons
• Verdict: Ruled out

Gate: Each top-3 hypothesis must survive at least one serious challenge.

Stage 4: Best Explanation

Synthesize findings into conclusion.

Required elements:

1. Primary cause with confidence (0-100%)
2. Mechanism explaining how cause produces effect
3. Minority report if second hypothesis ≥40% confidence
4. Ruled out hypotheses with reasons
5. Remaining uncertainty
6. Next steps

Output format:

## Diagnosis: [Anomaly Name]

**Primary cause ([X]% confidence):** [Explanation in 1-2 sentences]

**Mechanism:** [How cause produces effect]

**Minority view ([Y]% confidence):** [Second hypothesis if ≥40%]
- Revisit if: [Conditions that would change conclusion]

**Ruled out:**
- [H3]: [Evidence that eliminated it]
- [H4]: [Evidence that eliminated it]

**Remaining uncertainty:** [What we still don't know]

**Next steps:**
1. [Immediate action]
2. [Monitoring plan]

Example:

Diagnosis: Enterprise Conversion Drop

Primary cause (72% confidence): AM departures created relationship gaps, stalling deals in existing accounts.

Mechanism: Key AMs left Q3 → relationship continuity lost → deals in progress stalled → appears as conversion drop in Q4.

Minority view (45% confidence): Economic uncertainty explains slowdown in new accounts (15% of total effect).

• Revisit if: Recovery doesn't materialize after AM backfill

Ruled out:

• Competitor: Only 6% of losses mentioned them
• Missing features: Zero mentions in loss reasons

Remaining uncertainty: Whether elongated cycles will eventually convert or are truly lost.

Next steps:

1. Backfill AM positions with enterprise experience
2. Re-engage stalled deals with new relationship owners
3. Monitor Q1 conversion with weekly tracking

AI System Diagnosis Example

## Observation

Agent task completion dropped from 85% to 52% after model upgrade.
- Timeline: Immediate, started with deployment
- Scope: Multi-step tasks only; single-step tasks unaffected
- Baseline: 85% was 30-day average before upgrade

## Hypotheses

**H1 (Reasoning layer):** New model has thinner geometry for tool sequencing.

**H2 (Agency layer):** Tool schemas incompatible with new model's output format.

**H3 (Memory layer):** Context window handling changed, losing mid-task state.

**H4 (Orchestration layer):** CoT prompts were tuned for old model.

**H5 (Representation layer):** Token boundaries different, breaking structured output parsing.

## Evidence Debate

**H1 test:** If geometry is thinner, single-step tasks with same tools should also degrade.
- Finding: Single-step tasks unaffected.
- Verdict: H1 less likely.

**H2 test:** Check tool call logs for format errors.
- Finding: 67% of failures show malformed tool parameters.
- Verdict: H2 supported.

**H4 test:** Review CoT prompts for model-specific assumptions.
- Finding: Prompts reference old model's output structure.
- Verdict: H4 supported, may combine with H2.

## Diagnosis

**Primary cause (68% confidence):** Tool schemas and CoT prompts assume old model's output format (H2 + H4 combined).

**Mechanism:** New model produces valid JSON but with different field ordering and optional field handling → parsing fails → tool calls malformed → multi-step chains break.

**Next steps:**
1. Update tool schemas for new model's output format
2. Revise CoT prompts to be model-agnostic
3. Add output validation layer between model and tool execution

Output Format

## Diagnosis: [Anomaly Name]

### Observation
**Anomaly:** [Quantified deviation]
**Timeline:** [When started, duration]
**Scope:** [What's affected, what's not]
**Baseline:** [Expected value and source]

### Hypotheses Considered

| # | Hypothesis | Category | Confidence |
|---|------------|----------|------------|
| H1 | [Cause] | [Category] | [%] |
| H2 | [Cause] | [Category] | [%] |
| H3 | [Cause] | [Category] | [%] |

### Evidence Analysis

**H1: [Hypothesis]**
- Supporting: [Evidence for]
- Contradicting: [Evidence against]
- Verdict: [Supported/Weakened/Ruled out]

**H2: [Hypothesis]**
- Supporting: [Evidence for]
- Contradicting: [Evidence against]
- Verdict: [Supported/Weakened/Ruled out]

### Conclusion

**Primary cause ([X]% confidence):** [Explanation in 1-2 sentences]

**Mechanism:** [How cause produces effect]

**Minority view ([Y]% confidence):** [Second hypothesis if ≥40%]
- Revisit if: [Conditions that would change conclusion]

### Ruled Out
- **[H3]:** [Evidence that eliminated it]

### Remaining Uncertainty
- [What we still don't know]

### Next Steps
1. [Immediate action]
2. [Monitoring plan]

Quality Gates

Anti-Patterns