Causal Reasoning Klein

SKILL.md — Causal Reasoning in the Wild (Klein & Hoffman)

license: Apache-2.0

Metadata

name: causal-reasoning-klein
description: >
  Skill for naturalistic causal reasoning — diagnosing why things happened,
  attributing causes under uncertainty, building explanatory frames, and
  deciding when to stop analyzing and start acting. Based on Klein & Hoffman's
  empirical taxonomy of how experts actually reason about causality in messy,
  real-world domains.
activation_triggers:
  - "why did this happen"
  - "what caused this failure"
  - "root cause analysis"
  - "debugging / post-mortem / incident review"
  - "causal attribution"
  - "explain this outcome"
  - "what went wrong"
  - "how do we prevent this"
  - "diagnosis / diagnosing"
  - "what's the real reason"
  - "multiple possible causes"
  - "I can't find the cause"
version: 1.0
source: >
  Klein, G. & Hoffman, R. — "Causal Reasoning: Initial Report of a
  Naturalistic Study of Causal Inferences"

When to Use This Skill

Load this skill when:

• Diagnosing failures — a system, process, decision, or outcome went wrong and you need to explain why
• Causal attribution is contested — multiple competing explanations exist and you need a framework for evaluating them
• The "obvious" cause feels too simple — someone is reaching for a single root cause in a situation that is clearly complex
• You're designing diagnostic agents — building an agent or orchestration system that must investigate and explain failures
• Closure is hard — analysis keeps expanding and you need principled criteria for when to stop
• Explanation type matters — you need to choose how to explain something, not just what to explain
• Post-mortems or retrospectives — organizational, technical, or strategic failure reviews
• Experts disagree about cause — different framings produce different cause-sets and you need to understand why

Do not primarily rely on this skill for:

• Pure statistical causal inference (use Judea Pearl / do-calculus frameworks instead)
• Formal fault tree analysis with known component failure rates
• Situations where full information is available and the cause is unambiguous

Core Mental Models

1. Five Explanation Types — Form Shapes What Gets Found

Real-world causal reasoning uses five structurally different explanation forms. The form chosen is not neutral — it determines what counts as a "relevant cause" to search for.

Why it matters: An agent using only event-type reasoning in a systemic failure will miss structural conditions. An agent defaulting to list-type reasoning will enumerate causes without modeling how they interact.

2. The Reciprocal Model — Frames and Causes Co-Construct Each Other

The search for causes and the construction of an explanatory frame happen simultaneously, not sequentially. The frame determines what counts as a relevant cause; the causes found reshape the frame.

[Candidate Causes] ←→ [Explanatory Frame]
        ↑                       ↑
        └─── each reshapes the other in real time ───┘

Critical implication: If you start with the wrong frame, you will systematically miss causes that don't fit it — not because you're careless, but because the frame defines the search space. Changing the frame isn't "trying harder"; it's accessing a different set of candidate causes entirely.

3. Three Attribution Criteria — Propensity, Reversibility, Covariation

These are the workhorses of causal judgment. They operate independently and can produce conflicting verdicts:

• Propensity: Could this plausibly lead to that? (mechanism, plausibility)
• Reversibility (Mutability): Would removing this cause eliminate the effect? (counterfactual test)
• Covariation: Do cause and effect travel together statistically? (correlation, pattern)

The watch-factory problem: Context can flip all three verdicts simultaneously. A cause that passes all three criteria in isolation may fail all three in a different systemic context. Always apply all three; always ask whether context reverses the verdict.

4. The Reductive Tendency — Necessary Trap

Humans systematically compress dynamic, nonlinear, simultaneous causality into linear, sequential chains. This is both:

• Cognitively necessary — you cannot act on a fully simultaneous multi-causal model
• Epistemically dangerous — the compression creates fiction that becomes invisible

The correct response is not to eliminate reduction but to know which register you're in:

• Diagnostic mode: resist reduction, hold complexity, use story-type explanation
• Action mode: embrace reduction, identify the actionable cause, achieve closure

Agents that operate only in one register will fail in the other.

5. Indeterminate Causation — The Norm, Not the Exception

In organizational, political, military, and economic domains, there is no single true cause. Multiple interacting partial causes are the rule. The quest for the root cause:

• Is an epistemological error
• Produces oversimplification
• Generates interventions that fix the named cause while leaving structural vulnerabilities intact

The correct output of causal reasoning in complex domains is a partial-cause set with explicit representation of uncertainty, not a single root cause with false precision.

Decision Frameworks

Which Explanation Type Should I Use?

IF the domain is competitive/event-driven AND reversibility is high
  → USE Event explanation (counterfactual framing)

IF you recognize a known failure pattern from prior cases
  → USE Abstraction explanation (pattern matching)

IF the failure was structurally overdetermined (would have happened anyway)
  → USE Condition explanation (enabling conditions framing)

IF causes are numerous, interactive, and mechanism is unclear
  → USE Story explanation (rich causal narrative)

IF you are at an early diagnostic stage with incomplete information
  → USE List explanation (enumeration) AS A TEMPORARY SCAFFOLD
  → Plan to upgrade to Story or Condition as information arrives

Am I Using the Right Frame?

IF you've been searching for causes and keep finding nothing
  → The frame is probably wrong, not the search
  → Load: references/reciprocal-framing-and-causal-search.md
  → Deliberately adopt an alternative frame and restart

IF experts with different backgrounds disagree about cause
  → They are likely using different frames, not different evidence
  → Map which explanation type each expert is using
  → Look for the cause-set each frame is structurally incapable of seeing

IF the causal story feels too clean or too linear
  → You have over-reduced
  → Load: references/reductive-tendency-and-when-to-resist-it.md
  → Ask: what simultaneity and interaction have I compressed away?

Evaluating a Candidate Cause

FOR each candidate cause, apply all three criteria:

  Propensity test:
    "Is there a plausible mechanism by which this could produce the effect?"
    IF no → weak candidate, but don't eliminate (unknown mechanisms exist)
    IF yes → proceed to reversibility

  Reversibility test:
    "If we remove this cause, does the effect go away?"
    IF no → this is an enabling condition, not a trigger cause
    IF yes → strong candidate
    IF unclear → the system may be overdetermined (effect has multiple sufficient causes)

  Covariation test:
    "Does this cause co-occur with this effect across cases?"
    IF no → likely spurious or context-specific
    IF yes → corroborating evidence (not proof)

  Context reversal check:
    "Does the systemic context flip any of these verdicts?"
    Load: references/three-causal-criteria-for-agent-diagnosis.md

When to Stop Analyzing (Closure Decision)

IF stakes are low AND time pressure is high
  → Close early, use action-cause (the single most actionable cause)
  → Document that full causal story is unresolved

IF stakes are high AND time is available
  → Continue until Story-type explanation is coherent
  → Require that enabling conditions are identified, not just trigger causes

IF analysis keeps expanding without convergence
  → Check: are you holding multiple frames simultaneously?
  → Apply proportionality: closure depth must match decision stakes
  → Load: references/closure-decisions-and-the-action-understanding-tradeoff.md

IF you must act before understanding is complete
  → Design the action as a probe: what will this action reveal about cause?
  → Action-as-investigation is how experts close the analysis/action gap

Designing a Diagnostic Agent or Orchestration

IF building parallel investigation agents
  → Deliberately assign DIFFERENT initial frames to each agent
  → Frame diversity is not inefficiency — it is the mechanism for finding
    causes that a single frame would miss

IF deciding when to commit an orchestrator to an output
  → Apply the closure criteria above
  → Require dual output: action-cause (for intervention) + causal story (for structure)

IF decomposing a complex failure into sub-investigations
  → Map to explanation type first
  → Story-type failures → decompose into parallel causal streams modeled jointly
  → Condition-type failures → decompose around structural features
  → NEVER decompose simultaneous processes as sequential (reductive trap)
  → Load: references/causal-stories-as-system-models.md

Reference Table

Load these files on demand when the specific sub-problem arises. Do not load all at once.

Anti-Patterns

These are the mistakes Klein & Hoffman's research most directly warns against:

1. Root Cause Fundamentalism Treating "find the single root cause" as the goal of causal analysis. In complex domains, this produces a named cause that absorbs blame while leaving structural vulnerabilities intact. The right output is a partial-cause set with interaction structure.

2. Frame Blindness Searching harder within a frame that isn't working, rather than changing the frame. When a search stalls, the failure is usually in the frame, not in the effort. Frame-switching is a first-class diagnostic move, not an admission of failure.

3. Sequential Compression of Simultaneous Causation Describing processes that happen in parallel and interact as if they happened in sequence. This is the reductive tendency at its most dangerous — the narrative becomes plausible and the simultaneity that produced the failure becomes invisible.

4. Explanation Type Mismatch Using event-type explanation for condition-caused failures (e.g., "if only the pilot had done X differently" when the system was structurally unsafe). Or using list-type explanation when a story-type account is needed to model interaction effects.

5. Premature Closure Under Complexity Achieving closure at the action-cause level and treating it as complete understanding. Action-causes (the trigger to intervene on) are not the same as causal stories (the structural understanding needed to prevent recurrence). Both are needed; treating one as sufficient produces intervention that fixes nothing durable.

6. Commentator Reasoning Applied to Participant Situations Analyzing expert decisions as if the expert had full information, no time pressure, and was not causally entangled in the situation. Commentator reasoning (post-hoc, reflective, unconstrained) is structurally different from participant reasoning (real-time, action-pressured, self-entangled). Applying the former standard to the latter situation produces unfair and useless diagnosis.

7. Covariation Treated as Sufficient for Causation Statistical co-occurrence confirms a causal hypothesis but does not establish it. Propensity (mechanistic plausibility) and reversibility (counterfactual test) must also be applied. Covariation-only reasoning produces interventions that address correlates rather than causes.

Shibboleths

How to tell if someone has actually internalized this book vs. skimmed a summary:

They will spontaneously ask "which explanation type is this?" Someone who has read only the summary will say "there are five types of causal explanation." Someone who has internalized it will, when encountering a causal claim, immediately ask which type is being used — and notice when the type is wrong for the situation.

They treat frame-switching as a first move, not a last resort. Naive readers conclude "try harder if the search fails." Internalized readers ask "is the frame the problem?" early and treat deliberate reframing as a normal diagnostic tool, not a sign of failure.

They distinguish action-causes from causal stories without prompting. They will say something like: "We know what to fix right now, but we don't yet understand why the system was vulnerable — those are two different questions and we need both answers."

They are suspicious of single-cause explanations in complex domains. Not because they oppose simplicity, but because they have a specific concept — indeterminate causation — that makes single-cause accounts in organizational or systemic contexts feel epistemologically naive, not just incomplete.

They understand why experts and novices reach for different frames. They won't say "the expert is biased" or "the novice is naive." They will explain, mechanistically, which causal features each frame makes visible and which it occludes — and why an expert's frame reflects accumulated knowledge about which features are typically diagnostic in that domain.

They use "propensity," "reversibility," and "covariation" as working vocabulary. And more importantly, they apply them independently — they know that a cause can satisfy one criterion and fail another, and they know that context can reverse verdicts they thought were settled.

They think about closure as a design parameter, not a discovery. They don't ask "when will we know enough?" They ask "given the stakes, how deep should closure be, and what is the minimum causal story sufficient for proportionate action?"

This skill indexes eight reference documents. Load them on demand as specific sub-problems arise. The skill itself is sufficient for most high-level causal reasoning tasks; the reference files provide working depth for complex cases.