Causal Identification

Causal Identification

Overview

A regression coefficient is a correlation with good posture. It becomes a causal effect only when a design rules out the other explanations — and that design rests on assumptions no amount of clean data or tight standard errors can supply. The fatal error is silent: the code runs, the coefficient is significant, the sign is plausible, and it's still confounding wearing the costume of an effect.

Core principle: State the identification assumptions before you estimate, and test the ones that are testable. The estimate is only as credible as the assumption you can't test.

Before any model, answer the Angrist–Pischke question: if you could run the ideal randomized experiment, what would it be — and what real-world variation are you using as a stand-in? Name the source of variation in one sentence and argue it's as good as random, or you have a regression hoping to be an identification strategy.

The Design Card — sign-off before estimation

A causal claim locks a Design Card — invoke analysis-state-management, write it to docs/analysis/ (or point to the PAP), get sign-off before estimating. (Structural → model card, prediction → Prediction Spec — same gate, different document; mechanics owned by analysis-checkpoints.) Fields:

• Causal question + estimand — ATT/ATE/LATE, for which population.
• Design + source of variation — the "what's your experiment?" answer, one sentence.
• The untestable assumption, in plain language, and why it's plausible here.
• Diagnostics planned — the design-specific tests you'll run before reading the estimate.
• Robustness shortlist — the ~3 checks aimed at the main threat (approval-gated, see below).
• Primary spec — outcome, treatment, FE/controls (each control with its confounding story), SEs/clustering.

Entering mid-stream ("just run the DiD") does not waive the card: "the user already said regress Y on X" names the spec, not the design — reconstruct the card from context in ≤10 lines and confirm before estimating.

The discipline

1. Name the design and the source of variation: what's treated vs. control, and why is the control a valid counterfactual? If you can't name the design, you don't have identification — you have a regression.
2. State the assumptions out loud, especially the untestable one — every design has a load-bearing assumption you can't verify from data (exclusion, parallel trends, continuity, unconfoundedness). Name it and argue why it holds here.
3. Test the testable implications (the diagnostics below). Borderline diagnostics are a checkpoint, not a green light — a first-stage F of 8, a mildly sloped pre-trend, balance that almost resolves: surface these, don't proceed past them silently.
4. Estimate with inference appropriate to the design (clustering, weak-IV-robust, etc.).
5. Attack it — propose the ~3 threat-relevant robustness/placebo checks, get approval, then run them (see "Robustness, placebo, sensitivity" below — not the whole catalogue).
6. Reconcile the causal estimate with the raw descriptive picture. An effect that's invisible in the raw data and only appears after heavy modeling deserves suspicion.

Presenting a design — spell it out

A design presented to the user is a decision document. Every presentation of a specification — in a plan, a checkpoint, a results write-up — carries five things:

• The estimating equation, written out. Display math (or plain text), not a name-drop: $y_{it} = \beta D_{it} + \alpha_i + \gamma_t + \varepsilon_{it}$ — with every subscript defined (what is $i$, what is $t$, what population, what period) and the level of variation stated: what level treatment varies at, what level the outcome is measured at, what the fixed effects absorb, what level you cluster at and why. "TWFE DiD" is a family, not a specification.
• The economic intuition. Nearly every model and estimator has one — the comparison in plain language and why it recovers the effect ("adopting counties are compared to never-adopters in the same month, netting out anything fixed about a county and anything common to a month; identification rests on the two groups trending together absent the policy"). If you can't state the intuition, you don't understand the design well enough to run it. Convince the reader; don't assert.
• The literature precedent. Name the published design this follows — the nearest antecedent, top-5-journal focus (AER, QJE, JPE, ECMA, ReStud, plus the field's flagship). Don't invent a design when the literature has one. A specification with no named precedent is a flag: either you haven't done the literature review (do it — econ-writing:literature-review when the stakes warrant a real one) or the design is genuinely novel, which you state explicitly and justify as a deliberate departure the user signs off on.
• No shorthand. Never refer to a specification, hypothesis, or plan node by a bare internal id — "a1", "h1", "spec 3" mean nothing to a reader who didn't watch the plan being built. Restate what it is in words every time ("the reviewer-capacity event study"); an id may follow in parentheses, never stand alone.
• Interpretable from the start. State up front what units $\beta$ will be in and what magnitude would be economically meaningful (ties to the reporting red-line).

Choosing or changing the design is the user's decision

A failed diagnostic (pre-trends violated, weak first stage) or a newly discovered threat is analysis-checkpoints territory, not a silent upgrade: surface the threat, candidate remedies, and your recommendation — "parallel trends is violated; switch to triple-difference, restrict the sample, or report with a caveat" — and let the user decide.

Per-design assumptions and diagnostics

Tag discipline: Test items are diagnostics — run before/with estimation, no approval needed. Robustness items (and every placebo) belong to the approval-gated ~3-check shortlist (see "Robustness, placebo, sensitivity" below). Don't reclassify a placebo as a diagnostic to skip the checkpoint, or a pre-trend test as robustness to stall it.

Difference-in-differences / event study

• Load-bearing assumption: parallel trends — treated and control would have moved together absent treatment. Untestable directly; argue it.
• Test: pre-treatment trends (event-study coefficients; flat, insignificant leads support but don't prove parallel trends), anticipation (effects before treatment), and no compositional change in the panel around treatment.
• Staggered adoption is a trap: with variation in treatment timing, two-way fixed effects (TWFE) is biased by "forbidden comparisons" of late-treated to already-treated units. Use a modern estimator: Callaway–Sant'Anna, Sun–Abraham, Borusyak et al., de Chaisemartin–D'Haultfœuille, did2s — not vanilla TWFE.
• Inference: cluster SEs at the treated unit (e.g., state), and worry about too-few clusters.

Instrumental variables

• Relevance (testable): report the first-stage F; a weak instrument (F < 10 rule of thumb, prefer Olea–Pflueger) biases 2SLS and its SEs — use weak-instrument-robust inference (Anderson–Rubin) when in doubt.
• Exclusion (untestable): the instrument affects the outcome only through the treatment. Cannot be tested — argue it substantively; the whole IV stands or falls here.
• Monotonicity: no "defiers" — needed for the estimate to be a valid LATE (effect on compliers), not ATE.

Regression discontinuity

• Continuity (load-bearing): units just above and just below the cutoff are comparable; potential outcomes are continuous at the threshold.
• No manipulation: units can't precisely sort around the cutoff — test with a McCrary / density test for a jump in the running variable.
• Robustness: bandwidth sensitivity (a principled bandwidth via rdrobust), covariate smoothness at the cutoff, a donut spec excluding points near it, and placebo cutoffs away from the real one.

Matching / regression adjustment / propensity scores

• Unconfoundedness (untestable): selection into treatment is on observables only — the strongest assumption in the toolkit, argue it hard.
• Overlap / common support (testable): treated and control propensity distributions overlap. Trim or stop if they don't.
• Balance (testable): post-matching/weighting covariate balance — report standardized mean differences (rule of thumb |SMD| < 0.1), not just t-tests.

Panel fixed effects

• Identify off within-unit variation — confirm there is enough of it; a near-time-invariant regressor is barely identified.
• FE controls only time-invariant confounders; time-varying confounders still bite.
• Cluster SEs at the appropriate level.

Synthetic control

• Load-bearing assumption: no anticipation, and the treated unit's counterfactual lies in the convex hull of a genuinely comparable donor pool. Good pre-period fit is necessary but does not guarantee the post-period counterfactual.
• Inference: placebo/permutation across donor units (the RMSPE ratio), not a naïve p-value; report how extreme the treated unit's gap is in the placebo distribution.

ML in service of a causal effect (double/debiased ML, causal forests, ML propensity)

The estimator changed, not the assumptions:

• Cross-fit nuisance models (outcome, propensity) on folds disjoint from where their predictions enter the moment condition, or the bias leaks back in.
• Report overlap/positivity and nuisance diagnostics (fit quality, propensity distribution) — ML propensities pushed to 0/1 are a design failure, not just a modeling detail; report more than the final θ and its SE.
• Use the orthogonalized/debiased score — never read the estimate off a plug-in coefficient from the ML fit.
• CATE heterogeneity ≠ a targeting license — deploying causal-forest scores needs the same unconfoundedness argument as the ATE, plus predictive-modeling's deployment-matched evaluation.
• The Design Card still applies — ML can't repair a design; unconfoundedness/exclusion still carries the estimate.

Bad controls — the quiet killer of reduced-form work

Adding a control can create bias as easily as remove it. The rule: only condition on variables determined before treatment — and even that is necessary, not sufficient: a pre-treatment collider (M-bias) or a bias-amplifying near-instrument is still a bad control. Every control needs a confounding story, not just a timestamp. A control that is itself an outcome of the treatment reopens the very confounding you're trying to close.

• Post-treatment controls / mediators. Controlling for a channel the treatment works through (e.g. "effect of education on wages, controlling for occupation") nets out part of the effect — usually toward zero, sometimes unpredictably. If it could plausibly have been affected by treatment, it is not a control.
• Colliders. Conditioning on (or selecting the sample on) a variable that both treatment and outcome cause induces a spurious association where none existed.
• Selection on the outcome. Filtering the sample on the dependent variable (or anything downstream of it) manufactures correlation.

"I added more controls and it got more robust" is not reassurance — more controls can mean more bias. Each control needs a reason it's pre-determined, not just a wish to be thorough.

Robustness, placebo, sensitivity — not optional

These are part of the estimate, not a courtesy — but robustness is an argument, not an inventory. "Mandatory" means the threat-relevant checks, not the whole per-design catalogue: three checks that probe the real threat beat thirty that probe nothing, and a sprawling robustness table reads as a tell of weak identification. Propose the ~3-check shortlist with rationales and get approval before running it — a checkpoint, not an autonomous fan-out (executing-analysis-plans, analysis-checkpoints).

• Placebo / falsification: an effect on an outcome that shouldn't be affected, or in a period before treatment, signals that the design is picking up confounding.
• Sensitivity to unobserved confounding: how strong would an omitted confounder have to be to overturn the result? Use Oster's δ, Rosenbaum bounds, or e-values — a result that flips under a mild plausible confounder is not robust.
• Specification stability: the effect shouldn't hinge on one control or one functional form (run the pre-committed suite from pre-analysis-plan).

Tooling (R / Julia / Python)

Julia: FixedEffectModels.jl covers FE/DiD and IV; little else has a mature implementation. When a stack lacks one (much of staggered-DiD and RDD outside R), say so — call out to R or implement explicitly rather than silently falling back to biased TWFE.

Red flags — STOP

• Reporting "the effect of X" from a regression with no named design and no stated counterfactual.
• A staggered-treatment DiD estimated with plain TWFE and no mention of the bias.
• An IV with no reported first-stage F, or treating LATE as if it were ATE.
• An RDD with no manipulation/density test.
• Matching that reports significance but never reports covariate balance or overlap.
• Controlling for variables that could have been affected by treatment (post-treatment controls / mediators / colliders).

Common rationalizations

The Process

1. Earn the estimate — design named, untestable assumption argued, testable diagnostics passed, modern estimator used, threat-relevant robustness/placebo/sensitivity survived, reconciled with the raw data.
2. If any diagnostic fails or the design needs to change → STOP (see "Choosing or changing the design" above — never a silent upgrade).
3. If the estimate has the wrong sign or magnitude → invoke wrong-number-debugging first — rule out a data bug before blaming identification.
4. Once the design holds → offer result-verification before any number leaves the building — the user's call; it confirms what already ran, it does not re-run or add robustness. Do not end at "the coefficient is X".