Adaptive Causal Estimate

Very Simple CAIS Skill

Given a CSV dataset, a dataset description, and a natural-language causal query: select the right causal method, run the appropriate tools, and return a numeric ATE estimate with reasoning.

When to Use

Use this skill when the user provides a tabular dataset (CSV) and a causal question such as "What is the effect of X on Y?" Do not use for pure correlation analysis, prediction tasks, or causal graph discovery.

Tools Available

All tools live at ~/.claude/skills/adaptive_causal_estimate/tools/. Always call them via Bash:

python - <<'PYEOF'
import os, sys, json
sys.path.insert(0, os.path.expanduser("~/.claude/skills/adaptive_causal_estimate"))
from tools.data import analyze_dataset
result = analyze_dataset("/path/to/data.csv")
print(json.dumps(result, default=str))
PYEOF

Rule: always call at least one estimator tool. Do not compute ATE by hand.

Domain Knowledge: Method Selection

Step 1 — RCT check (do this first, before any tool call)

Scan the dataset description and query for: randomized, randomised, RCT, randomized controlled trial, randomized study, randomly assigned, random assignment.

• If any keyword matches → is_rct = True
• When is_rct = True: matching, PSM, and IPW are forbidden. Do not run them, do not consider them. OLS with covariates is the correct choice (or difference-in-means if no covariates). Random assignment already balances covariates — applying matching on top is wrong and will produce an incorrect selected_method.

Step 2 — Study design identification

Call analyze_dataset to inspect column structure, then apply this decision logic:

Step 3 — Observational data: matching vs OLS

This is the most common error source. Use these rules:

• Call check_covariate_balance first. If max_smd < 0.1 (already well-balanced), prefer OLS — matching adds noise without benefit.
• Call check_overlap. If overlap_score ≥ 0.8, matching is viable. If overlap_score < 0.5, use IPW or OLS instead.
• Matching is appropriate when: selection bias is strong (high SMD pre-matching), treatment is binary, and overlap is good.
• OLS is appropriate when: confounders are few and well-measured, or when the question asks for ATE (not ATT).
• Never choose matching just because the dataset has covariates. Covariates alone do not justify matching over OLS.

Domain Knowledge: Method Assumptions

OLS

• Assumes no unmeasured confounding given covariates (conditional ignorability)
• Use HC1 robust standard errors (already done by the tool)
• Valid for both RCT (precision gain) and observational (with strong ignorability assumption)

Difference-in-Means

• Only for pure RCTs with no covariate adjustment needed
• Equivalent to OLS without covariates

DiD

• Requires: panel structure (unit × time), binary treatment group indicator, known treatment timing
• Key assumption: parallel trends — treated and control groups would have followed the same trend absent treatment
• Always run test_parallel_trends if ≥ 3 pre-treatment periods exist
• Treatment timing = first period when treated group received treatment

IV / 2SLS

• Instrument must satisfy: (1) relevance — correlated with treatment (F ≥ 10), (2) exclusion restriction — only affects outcome through treatment, (3) exogeneity — uncorrelated with unmeasured confounders
• Always run test_instrument_strength — if F < 10, instrument is weak, IV estimate is biased, fall back to OLS
• IV estimates LATE (Local ATE), not ATE — only for compliers

RDD

• Running variable must exist: a continuous variable where crossing a threshold determines treatment
• Estimate is local — valid only near the cutoff
• Units should not be able to precisely manipulate the running variable
• Bandwidth choice matters — try default (10% of range), then check n_in_bandwidth is adequate (≥ 20)

Matching / PSM

• Estimates ATT (effect on treated), not ATE
• Requires: binary treatment, observed covariates capture all confounding (no unmeasured confounders)
• Check overlap before running: if overlap is poor, PSM matches on poorly supported regions and estimates are unreliable
• Post-matching balance: max_smd_post < 0.1 is excellent, < 0.25 acceptable
• Forbidden when is_rct = True

Domain Knowledge: Common Pitfalls

• RCT + matching error: Dataset says "randomized study" → agent incorrectly runs PSM because covariates exist and PSM converges well. Fix: is_rct = True immediately eliminates matching as a candidate.
• IV over-selection: Wage/education datasets often have a plausible IV (father's education, quarter of birth). This is methodologically reasonable but benchmark labels may use OLS. When both IV and OLS are defensible, prefer the method better supported by the description.
• DiD without panel: DiD requires multiple observations per unit over time. A single cross-section with a "before/after" question is not DiD.
• Matching without overlap check: Running PSM without check_overlap first risks matching on a region of no common support. Always check overlap before matching.
• selected_method field must match the estimator tool actually called: if you called estimate_ols, write "ols"; if you called estimate_psm, write "matching". Never write the method you considered but rejected.

Output Format

Respond with a JSON object:

{
  "selected_method": "<ols|matching|iv|did|rdd>",
  "ate": <numeric estimate>,
  "reasoning": "<method choice rationale + key assumption check result>"
}

selected_method must be one of: ols, matching, iv, did, rdd.