Spec Curve Sweep
The heavy sibling of robustness-battery. Use when the user wants the whole space mapped,
not five variants.
1. Enumerate researcher degrees of freedom
Read the primary estimation do-file and list every knob actually available:
- Clustering level (firm, industry, industry × year, two-way)
- Fixed-effect sets
- Control vectors (none / baseline / extended)
- Sample filters (rating bands, domestic-only, pre-COVID, balanced panel)
- Winsorization thresholds (none / 1% / 5%)
- Alternative outcome and treatment measures (fixed-pre vs. contemporaneous, levels vs. excess)
- Event-window lengths
2. Propose the grid before running it
Target 200–500 specifications. Present the cross-product and the exclusions: name every
combination ruled out as indefensible and say why (e.g., FE set that mechanically absorbs the
treatment, sample cut that leaves < 30 clusters). The user approves the grid before it runs.
3. Build the runner
- A parameterized driver do-file taking the knobs as arguments.
- A Python runner that executes the grid with parallelism, checkpoints each completed spec
to disk immediately, and is resumable — a killed run restarts from the checkpoint, not
from spec 1.
- One row per spec: all knob settings,
coef, se, t, p, n, n_clusters, status.
- Failed specs are recorded with their Stata error code, never dropped.
4. Launch in the background and monitor
Run detached; report the job ID. Check progress periodically rather than blocking.
5. Deliverables
- Specification curve figure — specs sorted by coefficient, significance shading, the
headline spec marked.
- Variance-decomposition table — which knob explains the most variation in the estimate.
- A short note naming the specifications where the result flips sign or loses significance.
- Compiled online-appendix
.tex containing the full grid.
- A PR with everything, plus a summary of what a hostile referee would seize on.
Honesty rule
Report fragility straight. Do not bury, footnote, or aggregate away the variants that hurt the
result — those are the ones the user needs before a referee finds them. If the result only
survives in a minority of defensible specs, that is the headline of the report.
1---2name: spec-curve-sweep3description: Use to map where a headline result lives and dies across the full space of defensible specifications — enumerates researcher degrees of freedom, runs a 200-500 spec grid in the background with checkpointing and resume, then produces a specification curve, a variance-decomposition table, a note on where the result flips, and a compiled online appendix. Trigger phrases include "specification curve", "overnight robustness sweep", "run every defensible spec", "how fragile is this result".4---56# Spec Curve Sweep78The heavy sibling of `robustness-battery`. Use when the user wants the whole space mapped,9not five variants.1011## 1. Enumerate researcher degrees of freedom1213Read the primary estimation do-file and list every knob actually available:1415- Clustering level (firm, industry, industry × year, two-way)16- Fixed-effect sets17- Control vectors (none / baseline / extended)18- Sample filters (rating bands, domestic-only, pre-COVID, balanced panel)19- Winsorization thresholds (none / 1% / 5%)20- Alternative outcome and treatment measures (fixed-pre vs. contemporaneous, levels vs. excess)21- Event-window lengths2223## 2. Propose the grid before running it2425Target 200–500 specifications. Present the cross-product **and the exclusions**: name every26combination ruled out as indefensible and say why (e.g., FE set that mechanically absorbs the27treatment, sample cut that leaves < 30 clusters). The user approves the grid before it runs.2829## 3. Build the runner3031- A **parameterized driver do-file** taking the knobs as arguments.32- A **Python runner** that executes the grid with parallelism, checkpoints each completed spec33 to disk immediately, and is **resumable** — a killed run restarts from the checkpoint, not34 from spec 1.35- One row per spec: all knob settings, `coef`, `se`, `t`, `p`, `n`, `n_clusters`, `status`.36- Failed specs are recorded with their Stata error code, never dropped.3738## 4. Launch in the background and monitor3940Run detached; report the job ID. Check progress periodically rather than blocking.4142## 5. Deliverables43441. **Specification curve figure** — specs sorted by coefficient, significance shading, the45 headline spec marked.462. **Variance-decomposition table** — which knob explains the most variation in the estimate.473. **A short note** naming the specifications where the result flips sign or loses significance.484. **Compiled online-appendix `.tex`** containing the full grid.495. **A PR** with everything, plus a summary of what a hostile referee would seize on.5051## Honesty rule5253Report fragility straight. Do not bury, footnote, or aggregate away the variants that hurt the54result — those are the ones the user needs before a referee finds them. If the result only55survives in a minority of defensible specs, that is the headline of the report.
zirui-song