Research Design (sf-research-design)
Social Forces is known for methodological rigor, and its reviewers are demanding about each
tradition. The design must credibly connect the argument (sf-theory-building) to evidence and rule
out the strongest alternative. This skill is mode-aware: pick the section that matches your work and
defend it on its own terms.
When to trigger
- Specifying identification, demographic design, case selection, or a network/computational pipeline
- A reviewer questioned causal claims, case choice, measurement, or a confound
- Deciding what design buys the cleanest test of your mechanism within a tight word budget
- Justifying why your design adjudicates the rival account from
sf-literature-positioning
Quantitative / causal inference
- Identification first. State the estimand and the assumptions that license a causal reading
(ignorability, parallel trends, exclusion, continuity). Defend them, don't assert them.
- Designs: panel/fixed effects, DID/event study (use modern staggered-adoption estimators, not
naive TWFE), IV (first-stage strength, exclusion, weak-IV-robust inference), RDD (density tests,
bandwidth robustness), matching/weighting with balance + sensitivity.
- Inference: cluster at the level of treatment/sampling; account for complex survey designs and
weights; correct for multiple comparisons when testing many implications.
- Sensitivity: how strong must an unobserved confounder be to overturn the result?
Demographic
- Be explicit about period vs. cohort, exposure, and standardization/decomposition choices.
- Handle censoring and competing risks correctly in event-history work; justify the hazard form.
Comparative-historical / ethnographic
- Case selection justified by design logic (typical, deviant, most/least-likely, paired
comparison) — not convenience. Say what the case is a case of.
- Process tracing with explicit tests (hoop, smoking-gun, straw-in-the-wind); state what evidence
would have disconfirmed the argument.
- Source transparency: archives, interviews, fieldnotes — plan how they will be documented and
cited (see
sf-data-and-transparency).
Network / computational
- Justify boundary specification, tie definition, and the null/baseline you compare against.
- Validate any computational measure (e.g., classifier, topic model) against human-labeled samples.
The adjudication test (SF-specific)
For the single strongest rival explanation, write one sentence: "If the rival were true rather
than my argument, the data would look like ___; instead they look like ___." If you cannot, the
design does not yet identify the contribution.
Execution bridge (StatsPAI / Stata MCP)
Estimate and audit the design, don't only describe it. Full map:
execution-with-mcp. Social Forces is quantitative sociology — survey and administrative panels; emphasize identification, decomposition, and multilevel inference.
detect_design → recommend → fit with as_handle=true → audit_result.
- Observational causal claims: staggered DiD (
callaway_santanna / sun_abraham +
bacon_decomposition + honest_did_from_result); IV (effective_f_test +
anderson_rubin_ci); RDD (rdrobust + mccrary_test).
- Experiments: randomization-based inference,
romano_wolf for many-outcome
family-wise control, and mediate for mediation (not naive controlling-away).
- Sensitivity:
oster_delta / sensemakr for observational claims.
Report the effect size in interpretable units; route the full battery to the
appendix/supplement. A run end-to-end (synthetic data, real returns) is in the
JF execution walkthrough.
Anti-patterns
- Naive TWFE on staggered treatment; clustering at the wrong level
- "Causal" language on a design that only supports association
- Convenience case selection dressed up as theory-driven
- An unvalidated computational measure treated as ground truth
- A design that cannot distinguish your argument from the leading alternative
What SF referees probe in the design
Because Social Forces built its standing on methodological rigor across a broad discipline, its referees
read the design for whether the strongest alternative is actually ruled out — not whether the method is
fashionable. A practical gate by mode:
| Design mode |
The check an SF referee runs first |
Common decline trigger |
| Quant-causal |
Estimand stated and key assumption defended? |
"Causal" verbs on an associational design |
| Panel / DID |
Modern staggered estimator + parallel-trends evidence? |
Naive TWFE on staggered adoption |
| Demographic |
Period vs. cohort, exposure, standardization explicit? |
Rates compared without standardization |
| Comparative-historical |
Case justified as a case of something? |
Convenience case dressed as theory-driven |
| Network / computational |
Boundary, tie definition, validated measure vs. a null? |
Classifier output as ground truth |
Calibration (hedged): SF welcomes all these traditions, but the bar is rigor on the tradition's own
terms plus general-sociology significance — less theory-maximalist than AJS/ASR yet far stricter on
identification than a descriptive outlet. Confirm method-specific expectations against current practice.
Worked vignette (illustrative)
A neighborhoods-and-attainment study uses a sibling comparison: children in one family exposed to
different neighborhood poverty via a mid-childhood move. Movers to lower-poverty tracts show a 0.12 SD
test-score gain (illustrative). SF-grade adjudication: "If the effect were pure selection it should
vanish within families; instead the within-family estimate is 0.09 SD, so selection explains at most a
quarter." Pairing this with an Oster-style sensitivity bound moves an SF referee from skeptic to advocate.
Referee-pushback patterns and the SF fix
- "Selection threatens the inference" → add a within-unit comparison or sensitivity bound.
- "Mechanism under-specified" → state the observable implication tested and what would have disconfirmed it.
- "Clustering at the wrong level" → cluster at treatment/sampling level; wild-cluster bootstrap if few.
Output format
【Mode】quant-causal / demographic / comparative-historical / ethnographic / network-computational
【Estimand or claim】what is being identified/shown
【Key assumption(s)】and how each is defended
【Rival ruled out】the adjudication sentence
【Robustness/sensitivity】planned checks
【Next】sf-data-analysis
Supplementary resources
1---2name: sf-research-design3description: Use when defending the research design of a Social Forces (SF) manuscript — causal identification for quantitative work, formal demographic design, case selection and process tracing for comparative-historical and ethnographic work, network and computational designs. SF's reputation rests on methodological rigor. Strengthens the design; it does not write code.4metadata:5 source: https://skillmd.com/skills/brycewang-stanford/sf-research-design6---78# Research Design (sf-research-design)910Social Forces is known for **methodological rigor**, and its reviewers are demanding about each11tradition. The design must credibly connect the argument (`sf-theory-building`) to evidence and rule12out the strongest alternative. This skill is mode-aware: pick the section that matches your work and13defend it on its own terms.1415## When to trigger1617- Specifying identification, demographic design, case selection, or a network/computational pipeline18- A reviewer questioned causal claims, case choice, measurement, or a confound19- Deciding what design buys the cleanest test of your mechanism within a tight word budget20- Justifying why your design adjudicates the rival account from `sf-literature-positioning`2122## Quantitative / causal inference23- **Identification first.** State the estimand and the assumptions that license a causal reading24 (ignorability, parallel trends, exclusion, continuity). Defend them, don't assert them.25- **Designs**: panel/fixed effects, DID/event study (use modern staggered-adoption estimators, not26 naive TWFE), IV (first-stage strength, exclusion, weak-IV-robust inference), RDD (density tests,27 bandwidth robustness), matching/weighting with balance + sensitivity.28- **Inference**: cluster at the level of treatment/sampling; account for complex survey designs and29 weights; correct for multiple comparisons when testing many implications.30- **Sensitivity**: how strong must an unobserved confounder be to overturn the result?3132## Demographic33- Be explicit about period vs. cohort, exposure, and standardization/decomposition choices.34- Handle censoring and competing risks correctly in event-history work; justify the hazard form.3536## Comparative-historical / ethnographic37- **Case selection** justified by design logic (typical, deviant, most/least-likely, paired38 comparison) — not convenience. Say what the case is a case *of*.39- **Process tracing** with explicit tests (hoop, smoking-gun, straw-in-the-wind); state what evidence40 would have **disconfirmed** the argument.41- **Source transparency**: archives, interviews, fieldnotes — plan how they will be documented and42 cited (see `sf-data-and-transparency`).4344## Network / computational45- Justify boundary specification, tie definition, and the null/baseline you compare against.46- Validate any computational measure (e.g., classifier, topic model) against human-labeled samples.4748## The adjudication test (SF-specific)4950For the **single strongest rival explanation**, write one sentence: *"If the rival were true rather51than my argument, the data would look like ___; instead they look like ___."* If you cannot, the52design does not yet identify the contribution.5354## Execution bridge (StatsPAI / Stata MCP)5556Estimate and audit the design, don't only describe it. Full map:57[`execution-with-mcp`](../../../shared-resources/empirical-methods/execution-with-mcp.md). Social Forces is quantitative sociology — survey and administrative panels; emphasize identification, decomposition, and multilevel inference.5859- `detect_design` → `recommend` → fit with `as_handle=true` → `audit_result`.60- **Observational causal claims:** staggered DiD (`callaway_santanna` / `sun_abraham` +61 `bacon_decomposition` + `honest_did_from_result`); IV (`effective_f_test` +62 `anderson_rubin_ci`); RDD (`rdrobust` + `mccrary_test`).63- **Experiments:** randomization-based inference, `romano_wolf` for many-outcome64 family-wise control, and `mediate` for mediation (not naive controlling-away).65- **Sensitivity:** `oster_delta` / `sensemakr` for observational claims.6667Report the effect size in interpretable units; route the full battery to the68appendix/supplement. A run end-to-end (synthetic data, real returns) is in the69[JF execution walkthrough](../../../Journal-of-Finance-Skills/resources/worked-examples/02-execution-walkthrough.md).70## Anti-patterns7172- Naive TWFE on staggered treatment; clustering at the wrong level73- "Causal" language on a design that only supports association74- Convenience case selection dressed up as theory-driven75- An unvalidated computational measure treated as ground truth76- A design that cannot distinguish your argument from the leading alternative7778## What SF referees probe in the design7980Because Social Forces built its standing on methodological rigor across a broad discipline, its referees81read the design for whether the strongest alternative is actually ruled out — not whether the method is82fashionable. A practical gate by mode:8384| Design mode | The check an SF referee runs first | Common decline trigger |85|-------------|-----------------------------------|------------------------|86| Quant-causal | Estimand stated and key assumption defended? | "Causal" verbs on an associational design |87| Panel / DID | Modern staggered estimator + parallel-trends evidence? | Naive TWFE on staggered adoption |88| Demographic | Period vs. cohort, exposure, standardization explicit? | Rates compared without standardization |89| Comparative-historical | Case justified as a case *of* something? | Convenience case dressed as theory-driven |90| Network / computational | Boundary, tie definition, validated measure vs. a null? | Classifier output as ground truth |9192Calibration (hedged): SF welcomes all these traditions, but the bar is rigor on the tradition's own93terms plus general-sociology significance — less theory-maximalist than AJS/ASR yet far stricter on94identification than a descriptive outlet. Confirm method-specific expectations against current practice.9596## Worked vignette (illustrative)9798A neighborhoods-and-attainment study uses a sibling comparison: children in one family exposed to99different neighborhood poverty via a mid-childhood move. Movers to lower-poverty tracts show a 0.12 SD100test-score gain (illustrative). SF-grade adjudication: "If the effect were pure selection it should101vanish within families; instead the within-family estimate is 0.09 SD, so selection explains at most a102quarter." Pairing this with an Oster-style sensitivity bound moves an SF referee from skeptic to advocate.103104## Referee-pushback patterns and the SF fix105106- "Selection threatens the inference" → add a within-unit comparison or sensitivity bound.107- "Mechanism under-specified" → state the observable implication tested and what would have disconfirmed it.108- "Clustering at the wrong level" → cluster at treatment/sampling level; wild-cluster bootstrap if few.109110## Output format111112```113【Mode】quant-causal / demographic / comparative-historical / ethnographic / network-computational114【Estimand or claim】what is being identified/shown115【Key assumption(s)】and how each is defended116【Rival ruled out】the adjudication sentence117【Robustness/sensitivity】planned checks118【Next】sf-data-analysis119```120121## Supplementary resources122123- [`../../resources/external_tools.md`](../../resources/external_tools.md) — design/identification packages (R/Stata/Python), demography, networks, CAQDAS/QCA124- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — SF rigor reputation and scope