Research Design & Methods (amj-methods)
When to trigger
- The design may not match the theory's level, timing, or causal claim
- Data are single-source, single-wave, and self-reported (common-method bias risk)
- The theory is causal but the design is cross-sectional/correlational
- Constructs lack established, validated measures
- A reviewer says "the design cannot test this hypothesis" or "endogeneity is unaddressed"
Match the design to the question
AMJ explicitly welcomes all empirical methods — qualitative, quantitative, field, laboratory, meta-analytic, and mixed. The bar is fit and rigor, not a single preferred method, and qualitative designs are held to an equally demanding standard (the Eisenhardt multiple-case approach and the Gioia methodology for grounded qualitative rigor are the field's reference points).
| Theoretical claim |
Design that earns it |
| Causal effect of a manipulable cause |
Experiment (lab/field/online), or natural experiment |
| Process unfolding over time |
Multi-wave panel; longitudinal/lagged design |
| Firm/strategy outcomes from archival cause |
Panel archival with fixed effects + endogeneity strategy |
| Cross-level mechanism (e.g., team→indiv.) |
Multilevel/nested data with HLM-appropriate structure |
| Rich, novel, or contested phenomenon |
Qualitative or multi-method (often paired with a study 2) |
A two-study design (e.g., field study for generalizability + experiment for causal mechanism) is a common AMJ strength — it answers both internal and external validity.
Designing against the threats AMJ cares about
- Common-method bias (CMB): separate sources for predictor and outcome; temporal separation across waves; objective/archival outcomes where possible. Procedural remedies beat statistical fixes (the Podsakoff et al. guidance is the standard reference). Plan this before collecting data.
- Endogeneity (archival): anticipate omitted variables, reverse causality, and selection. Plan an identification strategy (instrument, natural experiment, panel fixed effects, difference-in-differences, Heckman/2SLS, propensity matching) and the assumptions each requires.
- Measurement: use validated multi-item scales; pilot new measures; plan a CFA. State the level at which each construct is measured and how cross-level data are aggregated (with justification: ICC, r_wg, aggregation theory).
- Sampling and power: justify the sampling frame, response rate, and statistical power for the focal and interaction effects (interactions need more power).
Level-of-analysis discipline
State the level for theory, measurement, and analysis, and keep them aligned. If theory is at the team level but data are individual, justify aggregation; if effects are cross-level, the analysis must model the nesting (do not run OLS on nested data).
Execution bridge (StatsPAI / Stata MCP)
For the empirical / causal lane, estimate and audit rather than only specify. Full
map: execution-with-mcp. AMJ is empirical management — panel, multilevel, DiD, IV, and field/lab experiments; the chain below serves that lane, while grounded-theory / qualitative work uses its own standards.
detect_design → recommend → fit with as_handle=true → audit_result to
enumerate the checks the design owes.- Panel / staggered DiD:
callaway_santanna / sun_abraham + bacon_decomposition
honest_did_from_result. IV: effective_f_test + anderson_rubin_ci. RDD:
rdrobust + mccrary_test.
- Experiments: randomization-based inference and
romano_wolf for the many-outcome
family-wise correction reviewers expect.
Match the toolchain to the reviewer pool, and report the effect size the venue
wants. A run end-to-end (synthetic data, real returns) is in the
JF execution walkthrough.
Checklist
Anti-patterns
- Cross-sectional causal claims: "X causes Y" from one-wave correlational data.
- CMB as afterthought: relying solely on a Harman single-factor test instead of designed separation.
- Ignored endogeneity: archival "effect" with an obviously endogenous regressor and no strategy.
- Mismatched levels: theorizing at the team level, testing with disaggregated individual data via OLS.
- Unvalidated home-grown scales with no evidence of reliability or construct validity.
- Underpowered interactions presented as null "boundary conditions."
Output format
【Design】experiment / panel-archival / multilevel survey / qualitative / multi-method
【Hypothesis-design fit】each H testable? notes ...
【CMB plan】procedural remedies ...
【Endogeneity strategy】(if archival) instrument / NE / FE / DiD / matching ...
【Measures】validated? new (piloted)? CFA planned?
【Levels】theory / measurement / analysis aligned? aggregation justification ...
【Power & sampling】frame, N, power for interactions ...
【Next step】amj-data-analysis
Source: brycewang-stanford/Awesome-Journal-Skills → Academy-of-Management-Journal-Skills/skills/amj-methods/SKILL.md
1---2name: amj-methods3description: Use when the research design and method are the bottleneck for an Academy of Management Journal (AMJ) manuscript — matching design (archival, survey, experiment, multi-method, field) and level of analysis to the theoretical question. Designs the study; it does not run the estimation or validity checks (amj-data-analysis).4---5
6
7# Research Design & Methods (amj-methods)
8
9## When to trigger
10
11- The design may not match the theory's level, timing, or causal claim
12- Data are single-source, single-wave, and self-reported (common-method bias risk)
13- The theory is causal but the design is cross-sectional/correlational
14- Constructs lack established, validated measures
15- A reviewer says "the design cannot test this hypothesis" or "endogeneity is unaddressed"
16
17## Match the design to the question
18
19AMJ explicitly welcomes **all empirical methods** — qualitative, quantitative, field, laboratory, meta-analytic, and mixed. The bar is *fit and rigor*, not a single preferred method, and qualitative designs are held to an equally demanding standard (the Eisenhardt multiple-case approach and the Gioia methodology for grounded qualitative rigor are the field's reference points).
20
21| Theoretical claim | Design that earns it |
22|--------------------------------------------|----------------------------------------------------------|
23| Causal effect of a manipulable cause | Experiment (lab/field/online), or natural experiment |
24| Process unfolding over time | Multi-wave panel; longitudinal/lagged design |
25| Firm/strategy outcomes from archival cause | Panel archival with fixed effects + endogeneity strategy |
26| Cross-level mechanism (e.g., team→indiv.) | Multilevel/nested data with HLM-appropriate structure |
27| Rich, novel, or contested phenomenon | Qualitative or multi-method (often paired with a study 2)|
28
29A two-study design (e.g., field study for generalizability + experiment for causal mechanism) is a common AMJ strength — it answers both internal and external validity.
30
31## Designing against the threats AMJ cares about
32
33- **Common-method bias (CMB)**: separate sources for predictor and outcome; temporal separation across waves; objective/archival outcomes where possible. Procedural remedies beat statistical fixes (the Podsakoff et al. guidance is the standard reference). Plan this *before* collecting data.
34- **Endogeneity (archival)**: anticipate omitted variables, reverse causality, and selection. Plan an identification strategy (instrument, natural experiment, panel fixed effects, difference-in-differences, Heckman/2SLS, propensity matching) and the assumptions each requires.
35- **Measurement**: use validated multi-item scales; pilot new measures; plan a CFA. State the level at which each construct is measured and how cross-level data are aggregated (with justification: ICC, r_wg, aggregation theory).
36- **Sampling and power**: justify the sampling frame, response rate, and statistical power for the focal and interaction effects (interactions need more power).
37
38## Level-of-analysis discipline
39
40State the level for theory, measurement, and analysis, and keep them aligned. If theory is at the team level but data are individual, justify aggregation; if effects are cross-level, the analysis must model the nesting (do not run OLS on nested data).
41
42## Execution bridge (StatsPAI / Stata MCP)
43
44For the **empirical / causal lane**, estimate and audit rather than only specify. Full
45map: [`execution-with-mcp`](../../../shared-resources/empirical-methods/execution-with-mcp.md). AMJ is empirical management — panel, multilevel, DiD, IV, and field/lab experiments; the chain below serves that lane, while grounded-theory / qualitative work uses its own standards.
46
47- `detect_design` → `recommend` → fit with `as_handle=true` → `audit_result` to
48 enumerate the checks the design owes.
49- **Panel / staggered DiD:** `callaway_santanna` / `sun_abraham` + `bacon_decomposition`
50 + `honest_did_from_result`. **IV:** `effective_f_test` + `anderson_rubin_ci`. **RDD:**
51 `rdrobust` + `mccrary_test`.
52- **Experiments:** randomization-based inference and `romano_wolf` for the many-outcome
53 family-wise correction reviewers expect.
54
55Match the toolchain to the **reviewer pool**, and report the effect size the venue
56wants. A run end-to-end (synthetic data, real returns) is in the
57[JF execution walkthrough](../../../Journal-of-Finance-Skills/resources/worked-examples/02-execution-walkthrough.md).
58## Checklist
59
60- [ ] Design can actually test each hypothesis (causal claims have causal leverage)
61- [ ] CMB addressed by procedural design (separate sources/time), not just a post-hoc test
62- [ ] Endogeneity strategy specified for archival/observational causal claims
63- [ ] Constructs use validated measures; new measures piloted; CFA planned
64- [ ] Level of analysis consistent across theory, measurement, and analysis; aggregation justified
65- [ ] Sampling frame, response rate, and power (including for interactions) justified
66- [ ] Where feasible, a second study triangulates the causal mechanism
67
68## Anti-patterns
69
70- **Cross-sectional causal claims**: "X causes Y" from one-wave correlational data.
71- **CMB as afterthought**: relying solely on a Harman single-factor test instead of designed separation.
72- **Ignored endogeneity**: archival "effect" with an obviously endogenous regressor and no strategy.
73- **Mismatched levels**: theorizing at the team level, testing with disaggregated individual data via OLS.
74- **Unvalidated home-grown scales** with no evidence of reliability or construct validity.
75- **Underpowered interactions** presented as null "boundary conditions."
76
77## Output format
78
79```
80【Design】experiment / panel-archival / multilevel survey / qualitative / multi-method
81【Hypothesis-design fit】each H testable? notes ...
82【CMB plan】procedural remedies ...
83【Endogeneity strategy】(if archival) instrument / NE / FE / DiD / matching ...
84【Measures】validated? new (piloted)? CFA planned?
85【Levels】theory / measurement / analysis aligned? aggregation justification ...
86【Power & sampling】frame, N, power for interactions ...
87【Next step】amj-data-analysis
88```
89
90---
91
92**Source:** [`brycewang-stanford/Awesome-Journal-Skills`](https://github.com/brycewang-stanford/Awesome-Journal-Skills) → `Academy-of-Management-Journal-Skills/skills/amj-methods/SKILL.md`
thedixitjain