Data Analysis (gec-data-analysis)
GEC reviewers span disciplines and are demanding about rigor on each tradition's own terms. The analysis
must deliver the test the framework (gec-conceptual-framework) and design (gec-research-design) set
up, and report it so a reader in another discipline can judge it. This skill covers execution and
reporting norms across methods.
When to trigger
- Running main and supporting analyses; building the results section
- A reviewer asked for robustness, heterogeneity, alternative measures, or qualitative depth
- Reconciling quantitative and qualitative strands in a mixed-methods paper
- Making the analysis reproducible before deposit (see
gec-submission)
Analysis norms GEC expects
- Report uncertainty honestly. Confidence/credible intervals and effect magnitudes, not just
stars; the substantive meaning of the estimate for the human/policy question.
- Robustness that probes, not decorates. Show specifications that could break the result
(alternative measures, samples, estimators, fixed effects, scale of aggregation), and say what you learn.
- Heterogeneity with discipline. Pre-specify subgroups where possible (e.g., by vulnerability,
region, income); correct for multiple comparisons; avoid mining and post-hoc theorizing.
- Right inference. Cluster at the assignment/sampling level; address spatial autocorrelation;
use small-cluster corrections when clusters are few.
- Measurement. Validate constructs (vulnerability indices, governance measures, attitude scales);
report reliability; show results are not an artifact of a coding/scaling/weighting choice.
Qualitative & mixed-methods analysis
- For qualitative work, show the coding scheme, intercoder agreement where relevant, and evidence
trail from data to interpretation; quote enough to let readers judge.
- For mixed methods, present the integration (joint displays, triangulation) and interpret
convergence and divergence — do not report two parallel papers.
Reproducibility while you work (not at the end)
- One master script / documented workflow regenerates every table and figure from the data.
- Set and report seeds for any stochastic step (bootstrap, simulation, randomization inference).
- Pin software/package versions (
renv.lock, requirements.txt, recorded installs).
- Prepare the Data Availability Statement and a clean archive as you go (see
gec-submission).
Anti-patterns
- Stars-only tables with no effect sizes or intervals
- "Robustness" that only reruns near-identical specs to manufacture stability
- p-hacking / fishing for a significant interaction; HARKing exploratory results into hypotheses
- Qualitative claims with no visible evidence trail; mixed methods that never integrate
- A results section whose numbers the workflow cannot reproduce
What interdisciplinary GEC referees check, by strand
GEC sends papers to reviewers from more than one tradition, so the analysis is judged twice — once for technical correctness and once for whether the human-dimensions claim is earned. Use this as a self-audit before submission.
| Strand |
The reviewer's first probe |
Pass signal at GEC |
| Quantitative |
"Is the inference level right, and does the effect mean anything socially?" |
Clustered/spatially-corrected SEs plus a sentence translating the coefficient into adaptation, exposure, or equity terms |
| Qualitative |
"Can I trace a claim back to the data?" |
Visible coding scheme, an evidence trail, and quotes that let the reader adjudicate |
| Mixed methods |
"Did the strands actually meet?" |
A joint display where convergence and divergence are both interpreted, not two appendices |
| Measurement |
"Is the vulnerability/governance index an artifact?" |
Reliability reported and the result shown stable to an alternative scaling or weighting |
Worked micro-example (illustrative — land-use change drivers)
A global analysis regresses forest-loss rates on a governance-quality index and commodity-price exposure across districts. Numbers are illustrative.
- Thin version: a stars-only table reporting "governance significant at p<0.01" and a closing claim that "better governance reduces deforestation."
- GEC-rigorous version: reports the magnitude — a one-SD rise in governance quality is associated with 0.8 percentage points (95% CI 0.3–1.3, illustrative) lower annual forest loss — clusters SEs at the administrative unit, and shows the estimate survives dropping the top commodity-exporting decile and re-weighting the index. Heterogeneity by tenure regime is pre-specified and Holm-adjusted, so the one significant interaction is not mined.
- Why it clears review: the magnitude is socially interpretable, the robustness probes could have broken it, and the human-dimensions reading (institutions, not just biophysics) is licensed by the design.
Referee-pushback patterns and the fix
- "Robustness only reruns near-identical specs" → swap in specifications that could plausibly overturn the result (different aggregation scale, alternative measure) and report what you learned, including nulls.
- "The qualitative claims have no visible evidence trail" → expose the coding scheme and quote enough raw material for the reader to judge the inference.
- "This is two parallel papers, not mixed methods" → build one joint display and interpret where the strands disagree.
Calibration anchors (hedged)
- Honest-uncertainty bar: intervals and effect sizes, not significance stars alone, are the GEC norm; a magnitude a policy reader can act on beats a p-value.
- Triangulation bar: convergence across methods strengthens a claim; unexplained divergence is a finding, not an embarrassment to hide.
- Confirm the journal's current research-data and reproducibility expectations against its author guidelines before deposit, as Elsevier policy detail evolves.
Output format
【Main result】magnitude + interval (or qualitative finding) + substantive meaning
【Identification / evidence check】(per research-design) result
【Robustness】specs that could break it → what held
【Heterogeneity】pre-specified? MHT-adjusted?
【Integration】(mixed methods) convergence/divergence interpreted?
【Reproducible】workflow + seeds + pinned versions? [Y/N]
【Next】gec-figures-and-tables
Supplementary resources
1---2name: gec-data-analysis3description: Use when executing and reporting the analysis for a Global Environmental Change (GEC) manuscript so it survives expert, interdisciplinary review — honest uncertainty, robustness, and triangulation appropriate to quantitative, qualitative, or mixed-methods work. Guides analysis norms; it does not fabricate results.4---56# Data Analysis (gec-data-analysis)78GEC reviewers span disciplines and are demanding about rigor on each tradition's own terms. The analysis9must deliver the test the framework (`gec-conceptual-framework`) and design (`gec-research-design`) set10up, and report it so a reader in another discipline can judge it. This skill covers execution and11reporting norms across methods.1213## When to trigger1415- Running main and supporting analyses; building the results section16- A reviewer asked for robustness, heterogeneity, alternative measures, or qualitative depth17- Reconciling quantitative and qualitative strands in a mixed-methods paper18- Making the analysis reproducible before deposit (see `gec-submission`)1920## Analysis norms GEC expects21221. **Report uncertainty honestly.** Confidence/credible intervals and effect magnitudes, not just23 stars; the substantive meaning of the estimate for the human/policy question.242. **Robustness that probes, not decorates.** Show specifications that could *break* the result25 (alternative measures, samples, estimators, fixed effects, scale of aggregation), and say what you learn.263. **Heterogeneity with discipline.** Pre-specify subgroups where possible (e.g., by vulnerability,27 region, income); correct for multiple comparisons; avoid mining and post-hoc theorizing.284. **Right inference.** Cluster at the assignment/sampling level; address spatial autocorrelation;29 use small-cluster corrections when clusters are few.305. **Measurement.** Validate constructs (vulnerability indices, governance measures, attitude scales);31 report reliability; show results are not an artifact of a coding/scaling/weighting choice.3233## Qualitative & mixed-methods analysis34- For qualitative work, show the **coding scheme, intercoder agreement where relevant, and evidence35 trail** from data to interpretation; quote enough to let readers judge.36- For mixed methods, present the **integration** (joint displays, triangulation) and interpret37 convergence *and* divergence — do not report two parallel papers.3839## Reproducibility while you work (not at the end)40- One **master script / documented workflow** regenerates every table and figure from the data.41- **Set and report seeds** for any stochastic step (bootstrap, simulation, randomization inference).42- Pin software/package versions (`renv.lock`, `requirements.txt`, recorded installs).43- Prepare the **Data Availability Statement** and a clean archive as you go (see `gec-submission`).4445## Anti-patterns4647- Stars-only tables with no effect sizes or intervals48- "Robustness" that only reruns near-identical specs to manufacture stability49- p-hacking / fishing for a significant interaction; HARKing exploratory results into hypotheses50- Qualitative claims with no visible evidence trail; mixed methods that never integrate51- A results section whose numbers the workflow cannot reproduce5253## What interdisciplinary GEC referees check, by strand5455GEC sends papers to reviewers from more than one tradition, so the analysis is judged twice — once for technical correctness and once for whether the human-dimensions claim is earned. Use this as a self-audit before submission.5657| Strand | The reviewer's first probe | Pass signal at GEC |58|--------|---------------------------|--------------------|59| Quantitative | "Is the inference level right, and does the effect mean anything socially?" | Clustered/spatially-corrected SEs plus a sentence translating the coefficient into adaptation, exposure, or equity terms |60| Qualitative | "Can I trace a claim back to the data?" | Visible coding scheme, an evidence trail, and quotes that let the reader adjudicate |61| Mixed methods | "Did the strands actually meet?" | A joint display where convergence and divergence are both interpreted, not two appendices |62| Measurement | "Is the vulnerability/governance index an artifact?" | Reliability reported and the result shown stable to an alternative scaling or weighting |6364## Worked micro-example (illustrative — land-use change drivers)6566A global analysis regresses forest-loss rates on a governance-quality index and commodity-price exposure across districts. Numbers are illustrative.6768- **Thin version:** a stars-only table reporting "governance significant at p<0.01" and a closing claim that "better governance reduces deforestation."69- **GEC-rigorous version:** reports the magnitude — a one-SD rise in governance quality is associated with 0.8 percentage points (95% CI 0.3–1.3, illustrative) lower annual forest loss — clusters SEs at the administrative unit, and shows the estimate survives dropping the top commodity-exporting decile and re-weighting the index. Heterogeneity by tenure regime is pre-specified and Holm-adjusted, so the one significant interaction is not mined.70- **Why it clears review:** the magnitude is socially interpretable, the robustness probes could have broken it, and the human-dimensions reading (institutions, not just biophysics) is licensed by the design.7172## Referee-pushback patterns and the fix7374- *"Robustness only reruns near-identical specs"* → swap in specifications that could plausibly overturn the result (different aggregation scale, alternative measure) and report what you learned, including nulls.75- *"The qualitative claims have no visible evidence trail"* → expose the coding scheme and quote enough raw material for the reader to judge the inference.76- *"This is two parallel papers, not mixed methods"* → build one joint display and interpret where the strands disagree.7778## Calibration anchors (hedged)7980- **Honest-uncertainty bar:** intervals and effect sizes, not significance stars alone, are the GEC norm; a magnitude a policy reader can act on beats a p-value.81- **Triangulation bar:** convergence across methods strengthens a claim; unexplained divergence is a finding, not an embarrassment to hide.82- Confirm the journal's current research-data and reproducibility expectations against its author guidelines before deposit, as Elsevier policy detail evolves.8384## Output format8586```87【Main result】magnitude + interval (or qualitative finding) + substantive meaning88【Identification / evidence check】(per research-design) result89【Robustness】specs that could break it → what held90【Heterogeneity】pre-specified? MHT-adjusted?91【Integration】(mixed methods) convergence/divergence interpreted?92【Reproducible】workflow + seeds + pinned versions? [Y/N]93【Next】gec-figures-and-tables94```9596## Supplementary resources9798- [`../../resources/external_tools.md`](../../resources/external_tools.md) — estimation, inference, qualitative, and mixed-methods tools99- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — Elsevier research-data policy notes
brycewang-stanford