Meta-Analysis — Diagnostics & Primary Pooled Estimation
Table of Contents
Open-source skill.
Scope Boundary
Use this skill when:
- The inputs are study-level summary statistics and the first need is a transparent pooled estimate with heterogeneity diagnostics.
- A one-effect-size-at-a-time meta-analysis is appropriate before moderator, bias, or multilevel extensions.
Do not use this skill when:
- The project is network meta-analysis, IPD meta-analysis, or evidence synthesis without extractable study-level effects.
- The primary need is publication-bias modeling or meta-regression from the start.
Workflow
Read each step file in workflow/ before executing that step.
| Step |
Responsibility |
Executor |
Document |
Input |
Output |
| Collect |
Collect Inputs |
Main Agent |
workflow/step01-collect-inputs.md |
User input |
Structured input summary |
| Diagnose |
Check Distribution |
Main Agent |
workflow/step02-check-distribution.md |
Prior step output |
PART 1 code block |
| Test |
Run Primary Test |
Main Agent |
workflow/step03-run-primary-test.md |
Prior step output |
PART 2-3 code blocks |
Decision Tree
1. EFFECT SIZE TYPE
├── Continuous (SMD, MD) → escalc(measure="SMD") or "MD"
├── Binary (OR, RR, RD) → escalc(measure="OR") or "RR"/"RD"
└── Correlation (r) → escalc(measure="ZCOR") Fisher z transform
2. HETEROGENEITY ASSESSMENT
├── I² < 25% → low heterogeneity (fixed-effects may suffice)
├── 25% ≤ I² ≤ 75% → moderate (random-effects preferred)
└── I² > 75% → high (random-effects + moderator analysis needed)
Required Inputs
| Role |
What to collect |
| Study data |
Study ID, effect size, SE (or CI, or N per group + means/SDs) |
| Effect type |
SMD, MD, OR, RR, RD, or r |
| Moderators |
Study-level variables (year, quality, setting) — for recommendation |
| Model preference |
Fixed vs random (or let data decide) |
Code Structure
PART 0: Setup & Data Loading
PART 1: Study Table + Forest Plot + Heterogeneity → plot_01_forest.png
PART 2: Pooled Estimates (fixed + random) + Prediction Interval
PART 3: Recommendation Block → text listing additional analyses available
Reporting Standards
- Pooled effect: "pooled ES = X.XX, 95% CI [X.XX, X.XX], z = X.XX, p"
- Heterogeneity: "Q(df) = X.XX, p = .XXX; I² = XX.X%; tau² = X.XX"
- Prediction interval: always alongside CI
- p-values: "< .001" not "0.000"; exact to 3 decimals otherwise
- Weights: report study weights in forest plot
- Degrees of freedom: always with test statistics
- Sample size: report total N across studies and k (number of studies)
- Decimal places: 2 for effect sizes, 3 for p-values, 1 for I²
Example Dataset
Constructed example: 10 studies comparing treatment vs control on a
continuous outcome (standardized mean difference). NOT a built-in R
dataset — create a data frame with study_id, n_treatment, n_control,
mean_treatment, mean_control, sd_treatment, sd_control.
R: metafor package. Python: statsmodels or custom inverse-variance.
Method Status
| Status |
Methods |
| Implemented in this skill |
Effect-size construction, fixed/random pooled estimates, forest plot, heterogeneity metrics, prediction interval |
Implemented downstream in vera-data-meta-generating |
Publication-bias checks, sensitivity analysis, subgroup analysis, meta-regression, Bayesian and three-level extensions |
| Out of scope in this open-source baseline |
Network meta-analysis, IPD meta-analysis, and evidence-synthesis families not named above |
Minimal Smoke Test
- Smoke-test prompt: "Run
vera-data-meta-reviewing on a 10-study continuous-outcome example with study IDs, group means, SDs, and sample sizes. Produce the standard baseline artifacts."
Cross-Skill Interface
Output:
├── code_r → .R script
├── code_python → .py script
├── figures/ → 1 PNG (forest plot)
└── recommendations → text block (additional analyses available)
Next step: Invoke vera-data-meta-generating from this skillset to run the full pipeline (additional tests, subgroup analysis, modeling, manuscript generation). See ../../CROSS-SKILL-INTERFACE.md for the shared handoff contract.
1---2name: vera-data-meta-reviewing3description: Runs study-level diagnostics and primary pooled estimation for meta-analysis of summary statistics across multiple studies. Produces forest plot, heterogeneity assessment (Q, I-squared, tau-squared), pooled estimates under both fixed-effects and random-effects models, and prediction interval. Ends with a recommendation block listing Outputs .R and .py scripts with 1 publication-quality forest plot. Triggered when user says "meta-analysis," "systematic review," "pooled estimate," "combine studies," "forest plot," "heterogeneity," "I-squared," "effect size synthesis," or "aggregate results across studies." Does not handle network meta-analysis or individual participant data.4---56# Meta-Analysis — Diagnostics & Primary Pooled Estimation78## Table of Contents910- [Scope Boundary](#scope-boundary)11- [Workflow](#workflow)12- [Decision Tree](#decision-tree)13- [Required Inputs](#required-inputs)14- [Code Structure](#code-structure)15- [Reporting Standards](#reporting-standards)16- [Example Dataset](#example-dataset)17- [Method Status](#method-status)18- [Minimal Smoke Test](#minimal-smoke-test)19- [Cross-Skill Interface](#cross-skill-interface)202122Open-source skill.2324## Scope Boundary2526Use this skill when:27- The inputs are study-level summary statistics and the first need is a transparent pooled estimate with heterogeneity diagnostics.28- A one-effect-size-at-a-time meta-analysis is appropriate before moderator, bias, or multilevel extensions.2930Do not use this skill when:31- The project is network meta-analysis, IPD meta-analysis, or evidence synthesis without extractable study-level effects.32- The primary need is publication-bias modeling or meta-regression from the start.3334## Workflow3536Read each step file in `workflow/` before executing that step.3738| Step | Responsibility | Executor | Document | Input | Output |39|---|---|---|---|---|---|40| Collect | Collect Inputs | Main Agent | `workflow/step01-collect-inputs.md` | User input | Structured input summary |41| Diagnose | Check Distribution | Main Agent | `workflow/step02-check-distribution.md` | Prior step output | PART 1 code block |42| Test | Run Primary Test | Main Agent | `workflow/step03-run-primary-test.md` | Prior step output | PART 2-3 code blocks |4344## Decision Tree4546```471. EFFECT SIZE TYPE48 ├── Continuous (SMD, MD) → escalc(measure="SMD") or "MD"49 ├── Binary (OR, RR, RD) → escalc(measure="OR") or "RR"/"RD"50 └── Correlation (r) → escalc(measure="ZCOR") Fisher z transform51522. HETEROGENEITY ASSESSMENT53 ├── I² < 25% → low heterogeneity (fixed-effects may suffice)54 ├── 25% ≤ I² ≤ 75% → moderate (random-effects preferred)55 └── I² > 75% → high (random-effects + moderator analysis needed)56```5758## Required Inputs5960| Role | What to collect |61|---|---|62| **Study data** | Study ID, effect size, SE (or CI, or N per group + means/SDs) |63| **Effect type** | SMD, MD, OR, RR, RD, or r |64| **Moderators** | Study-level variables (year, quality, setting) — for recommendation |65| **Model preference** | Fixed vs random (or let data decide) |6667## Code Structure6869```70PART 0: Setup & Data Loading71PART 1: Study Table + Forest Plot + Heterogeneity → plot_01_forest.png72PART 2: Pooled Estimates (fixed + random) + Prediction Interval73PART 3: Recommendation Block → text listing additional analyses available74```7576## Reporting Standards77781. Pooled effect: "pooled ES = X.XX, 95% CI [X.XX, X.XX], z = X.XX, p"792. Heterogeneity: "Q(df) = X.XX, p = .XXX; I² = XX.X%; tau² = X.XX"803. Prediction interval: always alongside CI814. p-values: "< .001" not "0.000"; exact to 3 decimals otherwise825. Weights: report study weights in forest plot836. Degrees of freedom: always with test statistics847. Sample size: report total N across studies and k (number of studies)858. Decimal places: 2 for effect sizes, 3 for p-values, 1 for I²8687## Example Dataset8889Constructed example: 10 studies comparing treatment vs control on a90continuous outcome (standardized mean difference). NOT a built-in R91dataset — create a data frame with study_id, n_treatment, n_control,92mean_treatment, mean_control, sd_treatment, sd_control.9394R: `metafor` package. Python: `statsmodels` or custom inverse-variance.9596## Method Status9798| Status | Methods |99|---|---|100| Implemented in this skill | Effect-size construction, fixed/random pooled estimates, forest plot, heterogeneity metrics, prediction interval |101| Implemented downstream in `vera-data-meta-generating` | Publication-bias checks, sensitivity analysis, subgroup analysis, meta-regression, Bayesian and three-level extensions |102| Out of scope in this open-source baseline | Network meta-analysis, IPD meta-analysis, and evidence-synthesis families not named above |103104## Minimal Smoke Test105106- Smoke-test prompt: "Run `vera-data-meta-reviewing` on a 10-study continuous-outcome example with study IDs, group means, SDs, and sample sizes. Produce the standard baseline artifacts."107108## Cross-Skill Interface109110```111Output:112├── code_r → .R script113├── code_python → .py script114├── figures/ → 1 PNG (forest plot)115└── recommendations → text block (additional analyses available)116```117118Next step: Invoke `vera-data-meta-generating` from this skillset to run the full pipeline (additional tests, subgroup analysis, modeling, manuscript generation). See `../../CROSS-SKILL-INTERFACE.md` for the shared handoff contract.
VeraSuperHub