後設分析 (Meta-Analysis)
Overview
Meta-analysis statistically combines effect sizes from multiple independent studies to produce a pooled estimate with greater precision and generalizability. It quantifies between-study heterogeneity and tests for publication bias, providing a rigorous evidence synthesis that goes beyond narrative literature reviews.
When to Use
- Synthesizing quantitative findings from multiple studies on the same research question
- Resolving conflicting results across studies
- Estimating an overall effect size with tighter confidence intervals
- Identifying moderators that explain heterogeneity across studies
When NOT to Use
- Studies are too heterogeneous in constructs, measures, or populations to combine meaningfully
- Fewer than 5 studies are available (pooled estimates become unreliable)
- Primary studies have fundamentally different research designs (mixing RCTs with observational)
- The research question is qualitative or conceptual rather than quantitative
Assumptions
IRON LAW: A meta-analysis is only as good as the studies it includes —
garbage in, garbage out. Publication bias inflates pooled effect sizes
because non-significant findings go unpublished.
Key assumptions:
- Studies estimate the same underlying construct (conceptual homogeneity)
- Effect sizes are statistically independent (one effect per study, or use multilevel models)
- Study-level moderators are coded reliably and without bias
- The search strategy captures the relevant population of studies (no systematic omission)
Methodology
Step 1 — Extract and Code Effect Sizes
Convert study findings to a common effect size metric (Cohen's d, Hedges' g, r, OR). Code study-level moderators (sample size, design, context). See references/ for conversion formulas.
Step 2 — Choose Fixed-Effect vs Random-Effects Model
Fixed-effect assumes one true effect; random-effects assumes effects vary across studies. If studies span different populations or contexts, random-effects is almost always appropriate.
Step 3 — Assess Heterogeneity
Compute Q statistic (test of homogeneity), I² (proportion of variance due to heterogeneity), and τ² (between-study variance). I² > 75% indicates substantial heterogeneity warranting moderator analysis.
Step 4 — Test for Publication Bias and Report
Use funnel plot, Egger's regression test, and trim-and-fill method. Report pooled effect, CI, prediction interval, and results of bias assessment.
Output Format
## Meta-Analysis: [Research Question]
### Study Inclusion
| Criterion | Value |
|-----------|-------|
| Studies included (k) | xx |
| Total sample size (N) | xxxx |
| Effect size metric | [d / r / OR] |
### Pooled Effect Size
| Model | Effect | 95% CI | z | p-value |
|-------|--------|--------|---|---------|
| Fixed-effect | x.xx | [x.xx, x.xx] | x.xx | x.xx |
| Random-effects | x.xx | [x.xx, x.xx] | x.xx | x.xx |
### Heterogeneity
| Statistic | Value | Interpretation |
|-----------|-------|----------------|
| Q | x.xx (p = x.xx) | [significant/not] |
| I² | x.xx% | [low/moderate/high] |
| τ² | x.xx | [between-study variance] |
### Publication Bias
| Test | Result | Interpretation |
|------|--------|----------------|
| Funnel plot | [symmetric/asymmetric] | [bias suspected?] |
| Egger's test | p = x.xx | [significant?] |
| Trim-and-fill | adjusted effect = x.xx | [studies imputed: x] |
### Limitations
- [Note any assumption violations]
Gotchas
- Combining apples and oranges: statistically possible but conceptually meaningless if constructs differ
- Random-effects models give more weight to small studies, which are often lower quality
- I² depends on precision of included studies; low I² with imprecise studies does not mean homogeneity
- Funnel plot asymmetry can be caused by factors other than publication bias (small-study effects)
- File-drawer problem: unpublished null results are systematically missing
- Moderator analyses with many subgroups and few studies per subgroup are underpowered and unreliable
References
- Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to Meta-Analysis. Wiley.
- Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539-1558.
- Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication Bias in Meta-Analysis. Wiley.
1---2name: grad-meta-analysis3description: Apply meta-analysis to synthesize effect sizes across multiple studies, assess heterogeneity, and evaluate publication bias. Use this skill when the user needs to combine findings from prior research, compare fixed-effect vs random-effects models, compute pooled effect sizes, or when they ask 'what does the overall evidence say', 'how do I combine results across studies', or 'is there publication bias'.4---56# 後設分析 (Meta-Analysis)78## Overview910Meta-analysis statistically combines effect sizes from multiple independent studies to produce a pooled estimate with greater precision and generalizability. It quantifies between-study heterogeneity and tests for publication bias, providing a rigorous evidence synthesis that goes beyond narrative literature reviews.1112## When to Use1314- Synthesizing quantitative findings from multiple studies on the same research question15- Resolving conflicting results across studies16- Estimating an overall effect size with tighter confidence intervals17- Identifying moderators that explain heterogeneity across studies1819## When NOT to Use2021- Studies are too heterogeneous in constructs, measures, or populations to combine meaningfully22- Fewer than 5 studies are available (pooled estimates become unreliable)23- Primary studies have fundamentally different research designs (mixing RCTs with observational)24- The research question is qualitative or conceptual rather than quantitative2526## Assumptions2728```29IRON LAW: A meta-analysis is only as good as the studies it includes —30garbage in, garbage out. Publication bias inflates pooled effect sizes31because non-significant findings go unpublished.32```3334Key assumptions:351. Studies estimate the same underlying construct (conceptual homogeneity)362. Effect sizes are statistically independent (one effect per study, or use multilevel models)373. Study-level moderators are coded reliably and without bias384. The search strategy captures the relevant population of studies (no systematic omission)3940## Methodology4142### Step 1 — Extract and Code Effect Sizes4344Convert study findings to a common effect size metric (Cohen's d, Hedges' g, r, OR). Code study-level moderators (sample size, design, context). See `references/` for conversion formulas.4546### Step 2 — Choose Fixed-Effect vs Random-Effects Model4748Fixed-effect assumes one true effect; random-effects assumes effects vary across studies. If studies span different populations or contexts, random-effects is almost always appropriate.4950### Step 3 — Assess Heterogeneity5152Compute Q statistic (test of homogeneity), I² (proportion of variance due to heterogeneity), and τ² (between-study variance). I² > 75% indicates substantial heterogeneity warranting moderator analysis.5354### Step 4 — Test for Publication Bias and Report5556Use funnel plot, Egger's regression test, and trim-and-fill method. Report pooled effect, CI, prediction interval, and results of bias assessment.5758## Output Format5960```markdown61## Meta-Analysis: [Research Question]6263### Study Inclusion64| Criterion | Value |65|-----------|-------|66| Studies included (k) | xx |67| Total sample size (N) | xxxx |68| Effect size metric | [d / r / OR] |6970### Pooled Effect Size71| Model | Effect | 95% CI | z | p-value |72|-------|--------|--------|---|---------|73| Fixed-effect | x.xx | [x.xx, x.xx] | x.xx | x.xx |74| Random-effects | x.xx | [x.xx, x.xx] | x.xx | x.xx |7576### Heterogeneity77| Statistic | Value | Interpretation |78|-----------|-------|----------------|79| Q | x.xx (p = x.xx) | [significant/not] |80| I² | x.xx% | [low/moderate/high] |81| τ² | x.xx | [between-study variance] |8283### Publication Bias84| Test | Result | Interpretation |85|------|--------|----------------|86| Funnel plot | [symmetric/asymmetric] | [bias suspected?] |87| Egger's test | p = x.xx | [significant?] |88| Trim-and-fill | adjusted effect = x.xx | [studies imputed: x] |8990### Limitations91- [Note any assumption violations]92```9394## Gotchas9596- Combining apples and oranges: statistically possible but conceptually meaningless if constructs differ97- Random-effects models give more weight to small studies, which are often lower quality98- I² depends on precision of included studies; low I² with imprecise studies does not mean homogeneity99- Funnel plot asymmetry can be caused by factors other than publication bias (small-study effects)100- File-drawer problem: unpublished null results are systematically missing101- Moderator analyses with many subgroups and few studies per subgroup are underpowered and unreliable102103## References104105- Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). *Introduction to Meta-Analysis*. Wiley.106- Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. *Statistics in Medicine*, 21(11), 1539-1558.107- Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). *Publication Bias in Meta-Analysis*. Wiley.
asgard-ai-platform