Pairwise Ma Methodology

Pairwise Meta-Analysis Methodology

Comprehensive methodological guidance for conducting rigorous pairwise meta-analysis following Cochrane and PRISMA guidelines.

When to Use This Skill

• Planning a pairwise meta-analysis
• Choosing between fixed and random effects models
• Interpreting heterogeneity statistics
• Assessing publication bias
• Designing sensitivity analyses
• Reviewing pairwise MA code or results

Fixed vs Random Effects

Decision Framework

Are studies functionally identical?
├── Yes → Fixed-effect model appropriate
│   - Same population, intervention, comparator, outcome
│   - Estimating single "true" effect
│
└── No (usually the case) → Random-effects model
    - Studies differ in ways that affect true effect
    - Estimating mean of distribution of effects
    - More generalizable inference

When to Use Fixed-Effect

• Studies are very similar (rare in practice)
• Want to estimate effect in "identical" studies
• Very few studies (< 5) - random effects unreliable
• Sensitivity analysis alongside random effects

When to Use Random-Effects

• Studies differ in populations, settings, methods
• Want inference applicable beyond included studies
• Default choice for most meta-analyses
• Use with appropriate adjustments (Knapp-Hartung)

Key Differences

Heterogeneity Assessment

Statistics Overview

Q Statistic (Cochran's Q)

• Tests null hypothesis of homogeneity
• Follows chi-square distribution under null
• Low power with few studies
• Overpowered with many studies

# Interpretation
Q_pvalue < 0.10  # Suggests heterogeneity (use 0.10, not 0.05)

I² (Inconsistency Index)

• Percentage of variability due to heterogeneity (vs sampling error)
• Independent of number of studies
• Has wide confidence interval with few studies

Caution: These thresholds are rules of thumb, not strict cutoffs.

τ² (Tau-squared)

• Absolute between-study variance
• On scale of effect measure
• Used for prediction intervals
• Compare to typical effect sizes for context

H²

• Relative excess heterogeneity
• H² = Q/(k-1) where k = number of studies
• H² = 1 means no heterogeneity

Prediction Intervals

Critical: Always report prediction intervals alongside confidence intervals.

• CI: Uncertainty in mean effect estimate
• PI: Range where 95% of true study effects would lie

# In meta package
metabin(..., prediction = TRUE)

If PI includes null but CI doesn't:

• Mean effect is statistically significant
• But future studies might show no effect or opposite effect
• Heterogeneity is clinically important

Investigation of Heterogeneity

Subgroup Analysis

# Categorical moderator
update(ma_result, subgroup = risk_of_bias)

# Requirements:
# - Pre-specified in protocol
# - Limited number of subgroups
# - Biological/clinical rationale
# - Report within and between subgroup heterogeneity

Meta-Regression

# Continuous moderator
rma(yi, vi, mods = ~ year + sample_size, data = es_data)

# Requirements:
# - Minimum 10 studies per moderator
# - Pre-specified moderators
# - Avoid overfitting
# - Use Knapp-Hartung adjustment
# - Permutation test for multiple moderators

Rule of Thumb for Investigation

• Need ≥10 studies for meaningful subgroup analysis
• Meta-regression requires even more studies
• Pre-specify investigations in protocol
• Report all investigated moderators (avoid selective reporting)

Publication Bias Assessment

Visual Assessment: Funnel Plot

funnel(ma_result)
# Look for:
# - Asymmetry (small studies with large effects)
# - Missing studies in certain regions
# - Outliers

Statistical Tests

Egger's Test (Continuous Outcomes)

metabias(ma_result, method.bias = "linreg")
# P < 0.10 suggests asymmetry
# Low power with < 10 studies

Peters' Test (Binary Outcomes)

metabias(ma_result, method.bias = "peters")
# Better for OR than Egger's

Begg's Rank Test

metabias(ma_result, method.bias = "rank")
# Non-parametric alternative
# Lower power than regression tests

Adjustment Methods

Trim-and-Fill

trimfill(ma_result)
# Imputes "missing" studies
# Provides adjusted estimate
# Sensitivity analysis, not definitive correction

Selection Models

# More sophisticated approaches
# Model the selection process
# Available in metafor and weightr packages

Interpretation Cautions

• Asymmetry ≠ publication bias (could be true heterogeneity)
• Tests have low power with few studies
• Don't over-interpret with < 10 studies
• Multiple causes of asymmetry exist

Sensitivity Analyses

Essential Sensitivity Analyses

1. Fixed vs Random Effects

   • Report both; if results differ, investigate why

2. Leave-One-Out

   metainf(ma_result)
   # Identifies influential studies
   

3. Risk of Bias

   • Exclude high risk of bias studies
   • Subgroup by risk of bias

4. Influence Diagnostics

   influence(ma_result)
   # DFBETAS, Cook's distance
   

5. Different Effect Measures

   • OR vs RR vs RD for binary
   • May give different conclusions

6. Estimation Method

   • DerSimonian-Laird vs REML vs ML

GOSH Analysis

# Graphical display of study heterogeneity
gosh(ma_result)
# Identifies subsets with different results

Reporting Checklist (PRISMA)

Methods

• Effect measure and rationale
• Model choice (fixed/random) and rationale
• Heterogeneity measures planned
• Publication bias assessment planned
• Sensitivity analyses planned
• Software and packages used

Results

• Number of studies and participants
• Pooled effect with CI
• Prediction interval
• Heterogeneity statistics (Q, I², τ²)
• Forest plot
• Funnel plot (if ≥10 studies)
• Publication bias test results
• Sensitivity analysis results

Common Pitfalls

1. Using Q p-value to Choose Model

• Wrong: "Q p > 0.05, so use fixed-effect"
• Right: Choose based on study similarity, report both

2. Ignoring Prediction Intervals

• CI shows precision of mean estimate
• PI shows variability in true effects
• Both are clinically important

3. Over-interpreting I²

• I² has wide CI with few studies
• Context matters (clinical significance)
• Don't use arbitrary thresholds mechanically

4. Selective Subgroup Analysis

• Pre-specify in protocol
• Report all, not just significant ones
• Adjust for multiple testing

5. Publication Bias Assessment with Few Studies

• Tests unreliable with < 10 studies
• State this limitation, don't perform test

Quick Reference Code

library(meta)

# Basic random-effects MA (binary)
ma <- metabin(
  event.e, n.e, event.c, n.c,
  studlab = study,
  data = dat,
  sm = "OR",
  method = "MH",
  method.tau = "REML",
  hakn = TRUE,           # Knapp-Hartung adjustment
  prediction = TRUE      # Prediction interval
)

# Forest plot
forest(ma, sortvar = TE, prediction = TRUE)

# Funnel plot and Egger's test
funnel(ma)
metabias(ma, method.bias = "linreg")

# Influence analysis
metainf(ma)

# Subgroup analysis
update(ma, subgroup = risk_of_bias)

Resources

• Cochrane Handbook: https://training.cochrane.org/handbook
• PRISMA Statement: http://www.prisma-statement.org/
• Higgins & Green: Cochrane Handbook for Systematic Reviews
• Borenstein et al.: Introduction to Meta-Analysis