Bias Detection Strategy
Design a protocol to systematically assess biases that threaten the validity of meta-analytic conclusions.
Purpose
Bias in the evidence body (publication bias, outcome reporting bias, citation bias, time-lag bias, language bias) can invalidate pooled estimates. This strategy designs the complete bias detection and adjustment protocol — funnel plots, statistical tests, sensitivity analyses, and GRADE certainty downgrading.
Budget
| Resource |
Floor |
Target |
| Studies identified |
28 |
40 |
| Effect sizes extracted |
28 |
40 |
| Web searches |
28 |
40 |
| Bias domains assessed |
5 |
8 |
| Quality assessments |
20 |
40 |
Budget gate: cannot exit until 80% of floor met.
State Ledger
<HARD-GATE>
| Metric | Current | Floor | Target | Status |
|--------|---------|-------|--------|--------|
| Studies found | 0 | 28 | 40 | BLOCKED |
| Effect sizes planned | 0 | 28 | 40 | BLOCKED |
| Web searches done | 0 | 28 | 40 | BLOCKED |
| Bias domains assessed | 0 | 5 | 8 | BLOCKED |
| Quality assessed | 0 | 20 | 40 | BLOCKED |
</HARD-GATE>
Available Tactics
| Tactic |
When to Use |
| effect-size-extraction |
Extract effect sizes with precision (SE, CI) |
| quality-assessment-protocol |
Full RoB2 assessment per study |
| evidence-synthesis-planning |
Plan bias-adjusted models |
Available SOPs
| SOP |
When to Use |
| pico-formulation |
Frame the evidence assessment question |
| inclusion-criteria-design |
Include grey literature, preprints |
| effect-size-planning |
Ensure precision metrics extracted |
| data-extraction-form |
Template capturing reporting completeness |
| risk-of-bias-assessment |
Per-study RoB (core of this strategy) |
| publication-bias-assessment |
Core SOP — funnel plots, statistical tests |
| sensitivity-analysis-design |
Trim-and-fill, selection models |
| heterogeneity-source-analysis |
Bias as heterogeneity driver |
| meta-analysis-synthesis |
Final bias assessment protocol |
Execution Guidance
- Frame — Run
pico-formulation for the evidence reliability question
- Scope — Run
inclusion-criteria-design maximizing source diversity (grey lit, preprints, registries)
- Search — Search for published AND unpublished studies, trial registries
- Extract — Use
effect-size-extraction with precision metrics (SE, CI, N)
- Assess — Use
quality-assessment-protocol for comprehensive RoB2
- Detect — Run
publication-bias-assessment for statistical detection plan
- Investigate — Run
heterogeneity-source-analysis for bias-driven heterogeneity
- Adjust — Run
sensitivity-analysis-design for bias-adjustment methods
- Synthesize — Run
meta-analysis-synthesis for final protocol
Web searches target: trial registries, grey literature databases, dissertation repositories, conference abstracts.
Output Format
protocol:
question: [Is the evidence body for X biased?]
bias_domains:
publication_bias:
visual: [funnel plot, contour-enhanced funnel]
statistical: [Egger's test, Begg's test, Peters' test]
adjustment: [trim-and-fill, Copas selection model, PET-PEESE]
outcome_reporting_bias:
detection: [registry-publication comparison]
tool: [ROB-ME, ORBIT]
time_lag_bias:
detection: [time-to-publication analysis]
citation_bias:
detection: [citation network analysis]
language_bias:
mitigation: [multi-language search strategy]
small_study_effects:
detection: [funnel asymmetry, regression tests]
adjustment: [limit meta-analysis]
grey_literature_search: [databases, registries, contacts]
grade_assessment:
domain: publication_bias
downgrading_criteria: [when to downgrade certainty]
sensitivity_plan: [selection model, 3PSM, p-curve, z-curve]
reporting: PRISMA-2020 + ROB-ME guidelines
Available Tactics
Optional, no fixed order; the final leaf is always a sop.
| Tactic |
When to use |
| effect-size-extraction |
Systematically extract effect sizes and conditions from papers for meta-analytic synthesis |
| evidence-synthesis-planning |
Plan the statistical synthesis approach — model selection, heterogeneity strategy, and reporting |
| quality-assessment-protocol |
Methodological quality and bias risk assessment of included studies using validated tools |
Available SOPs
Optional, no fixed order; the final leaf is always a sop.
| SOP |
When to use |
| data-extraction-form |
Design structured data extraction form for systematic meta-analysis data collection |
| effect-size-planning |
Determine effect size types and calculation methods for meta-analytic synthesis |
| heterogeneity-source-analysis |
Identify and classify sources of between-study heterogeneity (clinical, methodological, statistical) |
| inclusion-criteria-design |
Define inclusion/exclusion criteria for systematic study selection in meta-analysis |
| meta-analysis-synthesis |
Produce final meta-analysis protocol document assembling all planning outputs into PRISMA-compliant protocol |
| pico-formulation |
Construct PICO/PECO framework for the meta-analysis research question |
| publication-bias-assessment |
Plan funnel plots, Egger's test, trim-and-fill, p-curve, and selection model analyses for publication bias |
| risk-of-bias-assessment |
Assess methodological bias using RoB2, PROBAST, or QUADAS-2 validated tools |
| sensitivity-analysis-design |
Design leave-one-out, influence diagnostics, subgroup analyses, and robustness checks |
1---2name: bias-detection3description: Assess systematic biases in the evidence body — publication bias, reporting bias, and selective outcome reporting. Budget: 40 studies, 40 effect sizes, 40 web searches.4---56# Bias Detection Strategy78Design a protocol to systematically assess biases that threaten the validity of meta-analytic conclusions.910## Purpose1112Bias in the evidence body (publication bias, outcome reporting bias, citation bias, time-lag bias, language bias) can invalidate pooled estimates. This strategy designs the complete bias detection and adjustment protocol — funnel plots, statistical tests, sensitivity analyses, and GRADE certainty downgrading.1314## Budget1516| Resource | Floor | Target |17|----------|-------|--------|18| Studies identified | 28 | 40 |19| Effect sizes extracted | 28 | 40 |20| Web searches | 28 | 40 |21| Bias domains assessed | 5 | 8 |22| Quality assessments | 20 | 40 |2324Budget gate: cannot exit until 80% of floor met.2526## State Ledger2728```29<HARD-GATE>30| Metric | Current | Floor | Target | Status |31|--------|---------|-------|--------|--------|32| Studies found | 0 | 28 | 40 | BLOCKED |33| Effect sizes planned | 0 | 28 | 40 | BLOCKED |34| Web searches done | 0 | 28 | 40 | BLOCKED |35| Bias domains assessed | 0 | 5 | 8 | BLOCKED |36| Quality assessed | 0 | 20 | 40 | BLOCKED |37</HARD-GATE>38```3940## Available Tactics4142| Tactic | When to Use |43|--------|-------------|44| effect-size-extraction | Extract effect sizes with precision (SE, CI) |45| quality-assessment-protocol | Full RoB2 assessment per study |46| evidence-synthesis-planning | Plan bias-adjusted models |4748## Available SOPs4950| SOP | When to Use |51|-----|-------------|52| pico-formulation | Frame the evidence assessment question |53| inclusion-criteria-design | Include grey literature, preprints |54| effect-size-planning | Ensure precision metrics extracted |55| data-extraction-form | Template capturing reporting completeness |56| risk-of-bias-assessment | Per-study RoB (core of this strategy) |57| publication-bias-assessment | Core SOP — funnel plots, statistical tests |58| sensitivity-analysis-design | Trim-and-fill, selection models |59| heterogeneity-source-analysis | Bias as heterogeneity driver |60| meta-analysis-synthesis | Final bias assessment protocol |6162## Execution Guidance63641. **Frame** — Run `pico-formulation` for the evidence reliability question652. **Scope** — Run `inclusion-criteria-design` maximizing source diversity (grey lit, preprints, registries)663. **Search** — Search for published AND unpublished studies, trial registries674. **Extract** — Use `effect-size-extraction` with precision metrics (SE, CI, N)685. **Assess** — Use `quality-assessment-protocol` for comprehensive RoB2696. **Detect** — Run `publication-bias-assessment` for statistical detection plan707. **Investigate** — Run `heterogeneity-source-analysis` for bias-driven heterogeneity718. **Adjust** — Run `sensitivity-analysis-design` for bias-adjustment methods729. **Synthesize** — Run `meta-analysis-synthesis` for final protocol7374Web searches target: trial registries, grey literature databases, dissertation repositories, conference abstracts.7576## Output Format7778```yaml79protocol:80 question: [Is the evidence body for X biased?]81 bias_domains:82 publication_bias:83 visual: [funnel plot, contour-enhanced funnel]84 statistical: [Egger's test, Begg's test, Peters' test]85 adjustment: [trim-and-fill, Copas selection model, PET-PEESE]86 outcome_reporting_bias:87 detection: [registry-publication comparison]88 tool: [ROB-ME, ORBIT]89 time_lag_bias:90 detection: [time-to-publication analysis]91 citation_bias:92 detection: [citation network analysis]93 language_bias:94 mitigation: [multi-language search strategy]95 small_study_effects:96 detection: [funnel asymmetry, regression tests]97 adjustment: [limit meta-analysis]98 grey_literature_search: [databases, registries, contacts]99 grade_assessment:100 domain: publication_bias101 downgrading_criteria: [when to downgrade certainty]102 sensitivity_plan: [selection model, 3PSM, p-curve, z-curve]103 reporting: PRISMA-2020 + ROB-ME guidelines104```105106<!-- BEGIN available-tables (generated) -->107108## Available Tactics109110Optional, no fixed order; the final leaf is always a sop.111112| Tactic | When to use |113| --- | --- |114| effect-size-extraction | Systematically extract effect sizes and conditions from papers for meta-analytic synthesis |115| evidence-synthesis-planning | Plan the statistical synthesis approach — model selection, heterogeneity strategy, and reporting |116| quality-assessment-protocol | Methodological quality and bias risk assessment of included studies using validated tools |117118## Available SOPs119120Optional, no fixed order; the final leaf is always a sop.121122| SOP | When to use |123| --- | --- |124| data-extraction-form | Design structured data extraction form for systematic meta-analysis data collection |125| effect-size-planning | Determine effect size types and calculation methods for meta-analytic synthesis |126| heterogeneity-source-analysis | Identify and classify sources of between-study heterogeneity (clinical, methodological, statistical) |127| inclusion-criteria-design | Define inclusion/exclusion criteria for systematic study selection in meta-analysis |128| meta-analysis-synthesis | Produce final meta-analysis protocol document assembling all planning outputs into PRISMA-compliant protocol |129| pico-formulation | Construct PICO/PECO framework for the meta-analysis research question |130| publication-bias-assessment | Plan funnel plots, Egger's test, trim-and-fill, p-curve, and selection model analyses for publication bias |131| risk-of-bias-assessment | Assess methodological bias using RoB2, PROBAST, or QUADAS-2 validated tools |132| sensitivity-analysis-design | Design leave-one-out, influence diagnostics, subgroup analyses, and robustness checks |133134<!-- END available-tables (generated) -->
yogsoth-ai