tools-only Evidence Synthesis and Guideline Integration Guide
Overview
Evidence synthesis involves systematically reviewing, analyzing, and integrating research findings to inform clinical recommendations. This guide covers guideline sources, evidence hierarchies, systematic reviews, meta-analyses, and integration of multiple evidence streams for clinical decision support.
Major Clinical Practice Guidelines
Oncology Guidelines
NCCN (National Comprehensive Cancer Network)
- Scope: 60+ cancer types, supportive care guidelines
- Update Frequency: Continuous (online), 1-3 updates per year per guideline
- Evidence Categories:
- Category 1: High-level evidence, uniform NCCN consensus
- Category 2A: Lower-level evidence, uniform consensus (appropriate)
- Category 2B: Lower-level evidence, non-uniform consensus (appropriate)
- Category 3: Major disagreement or insufficient evidence
- Access: Free for patients, subscription for providers (institutional access common)
- Application: US-focused, most widely used in clinical practice
ASCO (American Society of Clinical Oncology)
- Scope: Evidence-based clinical practice guidelines
- Methodology: Systematic review, GRADE-style evidence tables
- Endorsements: Often endorses NCCN, ESMO, or other guidelines
- Focused Topics: Specific clinical questions (e.g., biomarker testing, supportive care)
- Guideline Products: Full guidelines, rapid recommendations, endorsements
- Quality: Rigorous methodology, peer-reviewed publication
ESMO (European Society for Medical Oncology)
- Scope: European guidelines for cancer management
- Evidence Levels:
- I: Evidence from at least one large RCT or meta-analysis
- II: Evidence from at least one well-designed non-randomized trial, cohort study
- III: Evidence from well-designed non-experimental study
- IV: Evidence from expert committee reports or opinions
- V: Evidence from case series, case reports
- Recommendation Grades:
- A: Strong evidence for efficacy, substantial clinical benefit (strongly recommended)
- B: Strong or moderate evidence, limited clinical benefit (generally recommended)
- C: Insufficient evidence, benefit not sufficiently well established
- D: Moderate evidence against efficacy or for adverse effects (not recommended)
- E: Strong evidence against efficacy (never recommended)
- ESMO-MCBS: Magnitude of Clinical Benefit Scale (grades 1-5 for meaningful benefit)
Cardiovascular Guidelines
AHA/ACC (American Heart Association / American College of Cardiology)
- Scope: Cardiovascular disease prevention, diagnosis, management
- Class of Recommendation (COR):
- Class I: Strong recommendation - should be performed/administered
- Class IIa: Moderate recommendation - is reasonable
- Class IIb: Weak recommendation - may be considered
- Class III - No Benefit: Not recommended
- Class III - Harm: Potentially harmful
- Level of Evidence (LOE):
- A: High-quality evidence from >1 RCT, meta-analyses
- B-R: Moderate-quality evidence from ≥1 RCT
- B-NR: Moderate-quality evidence from non-randomized studies
- C-LD: Limited data from observational studies, registries
- C-EO: Expert opinion based on clinical experience
- Example: "Statin therapy is recommended for adults with LDL-C ≥190 mg/dL (Class I, LOE A)"
ESC (European Society of Cardiology)
- Scope: European cardiovascular guidelines
- Class of Recommendation:
- I: Recommended or indicated
- II: Should be considered
- III: Not recommended
- Level of Evidence: A (RCTs), B (single RCT or observational), C (expert opinion)
Other Specialties
IDSA (Infectious Diseases Society of America)
- Antimicrobial guidelines, infection management
- GRADE methodology
- Strong vs weak recommendations
ATS/ERS (American Thoracic Society / European Respiratory Society)
- Respiratory disease management
- GRADE methodology
ACR (American College of Rheumatology)
- Rheumatic disease guidelines
- Conditionally recommended vs strongly recommended
KDIGO (Kidney Disease: Improving Global Outcomes)
- Chronic kidney disease, dialysis, transplant
- GRADE-based recommendations
GRADE Methodology
Assessing Quality of Evidence
Initial Quality Assignment
Randomized Controlled Trials: Start at HIGH quality (⊕⊕⊕⊕)
Observational Studies: Start at LOW quality (⊕⊕○○)
Factors Decreasing Quality (Downgrade)
Risk of Bias (-1 or -2 levels)
- Lack of allocation concealment
- Lack of blinding
- Incomplete outcome data
- Selective outcome reporting
- Other sources of bias
Inconsistency (-1 or -2 levels)
- Unexplained heterogeneity in results across studies
- Wide variation in effect estimates
- Non-overlapping confidence intervals
- High I² statistic in meta-analysis (>50-75%)
Indirectness (-1 or -2 levels)
- Different population than target (younger patients in trials, applying to elderly)
- Different intervention (higher dose in trial than used in practice)
- Different comparator (placebo in trial, comparing to active treatment)
- Surrogate outcomes (PFS) when interested in survival (OS)
Imprecision (-1 or -2 levels)
- Wide confidence intervals crossing threshold of benefit/harm
- Small sample size, few events
- Optimal information size (OIS) not met
- Rule of thumb: <300 events for continuous outcomes, <200 events for dichotomous
Publication Bias (-1 level)
- Funnel plot asymmetry (if ≥10 studies)
- Known unpublished studies with negative results
- Selective outcome reporting
- Industry-sponsored studies only
Factors Increasing Quality (Upgrade - Observational Only)
Large Magnitude of Effect (+1 or +2 levels)
- +1: RR >2 or <0.5 (moderate effect)
- +2: RR >5 or <0.2 (large effect)
- No plausible confounders would reduce effect
Dose-Response Gradient (+1 level)
- Clear dose-response or duration-response relationship
- Strengthens causal inference
All Plausible Confounders Would Reduce Effect (+1 level)
- Observed effect despite confounders biasing toward null
- Rare, requires careful justification
Final Quality Rating
After adjustments, assign final quality:
- High (⊕⊕⊕⊕): Very confident in effect estimate
- Moderate (⊕⊕⊕○): Moderately confident; true effect likely close to estimate
- Low (⊕⊕○○): Limited confidence; true effect may be substantially different
- Very Low (⊕○○○): Very little confidence; true effect likely substantially different
Systematic Reviews and Meta-Analyses
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)
Search Strategy
- Databases: PubMed/MEDLINE, Embase, Cochrane Library, Web of Science
- Search Terms: PICO (Population, Intervention, Comparator, Outcome)
- Date Range: Typically last 10-20 years or comprehensive
- Language: English only or all languages with translation
- Grey Literature: Conference abstracts, trial registries, unpublished data
Study Selection
PRISMA Flow Diagram:
Records identified through database searching (n=2,450)
Additional records through other sources (n=15)
↓
Records after duplicates removed (n=1,823)
↓
Records screened (title/abstract) (n=1,823) → Excluded (n=1,652)
↓ - Not relevant topic (n=1,120)
Full-text articles assessed (n=171) - Animal studies (n=332)
↓ - Reviews (n=200)
Studies included in qualitative synthesis (n=38) → Excluded (n=133)
↓ - Wrong population (n=42)
Studies included in meta-analysis (n=24) - Wrong intervention (n=35)
- No outcomes reported (n=28)
- Duplicate data (n=18)
- Poor quality (n=10)
Data Extraction
- Study characteristics: Design, sample size, population, intervention
- Results: Outcomes, effect sizes, confidence intervals, p-values
- Quality assessment: Risk of bias tool (Cochrane RoB 2.0 for RCTs)
- Dual extraction: Two reviewers independently, resolve disagreements
Meta-Analysis Methods
Fixed-Effect Model
- Assumption: Single true effect size shared by all studies
- Weighting: By inverse variance (larger studies have more weight)
- Application: When heterogeneity is low (I² <25%)
- Interpretation: Estimate of common effect across studies
Random-Effects Model
- Assumption: True effect varies across studies (distribution of effects)
- Weighting: By inverse variance + between-study variance
- Application: When heterogeneity moderate to high (I² ≥25%)
- Interpretation: Estimate of average effect (center of distribution)
- Wider CI: Accounts for heterogeneity, more conservative
Heterogeneity Assessment
I² Statistic
- Percentage of variability due to heterogeneity rather than chance
- I² = 0-25%: Low heterogeneity
- I² = 25-50%: Moderate heterogeneity
- I² = 50-75%: Substantial heterogeneity
- I² = 75-100%: Considerable heterogeneity
Q Test (Cochran's Q)
- Test for heterogeneity
- p<0.10 suggests significant heterogeneity (liberal threshold)
- Low power when few studies, use I² as primary measure
Tau² (τ²)
- Estimate of between-study variance
- Used in random-effects weighting
Subgroup Analysis
- Explore sources of heterogeneity
- Pre-specified subgroups: Disease stage, biomarker status, treatment regimen
- Test for interaction between subgroups
Forest Plot Interpretation
Study n HR (95% CI) Weight
─────────────────────────────────────────────────────────────
Trial A 2018 450 0.62 (0.45-0.85) ●───┤ 28%
Trial B 2019 320 0.71 (0.49-1.02) ●────┤ 22%
Trial C 2020 580 0.55 (0.41-0.74) ●──┤ 32%
Trial D 2021 210 0.88 (0.56-1.38) ●──────┤ 18%
Overall (RE model) 1560 0.65 (0.53-0.80) ◆──┤
Heterogeneity: I²=42%, p=0.16
0.25 0.5 1.0 2.0 4.0
Favors Treatment Favors Control
Guideline Integration
Concordance Checking
Multi-Guideline Comparison
Recommendation: First-line treatment for advanced NSCLC, PD-L1 ≥50%
Guideline Version Recommendation Strength
─────────────────────────────────────────────────────────────────────────────
NCCN v4.2024 Pembrolizumab monotherapy (preferred) Category 1
ESMO 2023 Pembrolizumab monotherapy (preferred) I, A
ASCO 2022 Endorses NCCN guidelines Strong
NICE (UK) 2023 Pembrolizumab approved Recommended
Synthesis: Strong consensus across guidelines for pembrolizumab monotherapy.
Alternative: Pembrolizumab + chemotherapy also Category 1/I-A recommended.
Discordance Resolution
- Identify differences and reasons (geography, cost, access, evidence interpretation)
- Note date of each guideline (newer may incorporate recent trials)
- Consider regional applicability
- Favor guidelines with most rigorous methodology (GRADE-based)
Regulatory Approval Landscape
FDA Approvals
- Track indication-specific approvals
- Accelerated approval vs full approval
- Post-marketing requirements
- Contraindications and warnings
EMA (European Medicines Agency)
- May differ from FDA in approved indications
- Conditional marketing authorization
- Additional monitoring (black triangle)
Regional Variations
- Health Technology Assessment (HTA) agencies
- NICE (UK): Cost-effectiveness analysis, QALY thresholds
- CADTH (Canada): Therapeutic review and recommendations
- PBAC (Australia): Reimbursement decisions
Real-World Evidence (RWE)
Sources of RWE
Electronic Health Records (EHR)
- Clinical data from routine practice
- Large patient numbers
- Heterogeneous populations (more generalizable than RCTs)
- Limitations: Missing data, inconsistent documentation, selection bias
Claims Databases
- Administrative claims for billing/reimbursement
- Large scale (millions of patients)
- Outcomes: Mortality, hospitalizations, procedures
- Limitations: Lack clinical detail (labs, imaging, biomarkers)
Cancer Registries
- SEER (Surveillance, Epidemiology, and End Results): US cancer registry
- NCDB (National Cancer Database): Hospital registry data
- Population-level survival, treatment patterns
- Limited treatment detail, no toxicity data
Prospective Cohorts
- Framingham Heart Study, Nurses' Health Study
- Long-term follow-up, rich covariate data
- Expensive, time-consuming
RWE Applications
Comparative Effectiveness
- Compare treatments in real-world settings (less strict eligibility than RCTs)
- Complement RCT data with broader populations
- Example: Effectiveness of immunotherapy in elderly, poor PS patients excluded from trials
Safety Signal Detection
- Rare adverse events not detected in trials
- Long-term toxicities
- Drug-drug interactions in polypharmacy
- Postmarketing surveillance
Treatment Patterns and Access
- Guideline adherence in community practice
- Time to treatment initiation
- Disparities in care delivery
- Off-label use prevalence
Limitations of RWE
- Confounding by indication: Sicker patients receive more aggressive treatment
- Immortal time bias: Time between events affecting survival estimates
- Missing data: Incomplete or inconsistent data collection
- Causality: Association does not prove causation without randomization
Strengthening RWE
- Propensity score matching: Balance baseline characteristics between groups
- Multivariable adjustment: Adjust for measured confounders in Cox model
- Sensitivity analyses: Test robustness to unmeasured confounding
- Instrumental variables: Use natural experiments to approximate randomization
Meta-Analysis Techniques
Binary Outcomes (Response Rate, Event Rate)
Effect Measures
- Risk Ratio (RR): Ratio of event probabilities
- Odds Ratio (OR): Ratio of odds (less intuitive)
- Risk Difference (RD): Absolute difference in event rates
Example Calculation
Study 1:
- Treatment A: 30/100 responded (30%)
- Treatment B: 15/100 responded (15%)
- RR = 0.30/0.15 = 2.0 (95% CI 1.15-3.48)
- RD = 0.30 - 0.15 = 0.15 or 15% (95% CI 4.2%-25.8%)
- NNT = 1/RD = 1/0.15 = 6.7 (treat 7 patients to get 1 additional response)
Pooling Methods
- Mantel-Haenszel: Common fixed-effect method
- DerSimonian-Laird: Random-effects method
- Peto: For rare events (event rate <1%)
Time-to-Event Outcomes (Survival, PFS)
Hazard Ratio Pooling
- Extract HR and 95% CI (or log(HR) and SE) from each study
- Weight by inverse variance
- Pool using generic inverse variance method
- Report pooled HR with 95% CI, heterogeneity statistics
When HR Not Reported
- Extract from Kaplan-Meier curves (Parmar method, digitizing software)
- Calculate from log-rank p-value and event counts
- Request from study authors
Continuous Outcomes (Quality of Life, Lab Values)
Standardized Mean Difference (SMD)
- Application: Different scales used across studies
- SMD = (Mean₁ - Mean₂) / Pooled SD
- Interpretation: Cohen's d effect size (0.2 small, 0.5 medium, 0.8 large)
Mean Difference (MD)
- Application: Same scale/unit used across studies
- MD = Mean₁ - Mean₂
- More directly interpretable than SMD
Network Meta-Analysis
Purpose
Compare multiple treatments simultaneously when no head-to-head trials exist
Example Scenario
- Drug A vs placebo (Trial 1)
- Drug B vs placebo (Trial 2)
- Drug C vs Drug A (Trial 3)
- Question: How does Drug B compare to Drug C? (no direct comparison)
Methods
Fixed-Effect Network Meta-Analysis
- Assumes consistency (transitivity): A vs B effect = (A vs C effect) - (B vs C effect)
- Provides indirect comparison estimates
- Ranks treatments by P-score or SUCRA
Random-Effects Network Meta-Analysis
- Allows heterogeneity between studies
- More conservative estimates
Consistency Checking
- Compare direct vs indirect evidence for same comparison
- Node-splitting analysis
- Loop consistency (if closed loops in network)
Interpretation Cautions
- Transitivity assumption: May not hold if studies differ in important ways
- Indirect evidence: Less reliable than direct head-to-head trials
- Rankings: Probabilistic, not definitive ordering
- Clinical judgment: Consider beyond statistical rankings
Evidence Tables
Constructing Evidence Summary Tables
PICO Framework
- P (Population): Patient characteristics, disease stage, biomarker status
- I (Intervention): Treatment regimen, dose, schedule
- C (Comparator): Control arm (placebo, standard of care)
- O (Outcomes): Primary and secondary endpoints
Evidence Table Template
Study Design n Population Intervention vs Comparator Outcome Result Quality
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Smith 2020 RCT 450 Advanced NSCLC Drug A 10mg vs Median PFS 12 vs 6 months High
EGFR+ standard chemo (95% CI) (10-14 vs 5-7) ⊕⊕⊕⊕
HR (95% CI) 0.48 (0.36-0.64)
p-value p<0.001
ORR 65% vs 35%
Grade 3-4 AEs 42% vs 38%
Jones 2021 RCT 380 Advanced NSCLC Drug A 10mg vs Median PFS 10 vs 5.5 months High
EGFR+ placebo HR (95% CI) 0.42 (0.30-0.58) ⊕⊕⊕⊕
p-value p<0.001
Pooled Effect Pooled HR 0.45 (0.36-0.57) High
(Meta-analysis) I² 12% (low heterogeneity) ⊕⊕⊕⊕
Evidence to Decision Framework
Benefits and Harms
- Magnitude of desirable effects (ORR, PFS, OS improvement)
- Magnitude of undesirable effects (toxicity, quality of life impact)
- Balance of benefits and harms
- Net benefit calculation
Values and Preferences
- How do patients value outcomes? (survival vs quality of life)
- Variability in patient values
- Shared decision-making importance
Resource Considerations
- Cost of intervention
- Cost-effectiveness ($/QALY)
- Budget impact
- Equity and access
Feasibility and Acceptability
- Is treatment available in practice settings?
- Route of administration feasible? (oral vs IV vs subcutaneous)
- Monitoring requirements realistic?
- Patient and provider acceptability
Guideline Concordance Documentation
Synthesizing Multiple Guidelines
Concordant Recommendations
Clinical Question: Treatment for HER2+ metastatic breast cancer, first-line
Guideline Summary:
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
NCCN v3.2024 (Category 1):
Preferred: Pertuzumab + trastuzumab + taxane
Alternative: T-DM1, other HER2-targeted combinations
ESMO 2022 (Grade I, A):
Preferred: Pertuzumab + trastuzumab + docetaxel
Alternative: Trastuzumab + chemotherapy (if pertuzumab unavailable)
ASCO 2020 Endorsement:
Endorses NCCN guidelines, recommends pertuzumab-based first-line
Synthesis:
Strong consensus for pertuzumab + trastuzumab + taxane as first-line standard.
Evidence: CLEOPATRA trial (Swain 2015): median OS 56.5 vs 40.8 months (HR 0.68, p<0.001)
Recommendation:
Pertuzumab 840 mg IV loading then 420 mg + trastuzumab 8 mg/kg loading then 6 mg/kg
+ docetaxel 75 mg/m² every 3 weeks until progression.
Strength: Strong (GRADE 1A)
Evidence: High-quality, multiple RCTs, guideline concordance
Discordant Recommendations
Clinical Question: Adjuvant osimertinib for resected EGFR+ NSCLC
NCCN v4.2024 (Category 1):
Osimertinib 80 mg daily × 3 years after adjuvant chemotherapy
Evidence: ADAURA trial (median DFS not reached vs 28 months, HR 0.17)
ESMO 2023 (II, B):
Osimertinib may be considered
Note: Cost-effectiveness concerns, OS data immature
NICE (UK) 2022:
Not recommended for routine use
Reason: QALY analysis unfavorable at current pricing
Synthesis:
Efficacy demonstrated in phase 3 trial (ADAURA), FDA/EMA approved.
Guideline discordance based on cost-effectiveness, not clinical efficacy.
US practice: NCCN Category 1, widely adopted
European/UK: Variable adoption based on national HTA decisions
Recommendation Context-Dependent:
US: Strong recommendation if accessible (GRADE 1B)
Countries with cost constraints: Conditional recommendation (GRADE 2B)
Quality Assessment Tools
RCT Quality Assessment (Cochrane Risk of Bias 2.0)
Domains
- Bias from randomization process: Sequence generation, allocation concealment
- Bias from deviations from intended interventions: Blinding, protocol adherence
- Bias from missing outcome data: Attrition, intention-to-treat analysis
- Bias in outcome measurement: Blinded assessment, objective outcomes
- Bias in selection of reported result: Selective reporting, outcome switching
Judgment: Low risk, some concerns, high risk (for each domain)
Overall Risk of Bias: Based on highest-risk domain
Observational Study Quality (Newcastle-Ottawa Scale)
Selection (max 4 stars)
- Representativeness of exposed cohort
- Selection of non-exposed cohort
- Ascertainment of exposure
- Outcome not present at start
Comparability (max 2 stars)
- Comparability of cohorts (design/analysis adjustment for confounders)
Outcome (max 3 stars)
- Assessment of outcome
- Follow-up duration adequate
- Adequacy of follow-up (low attrition)
Total Score: 0-9 stars
- High quality: 7-9 stars
- Moderate quality: 4-6 stars
- Low quality: 0-3 stars
Translating Evidence to Recommendations
Recommendation Development Process
Step 1: PICO Question Formulation
Example PICO:
P - Population: Adults with type 2 diabetes and cardiovascular disease
I - Intervention: SGLT2 inhibitor (empagliflozin)
C - Comparator: Placebo (added to standard care)
O - Outcomes: Major adverse cardiovascular events (3P-MACE), hospitalization for heart failure
Step 2: Systematic Evidence Review
- Identify all relevant studies
- Assess quality using standardized tools
- Extract outcome data
- Synthesize findings (narrative or meta-analysis)
Step 3: GRADE Evidence Rating
- Start at high (RCTs) or low (observational)
- Downgrade for risk of bias, inconsistency, indirectness, imprecision, publication bias
- Upgrade for large effect, dose-response, confounders reducing effect (observational only)
- Assign final quality rating
Step 4: Recommendation Strength Determination
Strong Recommendation (Grade 1)
- Desirable effects clearly outweigh undesirable effects
- High or moderate quality evidence
- Little variability in patient values
- Intervention cost-effective
Conditional Recommendation (Grade 2)
- Trade-offs: Desirable and undesirable effects closely balanced
- Low or very low quality evidence
- Substantial variability in patient values/preferences
- Uncertain cost-effectiveness
Step 5: Wording the Recommendation
Strong: "We recommend..."
Example: "We recommend SGLT2 inhibitor therapy for adults with type 2 diabetes and
established cardiovascular disease to reduce risk of hospitalization for heart failure
and cardiovascular death (Strong recommendation, high-quality evidence - GRADE 1A)."
Conditional: "We suggest..."
Example: "We suggest considering GLP-1 receptor agonist therapy for adults with type 2
diabetes and CKD to reduce risk of kidney disease progression (Conditional recommendation,
moderate-quality evidence - GRADE 2B)."
Incorporating Emerging Evidence
Early-Phase Trial Data
Phase 1 Trials
- Purpose: Dose-finding, safety
- Outcomes: Maximum tolerated dose (MTD), dose-limiting toxicities (DLTs), pharmacokinetics
- Evidence level: Very low (expert opinion, case series)
- Clinical application: Investigational only, clinical trial enrollment
Phase 2 Trials
- Purpose: Preliminary efficacy signal
- Design: Single-arm (ORR primary endpoint) or randomized (PFS comparison)
- Evidence level: Low to moderate
- Clinical application: May support off-label use in refractory settings, clinical trial enrollment preferred
Phase 3 Trials
- Purpose: Confirmatory efficacy and safety
- Design: Randomized controlled trial, OS or PFS primary endpoint
- Evidence level: High (if well-designed and executed)
- Clinical application: Regulatory approval basis, guideline recommendations
Phase 4 Trials
- Purpose: Post-marketing surveillance, additional indications
- Evidence level: Variable (depends on design)
- Clinical application: Safety monitoring, expanded usage
Breakthrough Therapy Designation
FDA Fast-Track Programs
- Breakthrough Therapy: Preliminary evidence of substantial improvement over existing therapy
- Accelerated Approval: Approval based on surrogate endpoint (PFS, ORR)
- Post-marketing requirement: Confirmatory OS trial
- Priority Review: Shortened FDA review time (6 vs 10 months)
Implications for Guidelines
- May receive NCCN Category 2A before phase 3 data mature
- Upgrade to Category 1 when confirmatory data published
- Monitor for post-market confirmatory trial results
Updating Recommendations
Triggers for Update
- New phase 3 trial results (major journal publication)
- FDA/EMA approval for new indication or agent
- Guideline update from NCCN, ASCO, ESMO
- Safety alert or drug withdrawal
- Meta-analysis changing effect estimates
Rapid Update Process
- Critical appraisal of new evidence
- Assess impact on current recommendations
- Revise evidence grade and recommendation strength if needed
- Disseminate update to users
- Version control and change log
Conflicts of Interest and Bias
Identifying Potential Bias
Study Sponsorship
- Industry-sponsored: May favor sponsor's product (publication bias, outcome selection)
- Academic: May favor investigator's hypothesis
- Independent: Government funding (NIH, PCORI)
Author Conflicts of Interest
- Consulting fees, research funding, stock ownership
- Disclosure statements required by journals
- ICMJE Form for Disclosure of Potential COI
Mitigating Bias
- Register trials prospectively (ClinicalTrials.gov)
- Pre-specify primary endpoint and analysis plan
- Independent data monitoring committee (IDMC)
- Blinding of outcome assessors
- Intention-to-treat analysis
Transparency in Evidence Synthesis
Pre-Registration
- PROSPERO for systematic reviews
- Pre-specify PICO, search strategy, outcomes, analysis plan
- Prevents post-hoc changes to avoid negative findings
Reporting Checklists
- PRISMA for systematic reviews/meta-analyses
- CONSORT for RCTs
- STROBE for observational studies
Data Availability
- Individual patient data (IPD) sharing increases transparency
- Repositories: ClinicalTrials.gov results database, journal supplements
Practical Application
Evidence Summary for Clinical Document
EVIDENCE SYNTHESIS: Osimertinib for EGFR-Mutated NSCLC
Clinical Question:
Should adults with treatment-naïve advanced NSCLC harboring EGFR exon 19 deletion
or L858R mutation receive osimertinib versus first-generation EGFR TKIs?
Evidence Review:
┌──────────────────────────────────────────────────────────────────────┐
│ FLAURA Trial (Soria et al., NEJM 2018) │
├──────────────────────────────────────────────────────────────────────┤
│ Design: Phase 3 RCT, double-blind, 1:1 randomization │
│ Population: EGFR exon 19 del or L858R, stage IIIB/IV, ECOG 0-1 │
│ Sample Size: n=556 (279 osimertinib, 277 comparator) │
│ Intervention: Osimertinib 80 mg PO daily │
│ Comparator: Gefitinib 250 mg or erlotinib 150 mg PO daily │
│ Primary Endpoint: PFS by investigator assessment │
│ Secondary: OS, ORR, DOR, CNS progression, safety │
│ │
│ Results: │
│ - Median PFS: 18.9 vs 10.2 months (HR 0.46, 95% CI 0.37-0.57, p<0.001)│
│ - Median OS: 38.6 vs 31.8 months (HR 0.80, 95% CI 0.64-1.00, p=0.046)│
│ - ORR: 80% vs 76% (p=0.24) │
│ - Grade ≥3 AEs: 34% vs 45% │
│ - Quality: High (well-designed RCT, low risk of bias) │
└──────────────────────────────────────────────────────────────────────┘
Guideline Recommendations:
NCCN v4.2024: Category 1 preferred
ESMO 2022: Grade I, A
ASCO 2022: Endorsed
GRADE Assessment:
Quality of Evidence: ⊕⊕⊕⊕ HIGH
- Randomized controlled trial
- Low risk of bias (allocation concealment, blinding, ITT analysis)
- Consistent results (single large trial, consistent with phase 2 data)
- Direct evidence (target population and outcomes)
- Precise estimate (narrow CI, sufficient events)
- No publication bias concerns
Balance of Benefits and Harms:
- Large PFS benefit (8.7 month improvement, HR 0.46)
- OS benefit (6.8 month improvement, HR 0.80)
- Similar ORR, improved tolerability (lower grade 3-4 AEs)
- Desirable effects clearly outweigh undesirable effects
Patient Values: Little variability (most patients value survival extension)
Cost: Higher cost than first-gen TKIs, but widely accessible in developed countries
FINAL RECOMMENDATION:
Osimertinib 80 mg PO daily is recommended as first-line therapy for adults with
advanced NSCLC harboring EGFR exon 19 deletion or L858R mutation.
Strength: STRONG (Grade 1)
Quality of Evidence: HIGH (⊕⊕⊕⊕)
GRADE: 1A
Keeping Current
Literature Surveillance
Automated Alerts
- PubMed My NCBI (save searches, email alerts)
- Google Scholar alerts for specific topics
- Journal table of contents alerts (NEJM, Lancet, JCO)
- Guideline update notifications (NCCN, ASCO, ESMO email lists)
Conference Monitoring
- ASCO Annual Meeting (June)
- ESMO Congress (September)
- ASH Annual Meeting (December, hematology)
- AHA Scientific Sessions (November, cardiology)
- Plenary and press releases for practice-changing trials
Trial Results Databases
- ClinicalTrials.gov results database
- FDA approval letters and reviews
- EMA European public assessment reports (EPARs)
Critical Appraisal Workflow
Weekly Review
- Screen new publications (title/abstract)
- Full-text review of relevant studies
- Quality assessment using checklists
- Extract key findings
- Assess impact on current recommendations
Monthly Synthesis
- Review accumulated evidence
- Identify practice-changing findings
- Update evidence tables
- Revise recommendations if warranted
- Disseminate updates to clinical teams
Annual Comprehensive Review
- Systematic review of guideline updates
- Re-assess all recommendations
- Incorporate year's evidence
- Major version release
- Continuing education activities