Quality Assessment Protocol Tactic
Systematically assess methodological quality and risk of bias for each included study using domain-appropriate validated tools.
Stages
Stage 1: Tool Selection
Select the appropriate quality assessment tool based on study design.
| Study Design |
Tool |
Domains |
| RCT |
RoB 2.0 |
Randomization, deviations, missing data, measurement, selection |
| Non-randomized interventions |
ROBINS-I |
Confounding, selection, classification, deviations, missing, measurement, reporting |
| Diagnostic accuracy |
QUADAS-2 |
Patient selection, index test, reference standard, flow/timing |
| Prediction models |
PROBAST |
Participants, predictors, outcome, analysis |
| Observational |
Newcastle-Ottawa |
Selection, comparability, outcome/exposure |
Decision: Match tool to study design. Use one tool consistently within a meta-analysis.
Stage 2: Per-Study Assessment
Apply the selected tool to each included study.
- Assess each domain independently
- Use signaling questions to guide domain judgments
- Assign domain-level judgment (Low/Some Concerns/High risk)
- Document supporting rationale for each judgment
- Resolve disagreements with explicit criteria
SOPs: risk-of-bias-assessment
Stage 3: Summary and Visualization Planning
Plan the summary presentation of quality assessments.
- Traffic light plot (per-study, per-domain)
- Summary bar chart (proportion at each risk level per domain)
- Overall risk-of-bias judgment per study
- Sensitivity analysis groupings (low-risk only vs all)
- GRADE certainty assessment contribution
SOPs: sensitivity-analysis-design
Minimum Yield
Per execution of this tactic:
- At least 5 studies assessed
- All domains of the selected tool evaluated per study
- Supporting rationale documented for each judgment
- Summary visualization plan produced
- Sensitivity groupings defined
Output Format
quality_assessment:
tool_used: [RoB2/ROBINS-I/QUADAS-2/PROBAST/NOS]
assessments:
- study_id: [identifier]
domains:
- domain: [name]
judgment: [Low/Some Concerns/High]
rationale: [supporting text]
overall: [Low/Some Concerns/High]
summary:
low_risk_count: [N]
some_concerns_count: [N]
high_risk_count: [N]
problematic_domains: [most common high-risk domains]
sensitivity_groups:
low_risk_only: [study list]
excluding_high_risk: [study list]
Available SOPs
Optional, no fixed order; the final leaf is always a sop.
| SOP |
When to use |
| risk-of-bias-assessment |
Assess methodological bias using RoB2, PROBAST, or QUADAS-2 validated tools |
| sensitivity-analysis-design |
Design leave-one-out, influence diagnostics, subgroup analyses, and robustness checks |
1---2name: quality-assessment-protocol3description: Methodological quality and bias risk assessment of included studies using validated tools4---56# Quality Assessment Protocol Tactic78Systematically assess methodological quality and risk of bias for each included study using domain-appropriate validated tools.910## Stages1112### Stage 1: Tool Selection1314Select the appropriate quality assessment tool based on study design.1516| Study Design | Tool | Domains |17|--------------|------|---------|18| RCT | RoB 2.0 | Randomization, deviations, missing data, measurement, selection |19| Non-randomized interventions | ROBINS-I | Confounding, selection, classification, deviations, missing, measurement, reporting |20| Diagnostic accuracy | QUADAS-2 | Patient selection, index test, reference standard, flow/timing |21| Prediction models | PROBAST | Participants, predictors, outcome, analysis |22| Observational | Newcastle-Ottawa | Selection, comparability, outcome/exposure |2324**Decision**: Match tool to study design. Use one tool consistently within a meta-analysis.2526### Stage 2: Per-Study Assessment2728Apply the selected tool to each included study.2930- Assess each domain independently31- Use signaling questions to guide domain judgments32- Assign domain-level judgment (Low/Some Concerns/High risk)33- Document supporting rationale for each judgment34- Resolve disagreements with explicit criteria3536**SOPs**: risk-of-bias-assessment3738### Stage 3: Summary and Visualization Planning3940Plan the summary presentation of quality assessments.4142- Traffic light plot (per-study, per-domain)43- Summary bar chart (proportion at each risk level per domain)44- Overall risk-of-bias judgment per study45- Sensitivity analysis groupings (low-risk only vs all)46- GRADE certainty assessment contribution4748**SOPs**: sensitivity-analysis-design4950## Minimum Yield5152Per execution of this tactic:53- At least 5 studies assessed54- All domains of the selected tool evaluated per study55- Supporting rationale documented for each judgment56- Summary visualization plan produced57- Sensitivity groupings defined5859## Output Format6061```yaml62quality_assessment:63 tool_used: [RoB2/ROBINS-I/QUADAS-2/PROBAST/NOS]64 assessments:65 - study_id: [identifier]66 domains:67 - domain: [name]68 judgment: [Low/Some Concerns/High]69 rationale: [supporting text]70 overall: [Low/Some Concerns/High]71 summary:72 low_risk_count: [N]73 some_concerns_count: [N]74 high_risk_count: [N]75 problematic_domains: [most common high-risk domains]76 sensitivity_groups:77 low_risk_only: [study list]78 excluding_high_risk: [study list]79```8081<!-- BEGIN available-tables (generated) -->8283## Available SOPs8485Optional, no fixed order; the final leaf is always a sop.8687| SOP | When to use |88| --- | --- |89| risk-of-bias-assessment | Assess methodological bias using RoB2, PROBAST, or QUADAS-2 validated tools |90| sensitivity-analysis-design | Design leave-one-out, influence diagnostics, subgroup analyses, and robustness checks |9192<!-- END available-tables (generated) -->
yogsoth-ai