Evidence Synthesizer

Evidence Synthesizer

Goal

Turn multiple studies or sources into a coherent, confidence-aware evidence picture.

The job of this skill is not to force consensus. The job is to summarize what the literature supports, where it conflicts, what is still weak, and how strong the overall conclusion should be.

This skill synthesizes evidence. It does not replace literature search, and it should not silently smooth over low-quality or conflicting studies.

Default Posture

• synthesize after search and appraisal
• conflicts are information, not clutter
• evidence strength before narrative elegance
• quality spread matters
• uncertainty stays visible

When To Use

Run this skill when:

• multiple relevant studies have already been found
• at least light appraisal has been done or is possible
• the user wants to know what the overall evidence says
• a claim-strength judgment depends on seeing the full evidence picture
• scientific-method-selector routes to synthesis

This is a strong checkpoint before:

• making a research-backed conclusion
• comparing competing scientific claims
• identifying research gaps from the current literature

Scope Boundaries

In scope:

• summarize direction of evidence across sources
• note consistency, conflict, and quality spread
• state what seems supported, tentative, or weak
• keep confidence proportional to the literature base

Out of scope by default:

• pretending to be a full systematic review
• meta-analysis style quantitative pooling by default
• hiding conflicting studies to make the answer cleaner
• turning weak literature into firm guidance

Synthesis Workflow

1. State the target claim or question.
2. Group the relevant studies by:
   • study type
   • population or domain
   • intervention or exposure
   • outcome
3. Note the overall quality spread:
   • stronger studies
   • mixed studies
   • weak or indirect studies
4. Identify where the literature converges.
5. Identify where it conflicts or stays thin.
6. State the most defensible overall conclusion.
7. Name what would most improve confidence.

Escalation Conditions

Pause and keep the synthesis conservative when:

• the candidate set is still too noisy or weakly screened
• study quality varies so much that simple summary would mislead
• the underlying claim is broader than the evidence base
• the conflict is substantial enough that a clean overall conclusion would overstate certainty

If those conditions persist, recommend returning to study-quality-appraiser, literature-search, or a narrower claim before stronger synthesis.

Synthesis Heuristics

Prefer stronger overall confidence when:

• multiple higher-quality studies point in the same direction
• the literature directly matches the target claim
• outcome measures and populations are reasonably aligned
• conflicts are limited and explainable

Reduce overall confidence when:

• most evidence is observational, preclinical, or indirect
• the strongest studies are few or small
• outcomes, populations, or interventions are too heterogeneous
• conflict is substantial and unresolved
• the claim being made is broader than the literature actually covers
• the evidence base is dominated by preprints or non-reviewed sources

Certainty Lens

When summarizing the overall evidence, explicitly consider:

• risk_of_bias
• inconsistency
• indirectness
• imprecision
• publication_bias_or_reporting_limits

Use these dimensions to justify the confidence level instead of giving a bare confidence label.

Preprint-Aware Synthesis

When a meaningful share of the source set is preprint-only:

• state that the synthesis is discovery-rich but lower-certainty
• avoid language that implies settled consensus
• distinguish "emerging pattern" from "established support"
• recommend stronger reviewed corroboration when the claim matters

Output Contract

Always return:

1. target_question
2. source_set
3. evidence_direction
4. consistency_assessment
5. quality_spread
6. main_conflicts_or_limits
7. certainty_dimensions
8. preprint_load (none, low, mixed, high)
9. overall_confidence (high, moderate, low, very-low)
10. most_defensible_conclusion
11. recommended_followup
12. next_step

Guardrails

• Do not let quantity of papers substitute for quality of evidence.
• Do not hide conflicts just to produce a cleaner answer.
• Do not upgrade confidence because results are intuitively appealing.
• Keep preprint-heavy or low-quality evidence clearly marked.
• If the literature only supports a narrow claim, do not generalize beyond it.
• If preprints carry much of the signal, keep the conclusion explicitly provisional.

Anti-Patterns

• fake consensus
• narrative smoothing
• quality-blind averaging
• broad conclusion from narrow evidence
• calling the evidence "mixed" without explaining what is actually mixed

Example

User request:

We found several papers on cold exposure and mood. What does the overall evidence really support?

Expected shape of response:

1. target_question: what the current literature supports about cold exposure and mood-related outcomes
2. source_set: small set of relevant human and possibly indirect studies
3. evidence_direction: some positive short-term signals, weak support for broader mental health claims
4. consistency_assessment: partially consistent but limited and heterogeneous
5. quality_spread: mostly weak-to-moderate, with few strong studies
6. main_conflicts_or_limits: small samples, protocol variation, short follow-up, proxy outcomes
7. overall_confidence: low
8. most_defensible_conclusion: evidence is suggestive for short-term mood effects but far from established for broad mental health improvement
9. recommended_followup: narrower claim check or broader targeted literature search
10. next_step: separate short-term mood effects from broader psychiatric outcomes before making a stronger judgment