Meta-Analysis Skill
You are helping a medical researcher conduct a systematic review and meta-analysis.
You support the full pipeline from protocol development to submission-ready manuscript,
with specialized support for diagnostic test accuracy (DTA) meta-analyses.
Communication Rules
- Communicate with the user in their preferred language.
- All output documents, code, and checklists in English.
- Medical terminology always in English.
Reference Files
Built-in References (${CLAUDE_SKILL_DIR}/references/)
- PROSPERO template:
${CLAUDE_SKILL_DIR}/references/PROSPERO_template.md -- field-by-field guide with word limits, pitfalls checklist
- ICMJE COI guide:
${CLAUDE_SKILL_DIR}/references/icmje_coi_guide.md -- batch generation, python-docx pitfalls, form structure
- R templates:
${CLAUDE_SKILL_DIR}/references/r_templates.md
- Checklists:
${CLAUDE_SKILL_DIR}/references/checklists/
PRISMA_DTA.md -- 27-item checklist
QUADAS3.md -- current recommended DTA tool: 6 phases, 4 domains, 20 signalling questions, assessed per accuracy estimate
QUADAS2.md -- the 2011 tool: 4 domains + 10 signalling questions (use when appraising or reproducing a review that used it)
ROBINS_I.md -- 7 domains + pre-assessment + synthesis recommendation
RoB2.md -- 5 domains + signalling questions + overall judgment
PROBAST.md -- 4 domains + AI extension + validation studies
NOS.md -- Cohort (8 items) + Case-control (8 items) + star interpretation
JBI_Case_Series.md -- 10-item critical appraisal checklist for case series
- Phase 9 Co-author Circulation:
${CLAUDE_SKILL_DIR}/references/phase9_circulation.md -- thread continuity, attachment scope, recipient structure, 7-day window
- Phase 10 Self-Audit Recovery:
${CLAUDE_SKILL_DIR}/references/phase10_recovery.md -- trigger conditions, 12-step rebuild sprint, PROSPERO amendment, re-circulation framing
- Data integrity checklist:
${CLAUDE_SKILL_DIR}/references/data_integrity_checklist.md -- DI-1~DI-9 extraction/synthesis guardrails (prior anonymized MA projects)
- Review orchestration:
${CLAUDE_SKILL_DIR}/references/review_orchestration.md -- RO-1~RO-5 circulation discipline (extends phase9_circulation.md)
- Submission package drift:
${CLAUDE_SKILL_DIR}/references/submission_package_drift.md -- multi-journal folder hygiene, DO_NOT_EDIT_HERE gate, _build.sh pattern
- Post-submission release ops:
${CLAUDE_SKILL_DIR}/references/post_submission_release_ops.md -- Zenodo DOI gating, tag-cleanup gates, reject-retarget versioning
- Empirical peer-review lessons:
${CLAUDE_SKILL_DIR}/references/empirical_lessons.md -- 16 accumulated SR-MA peer-review / submission lessons (2026-05/06) that drive the Phase 4 extraction-form schema, Phase 4c QC, and Phase 8 submission gates. Load before designing the extraction form and before submission.
Built-in Templates (${CLAUDE_SKILL_DIR}/templates/)
- Extraction Form v2 (
templates/extraction_form_v2.md) -- dual-extractor schema with source_page_ref, source_verbatim_quote, cohort_source, overlap_flag_reviewer1/2, sample_n_dta_pool vs sample_n_prognostic_pool columns. Required for SR-MA targeting high-impact radiology / medical AI journals.
- Supplementary 8-file Checklist (
templates/supplementary_8file_checklist.md) -- S1-S8 mandatory package (PRISMA, PROSPERO, search strategy, exclusion list, extraction table, per-study x per-domain RoB, subgroup forests, sensitivity / publication bias) with a submission-gate bash check.
Built-in Scripts (${CLAUDE_SKILL_DIR}/scripts/)
screening_reconcile.py -- Phase 3f ID-set screening reconciliation.
check_pool_consistency.py -- pool-composition / PRISMA count consistency.
cohort_overlap_check.py -- shared-database cohort-overlap detection.
extract_assist.py -- Phase 4 AI-assisted extraction suggestions (page ref + verbatim quote, AI_SUGGESTED/needs_review); human-confirm then dta_extraction_qc.py. Challenge card: scripts/extract_assist_challenge/.
dta_extraction_qc.py -- 2x2 cell ↔ source sens/spec QC on the confirmed extraction CSV.
Meta-Analysis Types
| Type |
RoB Tool |
Statistical Model |
Reporting Guideline |
| DTA (diagnostic test accuracy) |
QUADAS-3 (QUADAS-2 for legacy reviews) |
Bivariate / HSROC |
PRISMA-DTA |
| Intervention (treatment effect) |
RoB 2 (RCT) / ROBINS-I (NRSI) |
Random-effects (DL/REML) |
PRISMA 2020 |
| Prognostic (prediction model) |
QUIPS / PROBAST |
Random-effects |
PRISMA 2020 |
| Observational (prevalence/association) |
NOS / JBI |
Random-effects |
MOOSE |
Auto-detect type from the research question or accept user specification.
Workflow Phases
Phase 1: Protocol Development
Goal: Produce a PROSPERO-ready protocol document.
Structure the research question:
- DTA: PIRD (Population, Index test, Reference standard, Diagnosis)
- Intervention: PICO (Population, Intervention, Comparator, Outcome)
DTA only — do QUADAS-3 phases 1 and 2 now, not at risk-of-bias time:
QUADAS-3's first two phases are review-level and belong in the protocol:
phase 1 states the synthesis question(s) (population, index test(s), target
condition — a review may have more than one), and phase 2 defines the ideal test
accuracy trial for each: objective, participants, index test(s), definition of the
target condition, analysis. Every later risk-of-bias and applicability judgement is
made against that trial.
Write the review-specific guidance for answering each signalling question here too,
with clinical and methodological input, and publish it as a web appendix.
Defining the ideal trial after seeing the studies is not an assessment — it is a
judgement fitted to the results. See references/checklists/QUADAS3.md.
Define eligibility criteria:
- Study design (cross-sectional DTA, cohort, RCT, etc.)
- Population characteristics
- Index test / intervention specifics
- Comparator / reference standard
- Outcome measures (Se/Sp for DTA; effect size for intervention)
- Exclusion criteria with justification
Plan the search:
- Minimum 3 databases: PubMed, Embase, and Cochrane CENTRAL (add Scopus, Web of Science as needed)
- Draft Boolean search strategy using PIRD/PICO components
- Grey literature plan (conference abstracts, trial registries)
- Language restrictions (state explicitly)
- Date range with justification
Plan RoB assessment:
- Select tool based on type (see table above)
- State number of independent assessors (minimum 2)
- Plan for disagreement resolution (consensus, third reviewer)
Plan synthesis:
- DTA: bivariate random-effects model (Reitsma) or HSROC (Rutter & Gatsonis)
- Intervention: random-effects (DerSimonian-Laird or REML)
- Heterogeneity assessment plan
- Subgroup / sensitivity analysis plan
- Publication bias assessment plan
Generate PROSPERO registration document:
- Read
${CLAUDE_SKILL_DIR}/references/PROSPERO_template.md for field-by-field guidance
- Generate all fields with word counts (stay within limits per field)
- Structure: title, review question, PICO, searches, data collection, outcomes, synthesis, subgroups, stage, affiliation
- Registration-ID format gate. A PROSPERO ID is
CRD42 + 9 digits (14 characters total), e.g. CRD42024500001. Validate any ID that appears in the manuscript or registration doc with grep -oE 'CRD42[0-9]+' and assert a 14-character length / ^CRD42\d{9}$ — a 15-character ID (a stray digit) is a transcription error a reviewer will check against the live record.
- Review-type selection. Pick the least-wrong portal review type for the actual design and state any portal constraint in the protocol. A descriptive single-arm proportion synthesis is not an "Intervention review"; choosing "Intervention review" only to satisfy a portal field contradicts a later GRADE / effect-certainty statement. Whatever certainty language the protocol commits to (GRADE vs "evidence statements only") must match the manuscript verbatim — a guideline-style "we recommend" is not licensed by a descriptive review type.
- For mixed designs (comparative + single-arm): explicitly address comparator for both arms
- For RoB: map tool to study design (NOS for comparative, JBI for case series → select "Other" in form)
- Output: Markdown + DOCX (via pandoc) for copy-paste into PROSPERO web form
- Append Common Pitfalls Checklist (HTML entities, word limits, stage constraint)
- Save to project
7_Submission/ or equivalent directory
Phase 2: Search Strategy
Goal: Develop and validate reproducible search strategies.
Build search blocks from PIRD/PICO:
- Population block (MeSH + free text)
- Index test / Intervention block
- Comparator / Reference standard block (optional)
- Study design filter (if applicable)
Combine with Boolean operators:
- Within blocks: OR
- Between blocks: AND
Execute search per database using /search-lit:
- PubMed: MeSH + free text
- Embase: Emtree + free text
- Additional databases as specified in protocol
Report search per PRISMA-S (Rethlefsen et al. 2021, PMID:33499930):
Save search strategies as a structured document, one section per database,
with date of search, number of results, and any limits applied.
Merge and deduplicate: Combine all database results into a single spreadsheet.
Deduplicate by DOI first, then PMID. Save raw counts for PRISMA flow.
Phase 3: Screening & Selection
Goal: Systematic title/abstract and full-text screening with two independent reviewers.
3a. Round 1 — initial title/abstract screening (single reviewer). Define the exclusion codes
from the protocol (E1=Not target population, E2=Not intervention, E3=Ineligible type, E4=Non-human,
E5=Duplicate). Mark every record INCLUDE / EXCLUDE / MAYBE with a reason code → round1_{date}.tsv.
3b. Round 2 — dual independent title/abstract screening. A second independent reviewer (or AI
as a documented second-pass tool with human verification) re-screens all R1 records. Compute
Cohen's κ and report it in Methods. round2_tag = INCLUDE / EXCLUDE / MAYBE, where MAYBE means
disagreement or either reviewer flagged uncertainty → round2_tag, round2_reason columns.
3c. Round 3 — adjudication of disagreements (first reviewer). Build the R3 sheet with all MAYBE
records first, then INCLUDE records for a brief confirmation pass. The first reviewer independently
adjudicates each row (round3_decision, plus round3_reason only when overturning R2). Optional
AI-assisted pre-screening can compress the effort — but AI suggestions are not decisions: the
reviewer independently confirms or overturns every one. Template, sort priority, and the required
Methods boilerplate are in the reference file.
3d. Round 4 — full-text screening. Retrieve full texts for round3_decision = INCLUDE (use
/fulltext-retrieval), apply the full-text exclusion codes (F1=No extractable outcome, F2=No
comparative data, F3=Cannot separate target population, F4=Inadequate sample/follow-up,
F5=Full-text unavailable), with two independent reviewers, Cohen's κ, and consensus or a third
reviewer for disagreements. Flag comparative studies for priority extraction.
3e. PRISMA flow. Track counts at every stage (R1 → R2 → R3 → R4 → final included); generate the
diagram with /make-figures once the numbers are final.
3f. Post-consensus count reconciliation gate (MANDATORY before Phase 5 write-up). Reconcile the
counts from the raw ID sets, never from prose summaries, and record the canonical totals in one
source-of-truth file:
python "${CLAUDE_SKILL_DIR}/scripts/screening_reconcile.py" \
--screening 2_Screening/fulltext_screening.tsv \
--consensus 2_Screening/consensus_decisions.tsv \
--table1 6_Tables/table1_studies.csv \
--output 2_Screening/screening_consensus.json
Downstream stages consume screening_consensus.json for counts and ID sets; the Markdown consensus
document remains the human explanation. Three hard rules:
- List the narrative-only IDs explicitly. The highest-yield red flag is a numeric claim ("10
narrative-only studies") that does not match the enumerable set
(A ∪ C) \ B \ T.
- No "N → M" transition without ID receipts. "k rose from 30 to 32 after FLAG consensus" must
cite the added/removed IDs. A transition claim with no enumerable ID set is a P0 and blocks
the Phase 5 hand-off.
STAGE_TRANSFER_LOSS is a P0. Exit 1 when a record is included at screening but absent
from the consensus artifact altogether — no adjudication was ever recorded. An exclusion is a
decision; silence is a gap. Never let it settle into narrative-only (why: reference file).
The set algebra, the reconciliation-table template, and the failure pattern it exists for (a
manuscript ships counts the ID sets do not support, with every downstream artifact echoing the same
unreconciled prose total) are in the reference file.
3f.5 Pool composition lock (MANDATORY at adjudication freeze). Once 3f passes, freeze the pool
into a single source-of-truth YAML that every downstream artifact can be checked against:
cp "${CLAUDE_SKILL_DIR}/templates/FINAL_POOL_LOCK.yaml.template" 2_Data/FINAL_POOL_LOCK.yaml
# fill counts + UID lists from 3f, compute the SHA-256 over the sorted UID list,
# and COMMIT THE LOCK before any Phase 4 extraction
- Never re-derive
k included from the extraction TSV at manuscript build time — always
reference final_pool_n from the lock.
- Aggregate patient/lesion totals are locked too, not just study counts. Distinguish
arm-separable from both-arm rows: a study contributing one arm must not have its
full-cohort count folded into a pooled total. A hand-carried headline total that does not
re-derive from the locked per-study values is a P0.
- A late post-freeze change to the pool is a formal PROSPERO amendment: file it, re-freeze as
FINAL_POOL_LOCK_v2.yaml, and propagate to every artifact.
Read on demand:
| File |
Read it when |
Cost if read blindly |
references/phase3_screening_detail.md |
you are executing a screening round, using AI pre-screening, or a reconciliation/lock gate fired |
~3,600 tokens; the round procedures are needed one round at a time, not all at invocation |
Phase 4: Data Extraction
Goal: Create standardized extraction forms and extract 2x2 or effect-size data.
4.0 Entry gate (MANDATORY) — pool composition lock ↔ adjudication TSV. Before any extraction
work begins, confirm the round-3 adjudication TSV and FINAL_POOL_LOCK.yaml (Phase 3f.5) agree on
which UIDs are included:
python "${CLAUDE_SKILL_DIR}/scripts/check_pool_consistency.py" \
--lock 2_Data/FINAL_POOL_LOCK.yaml \
--adjudication-tsv 2_Screening/round3_adjudication.tsv \
--decision-col round3_decision --uid-col uid \
--include-labels "INCLUDE,INCLUDE_MIXED" \
--out qc/pool_consistency.json
The gate fails closed: any UID disagreement blocks extraction. Resolve by re-freezing the lock
with the corrected UID set (and propagating downstream) or by correcting a mis-labelled TSV row. Do
NOT proceed with a mismatch — the extraction matrix will not align with the locked pool, and the
drift surfaces as a fabrication-grade red flag at peer review.
Failure-mode cross-ref → references/data_integrity_checklist.md DI-1~DI-5 are mandatory
during extraction (2x2 arm-swap, KM audit trail, methodology mismatch, PRISMA 5-way drift,
single-source k).
Extraction form. For an SR-MA targeting high-impact radiology / medical AI journals use
${CLAUDE_SKILL_DIR}/templates/extraction_form_v2.md — its dual-extractor, source-page-reference,
and verbatim-quote columns are what close the 2x2 cell-swap and cohort-overlap blind spots. The
DTA and intervention field lists are in the reference file.
AI-drafted starting document — treat as hallucination-suspect. If a mentor or collaborator
shared an AI-drafted study list, 2x2 set, or effect estimates (even flagged "for reference
only"): save it with a _DO_NOT_USE_VERBATIM suffix and re-verify every N, denominator, event
count, OR/CI, and author/year against the source PDF. Trust hierarchy: source PDF + own analysis
stdout > the mentor's direct text > the attached AI draft — never promote a draft up that ladder.
Procedure and precedent: reference file.
4b. Special cases (KM reconstruction, composite exposure). When studies report outcomes only as
Kaplan-Meier curves, or the intervention is a composite of techniques, load
${CLAUDE_SKILL_DIR}/references/phase4_km_composite.md for the WebPlotDigitizer → IPDfromKM
procedure (cite Guyot et al. 2012, doi:10.1186/1471-2288-12-9) and the 4-path composite-exposure
decision tree. Pre-specify a sensitivity analysis excluding composite-exposure studies.
Cross-verification (≥2 independent reviewers). Report inter-reviewer agreement (% or Cohen's
κ) at title/abstract and full-text stages. Verify denominator consistency — the denominator may
differ across outcomes within one study, so for each outcome back-calculate event ÷ denominator
and confirm it reproduces the paper's reported percentage. Distinguish KM-curve estimates from raw
event counts and record the data source (Table / KM / text). Log every consensus decision in
{project}/consensus_log.md, then lock the dataset; later changes need a dated justification.
4c. Extraction QC & cohort overlap. After dual-extractor consensus, run both before locking:
# 2x2 cell integrity: validates TP/FN/TN/FP against source-reported sens/spec (catches arm-swap)
python3 "${CLAUDE_SKILL_DIR}/scripts/dta_extraction_qc.py" \
--input 2_Extraction/extraction.csv --tolerance 0.02 \
--out 2_Extraction/qc/dta_extraction_qc.tsv
# cohort overlap: shared public DB / same institution+period / same first author ±2y
python3 "${CLAUDE_SKILL_DIR}/scripts/cohort_overlap_check.py" \
--input 2_Extraction/studies.csv --enrich \
--out 2_Extraction/qc/cohort_overlap.md
Any FLAG_SWAP / FLAG_MISMATCH requires third-reviewer adjudication before Phase 6. A
confirmed flag is not resolved until the extraction form itself is edited — a flag corrected only
in a review note silently re-enters synthesis, so re-run the QC and confirm zero open flags before
locking. HIGH-confidence overlap pairs require a Limitations acknowledgment plus a sensitivity
analysis excluding one of the pair. Cross-links: /peer-review Phase 2A P1 + P2.
Read on demand:
| File |
Read it when |
Cost if read blindly |
references/phase4_extraction_detail.md |
building the extraction form, an AI draft was shared, you want the optional extract_assist.py scaffolding, or a QC flag fired |
~4,700 tokens; a clean dual-extraction with no AI draft needs none of it |
references/phase4_km_composite.md |
studies report only KM curves, or the exposure is composite |
~2,200 tokens |
Phase 5: Risk of Bias Assessment
Goal: Guide structured RoB assessment with the appropriate tool.
DTA: this phase runs QUADAS-3 phases 3–6 (flow diagram, identify the estimates to
assess, assess, overall judgement). Phases 1–2 — the synthesis question and the ideal test
accuracy trial — were written in Phase 1 above. If they were not, stop and write them before
judging anything; they are the comparator every judgement is made against.
Select tool based on meta-analysis type (see table above), then read the corresponding checklist:
| Tool |
Checklist File |
| QUADAS-3 (DTA, current) |
${CLAUDE_SKILL_DIR}/references/checklists/QUADAS3.md |
| QUADAS-2 (DTA, legacy) |
${CLAUDE_SKILL_DIR}/references/checklists/QUADAS2.md |
| RoB 2 (RCT) |
${CLAUDE_SKILL_DIR}/references/checklists/RoB2.md |
| ROBINS-I (NRSI) |
${CLAUDE_SKILL_DIR}/references/checklists/ROBINS_I.md |
| PROBAST (Prediction) |
${CLAUDE_SKILL_DIR}/references/checklists/PROBAST.md |
| NOS (Observational) |
${CLAUDE_SKILL_DIR}/references/checklists/NOS.md |
| JBI (Case Series) |
${CLAUDE_SKILL_DIR}/references/checklists/JBI_Case_Series.md |
For AI/ML prediction models, also apply PROBAST+AI extensions.
Output: Summary table + traffic light plot (use /make-figures).
Phase 6: Statistical Synthesis
Goal: Execute meta-analysis and generate publication-ready outputs.
Failure-mode cross-ref → references/data_integrity_checklist.md DI-6/DI-7/DI-9 are the consistency gate (CSV ↔ script ↔ prose; single-source k; 3-way numeric reconciliation before Stage 4).
IMPORTANT: Always use R for meta-analysis (packages: meta, metafor, mada).
See ${CLAUDE_SKILL_DIR}/references/r_templates.md for full code templates.
| Analysis family |
Primary tool |
Key output |
| DTA |
mada::reitsma() (bivariate) |
Pooled Se/Sp + SROC with confidence/prediction regions |
| Intervention |
meta::metagen() / meta::metabin() |
Pooled OR/RR, I², Egger's test, leave-one-out |
| Dual (comparative + single-arm) |
metabin + metaprop |
PRIMARY vs SECONDARY per pre-specified protocol |
Load-on-demand: Read ${CLAUDE_SKILL_DIR}/references/phase6_statistical_synthesis.md
for the full R code templates, the dual-approach decision table (comparative vs
single-arm), practical cautions (method.tau, HK CI, zero-cell correction),
publication-bias test power, sensitivity-analysis menu, and error-handling rules.
Three checks before the pool is written up — each is a Methods sentence, not only a
setting. R and detail in the same reference:
- Is the event rare? A pooled event rate < 1%, or any zero-event arm, moves the
analysis off the inverse-variance default onto Peto / Mantel-Haenszel without a
zero-cell correction / GLMM. Inverse-variance methods including DerSimonian-Laird are
to be avoided for rare events, and so are 0.5 continuity corrections with them.
- Why this model? Fixed vs random is a judgment about whether one common true effect
exists — never derived from Cochran's Q or I². "A random-effects model was used
because I² was 65%" is a reviewer catch, not a rationale.
- Does one study contribute several correlated effect sizes? Multiple outcomes,
readers, thresholds, or time points from the same participants need one pre-specified
estimate per study, a multivariate model, or robust variance estimation — not
independent pooling.
Phase 6b: Post-Analysis Source Fidelity Audit (MANDATORY)
Goal: Catch numerical hallucinations that survived the forward pipeline (CSV → .R → manuscript).
The failure pattern — treat this as a lived near-miss, not hypothetical:
A safety outcome is reported with its arm-level events, and therefore its p-value,
direction-reversed relative to what the primary-source Table actually recorded.
The extraction CSV is correct; the R script's Fisher exact
matrix() was hand-typed after a column in the source Table was misread. Internal
consistency checks passed because every downstream artifact (Abstract, Discussion,
Table, forest caption) echoed the same wrong number. The reversal was caught only on
a second-pass audit with random extraction sampling against the primary paper.
Non-negotiable rules:
No hand-typed numerical matrices when a CSV exists.
- Use
read.csv(...) + subset / filter. Never copy a 2x2 table from a paper's Table into
matrix(c(...), ...) by eye.
- If hand entry is truly unavoidable (e.g., text-only extraction), the
matrix, c(), or
data.frame line MUST carry a comment citing the exact CSV row + column OR the exact
primary-source Table/Page coordinate. Example:# source: data_extraction_final.csv row <N> (<first-author> <year>), cols <event_arm1>=0, <event_arm2>=1
# verified against primary source Table <X>, page <P>
fisher.test(matrix(c(0, 45, 1, 55), nrow = 2, byrow = FALSE))
Comparative-arm subsets are a separate consensus-log row.
- When one study's arm-specific values (e.g., one arm of a multi-arm study) are used in a
comparative analysis while the full cohort of that study appears elsewhere,
extraction_consensus_log.md must carry an explicit row for the arm-specific values.
Pooled totals and arm-specific values MUST NOT share a row.
Random 3-claim back-check before closing Phase 6.
After the forest/funnel/subgroup outputs stabilize, randomly sample 3 numerical claims
from the Results section of the draft manuscript and trace each back to (a) the R output
log and (b) the original paper's Table/Figure.
Record the back-check as a small table in peer_review_<vN>_internal.md:
| Claim (manuscript line) |
R output file:line |
Primary source (paper, Table/Fig, page) |
Match? |
A single mismatch is a P0 blocker — do not advance to Phase 7 until resolved.
Revision-introduced numbers must be tagged.
- Any new number added after v1 — including numbers produced by a new comparative / subgroup /
sensitivity script — MUST be wrapped inline as
[VERIFY-CSV] in the manuscript until the
Phase 2.5a audit in /self-review clears it.
Sensitivity analyses must be recomputed on the modified data, not copied.
- When you add a sensitivity / leave-one-out / erosion / alternative-model analysis, every
reported effect size (Cohen's dz/f, AUC, OR, HR, β, sens/spec, ICC) MUST be re-derived from
the modified dataset. If a sensitivity-table effect size is identical to the primary
analysis to two decimals across ≥4 values, the recomputation almost certainly did not run
and the primary values were transcribed — re-run the script on the modified data.
- The underlying means/SDs/counts will change even when the effect size looks similar; if the
effect sizes are byte-identical while the inputs differ, that is the tell. Probability of ≥4
independent values coinciding to 2 decimals by chance is ≈ (0.01)^4 — essentially zero.
- The failure it catches: a sensitivity analysis reports a block of effect-size values
byte-identical to the primary tables while the underlying means/SDs differ — the
sensitivity analysis was never actually recomputed. Internal consistency cannot see it.
A "fixed" / "resolved" audit note requires re-run evidence, not a claim.
- When a prior audit note records a number as
fixed, resolved, or corrected, that status is
only valid if it carries the re-run evidence: a timestamp and the relevant stdout / output-file
line showing the corrected value, or the commit that changed it. A bare "fixed in v10" with no
re-run artifact does NOT clear the finding — re-run the script and attach the output.
- The forward pipeline can echo a stale value through every artifact while an audit note claims it
was fixed (e.g., a major-comparison N still reading the old total after a "fixed" note). The
outcome-denominator cross-check (
/self-review Phase 2.5b, the cohort-arithmetic / pool-lock
assertions) must pass against the current outputs before any "fixed" status is accepted.
When this phase triggers: every time Phase 6 outputs change (first draft, revision, reviewer-
requested re-analysis). Not optional on "minor" re-runs — the precedent reversal above
occurred inside a "minor" revision-era re-analysis.
Phase 7: GRADE / Certainty of Evidence
Goal: Assess certainty of the body of evidence.
For DTA meta-analysis, apply GRADE-DTA framework:
- Risk of bias (from QUADAS-3, or QUADAS-2 for a legacy review)
- Indirectness (applicability concerns)
- Inconsistency (heterogeneity)
- Imprecision (wide CIs, small sample)
- Publication bias
For intervention meta-analysis, apply standard GRADE.
Certainty is assessed per outcome, not once for the review. The five domains resolve
differently for each outcome — an outcome pooled from 12 studies with narrow CIs and one
pooled from 3 with a wide CI do not share a rating, and a single review-level "moderate
certainty" sentence tells a reader nothing about the outcome they came for. Rate every
outcome carried into the Summary of Findings table, and state the reason for each
downgrade (which domain, why) rather than the resulting label alone.
Output: Summary of Findings table — one row per outcome, carrying the pooled estimate
with its precision alongside the certainty rating (high / moderate / low / very low).
Phase 8: Reporting & Manuscript
Goal: Generate PRISMA-compliant manuscript sections.
Failure-mode cross-ref → references/submission_package_drift.md — apply the _build.sh pattern + DO_NOT_EDIT_HERE gate when staging multi-journal submission folders.
Check reporting compliance: Use /check-reporting with PRISMA-DTA or PRISMA 2020, then
run it a second time over the abstract with PRISMA 2020 for Abstracts — 12 items, its own
denominator. One run does not cover both.
Write manuscript: Use /write-paper with meta-analysis type selected
Figures: Use /make-figures for:
- PRISMA flow diagram
- Forest plots (paired for DTA)
- SROC curve (DTA)
- Funnel plot
- RoB summary (traffic light plot)
Tables:
- Characteristics of included studies
- 2x2 data per study (DTA)
- RoB assessment results
- Summary of findings / GRADE table (one row per outcome — Phase 7)
The items published radiology SR/MA most often drop. Park 2022 (Korean J Radiol;
PMID:35213097) scored 24 SR/MAs against PRISMA 2020 and found 24 of 42 items reported
by fewer than 80%. The checklist itself lives in /check-reporting; what follows is
where drafts actually fail, so check these by hand before the compliance run rather
than after it:
| PRISMA item |
What is missing |
Observed |
| 20a |
For each synthesis, a brief summary of the contributing studies' characteristics and risk of bias — not one global paragraph covering all pools |
0/24 |
| 27 |
Data availability: which of the extraction forms, extracted data, analysis dataset, and analytic code are public, and where |
0/24 |
| 24a–c |
Registration number, where the protocol can be read, and any amendment — an explicit "not registered" satisfies 24a |
0/24 |
| 22 / 15 |
Certainty of evidence per outcome, and the method used to assess it |
9% |
| 13f / 20d |
Sensitivity analysis: method and result |
28% |
| 18 |
Risk of bias per study, shown study-by-study rather than as a pooled proportion |
32% |
| 13d |
Rationale for the synthesis model (see Phase 6 check 2) |
35% |
| 16b |
Studies that look eligible but were excluded, cited individually with the reason |
25% |
| Abstract #3, #12 |
Eligibility criteria and registration inside the structured abstract |
0/24 each |
The abstract items are the cheapest of these and the most reliably forgotten. PRISMA 2020
devotes a separate 12-item instrument to the abstract — item 2 of the main checklist does
nothing but defer to it — so a manuscript can satisfy all 42 main-text items and still fail
most of the twelve. /check-reporting carries it as PRISMA_2020_Abstracts.md; run it as its
own pass and report its score separately, because folding twelve items into a 42-item total is
how they stay invisible.
Data availability statement: name what is being shared (extraction template,
locked dataset, analysis code, RoB judgments) and where — repository, DOI, or
supplementary file. "Available from the corresponding author on reasonable request"
satisfies few journals now and no longer satisfies item 27. If a Zenodo DOI is minted
post-acceptance, references/post_submission_release_ops.md covers propagating it
back into this statement.
Supplementary & analysis-code pre-submission gate (run before Phase 9 circulation and before portal upload). Presence of the 8-file package (Empirical Lesson 5) is necessary but not sufficient — each item must also be reviewer-ready:
- De-scaffold: strip internal-QC / tool artifacts before bundling — raw
/check-reporting output ("Assessed by: ", JSON blocks, "READY FOR SUBMISSION" verdicts, action-item lists), search-development planning docs (decision logs, expected-yield estimates, [Check on execution] placeholders, version-history dev notes), and stale version stamps. Ship a clean PRISMA 2020 checklist (27-item / 42-subitem table only) and an executed-method search-strategy doc, not the working drafts.
- Blind: supplementary goes to reviewers — remove author names/initials and sibling-project cross-references ("Designed by: ", "identical to a sibling review"). Same standard as the blinded manuscript.
- Cross-consistency with the manuscript: every supplementary number must match the main text — PRISMA counts, pool k/N, the Cochrane/CENTRAL search description, RoB counts. A supplement that says "Cochrane — NOT SEARCHED" while Methods report a confirmatory CENTRAL search is a contradiction reviewers catch.
- Submitted analysis code must reproduce and be self-contained: run it from a clean copy of the bundle. It must (a) read the bundled locked dataset (not an out-of-bundle path) and write to the working directory, and (b) regenerate every pool reported in the results table. A hard-coded study-id subset that drifts from the manuscript (e.g., a pool computed over k=7 while the manuscript reports k=9) is a P0 — fix and re-run; never ship stale code or stale figures derived from it.
- Run a supplementary-only review pass — the manuscript self-review/panel does not see the supplement; mirror
/self-review Phase 2.5c–2.5d (reference + cross-reference QC) over the supplementary files.
Phase 9: Co-author Circulation
Goal: Standardized pre-submission circulation of the manuscript to co-authors and
senior methodologist / reviewer, with a bounded review window and a controlled attachment
scope.
Trigger: Phase 8 is complete, and the draft has cleared Phase 6b source-fidelity
audit.
Summary: Reply to the prior-version email thread to preserve In-Reply-To continuity
(v1 → v2 → v3 tracked in one place). Attach the manuscript body with figures inline and,
for v≥2, a change summary — exclude graphical abstract, cover letter, COI forms, and
supplementary until the target journal is confirmed. TO = corresponding author + one
senior methodologist; CC = remaining co-authors. Set a 7-day deadline (5 business days +
weekend). Ask the corresponding author for target-journal preference, reviewer candidates,
and cover-letter framing.
Load-on-demand procedural detail (thread continuity, attachment scope rationale,
size-to-method table, journal-undetermined framing, response-tracking log):
${CLAUDE_SKILL_DIR}/references/phase9_circulation.md.
Failure-mode cross-ref → references/review_orchestration.md RO-1~RO-5 (dual-rating completeness, defensive-tone bias audit, response-matrix numeric tracking, 2nd-reviewer availability blocking).
Phase 10: Self-Audit Recovery (v{N} → v{N+1} sprint)
Goal: When an audit uncovers a structural data or protocol-application error,
withdraw the current version, rebuild, and re-circulate with a transparent audit trail.
Catching the error yourself before a journal reviewer does is the principal trust-building
move in this phase.
Trigger conditions (any one):
| # |
Trigger |
Source |
| T1 |
Extraction CSV ↔ primary source disagreement for a cell feeding a pooled/subgroup estimate or reported proportion |
Phase 6b audit |
| T2 |
Included/excluded study violates the pre-specified criteria on re-read |
Protocol review |
| T3 |
Hand-typed numerical literal in the analysis script traces to a wrong value |
Phase 6b audit |
| T4 |
PROSPERO protocol ↔ delivered analysis disagreement on outcome, subgroup, or eligibility |
Protocol ↔ analysis diff |
| T5 |
Dual-reviewer consensus record ↔ locked dataset disagreement on inclusion |
Consensus log diff |
Non-negotiable rule: if the trigger fires after Phase 9 circulation but before
journal submission, withdraw the current version within 24 hours. Reviewer discovery is
a strictly worse failure mode than self-withdrawal.
Sprint outline (12 steps): (10.1) audit log at qc/audit_vN_to_vNplus1.md →
(10.2) CSV re-verification with [VERIFY-CSV] tagging → (10.3) fresh script re-run
(fixed seed, logged) → (10.4) manuscript auto-sync (grep for v{N} residue) → (10.5)
supplementary regeneration (consensus log, RoB, GRADE/SoF, PRISMA flow) → (10.6) figure
regeneration via /make-figures → (10.7) change summary with delta table → (10.8)
PROSPERO amendment (application correction, not criteria change) → (10.9) re-circulation
in the Phase 9 thread with the "On re-review" framing → (10.10) anti-patterns to avoid
(hide-and-submit, "minor revision" reframe, cover-letter-only disclosure) → (10.11) post-
submission escalation path → (10.12) post-recovery loop (Phase 9 restart; tighten Phase
6b if a second sprint is needed).
Load-on-demand procedural detail (exact audit-log fields, delta-table template,
amendment language template, re-circulation paragraph template, anti-pattern rationale):
${CLAUDE_SKILL_DIR}/references/phase10_recovery.md.
Failure-mode cross-ref → references/post_submission_release_ops.md Gate 4 covers reject/revise Zenodo versioning, tag-cleanup gate, and re-target workflow (avoid "new version" misuse on re-target).
Failure Modes (prior MA projects, anonymized)
Failure patterns observed across three prior MA projects (anonymized). Each topical reference extends the phase it cross-references above — consult alongside phase procedural docs, not in isolation.
| Domain |
Phase span |
Load-on-demand reference |
| Data integrity (2x2 arm-swap, KM audit, methodology mismatch, PRISMA 5-way drift, single-source k) |
Phase 3 → 6 |
references/data_integrity_checklist.md (DI-1~DI-9) |
| Review orchestration (2nd-reviewer blocking, dual-rating completeness, defensive-tone audit, response-matrix tracking) |
Phase 9 circulation (extends phase9_circulation.md) |
references/review_orchestration.md (RO-1~RO-5) |
Submission package drift (multi-journal folder hygiene, DO_NOT_EDIT_HERE gate, build artifact vs master) |
Phase 8 → submission |
references/submission_package_drift.md |
| Post-submission release ops (Zenodo DOI timing, tag-cleanup gate, reject-retarget versioning) |
Submission → Phase 10 |
references/post_submission_release_ops.md |
Automation hooks (invoke at the phase listed)
| When |
Script |
Gate |
| Phase 3f reconciliation (before Phase 5 write-up) |
python3 ${CLAUDE_SKILL_DIR}/scripts/check_exclusion_code_validity.py --protocol 0_Protocol/protocol.md --screening 2_Screening/*.tsv --strict |
validates each applied exclusion code against the registered eligibility criteria: CODE_CONTRADICTS_ELIGIBILITY (a code excludes a design the protocol includes — the bulk study-loss defect no arithmetic/inter-rater gate can see), CODE_NOT_REGISTERED (off-protocol code), CODE_RENUMBERED (same code, two meanings). Challenge card: scripts/check_exclusion_code_validity_challenge/. |
| Phase 4 kickoff (before first extraction row) |
`python3 ${CLAUDE_SKILL_DIR}/../../scripts/extraction_consensus_log_init.py --output 2_Data/extraction_consensus_log |
|
…(truncated)
1---2name: meta-analysis3description: Systematic review and meta-analysis pipeline for medical research. Covers protocol registration (PROSPERO), search strategy, screening, data extraction, risk of bias assessment (QUADAS-2/ROBINS-I), statistical synthesis (bivariate/HSROC for DTA, random-effects for intervention), and PRISMA-compliant reporting. Supports both DTA and intervention meta-analyses.4triggers: meta-analysis, systematic review, PROSPERO, QUADAS-3, forest plot, funnel plot, PRISMA, QUADAS, ROBINS, HSROC, bivariate model, pooled sensitivity, pooled specificity, search strategy, study selection, data extraction form5tools: Read, Write, Edit, Bash, Grep, Glob6model: inherit7metadata:8 source: https://skillmd.com/skills/aperivue/meta-analysis9---1011# Meta-Analysis Skill1213You are helping a medical researcher conduct a systematic review and meta-analysis.14You support the full pipeline from protocol development to submission-ready manuscript,15with specialized support for diagnostic test accuracy (DTA) meta-analyses.1617## Communication Rules1819- Communicate with the user in their preferred language.20- All output documents, code, and checklists in English.21- Medical terminology always in English.2223## Reference Files2425### Built-in References (`${CLAUDE_SKILL_DIR}/references/`)2627- **PROSPERO template**: `${CLAUDE_SKILL_DIR}/references/PROSPERO_template.md` -- field-by-field guide with word limits, pitfalls checklist28- **ICMJE COI guide**: `${CLAUDE_SKILL_DIR}/references/icmje_coi_guide.md` -- batch generation, python-docx pitfalls, form structure29- **R templates**: `${CLAUDE_SKILL_DIR}/references/r_templates.md`30- **Checklists**: `${CLAUDE_SKILL_DIR}/references/checklists/`31 - `PRISMA_DTA.md` -- 27-item checklist32 - `QUADAS3.md` -- **current recommended DTA tool**: 6 phases, 4 domains, 20 signalling questions, assessed per accuracy estimate33 - `QUADAS2.md` -- the 2011 tool: 4 domains + 10 signalling questions (use when appraising or reproducing a review that used it)34 - `ROBINS_I.md` -- 7 domains + pre-assessment + synthesis recommendation35 - `RoB2.md` -- 5 domains + signalling questions + overall judgment36 - `PROBAST.md` -- 4 domains + AI extension + validation studies37 - `NOS.md` -- Cohort (8 items) + Case-control (8 items) + star interpretation38 - `JBI_Case_Series.md` -- 10-item critical appraisal checklist for case series39- **Phase 9 Co-author Circulation**: `${CLAUDE_SKILL_DIR}/references/phase9_circulation.md` -- thread continuity, attachment scope, recipient structure, 7-day window40- **Phase 10 Self-Audit Recovery**: `${CLAUDE_SKILL_DIR}/references/phase10_recovery.md` -- trigger conditions, 12-step rebuild sprint, PROSPERO amendment, re-circulation framing41- **Data integrity checklist**: `${CLAUDE_SKILL_DIR}/references/data_integrity_checklist.md` -- DI-1~DI-9 extraction/synthesis guardrails (prior anonymized MA projects)42- **Review orchestration**: `${CLAUDE_SKILL_DIR}/references/review_orchestration.md` -- RO-1~RO-5 circulation discipline (extends phase9_circulation.md)43- **Submission package drift**: `${CLAUDE_SKILL_DIR}/references/submission_package_drift.md` -- multi-journal folder hygiene, `DO_NOT_EDIT_HERE` gate, `_build.sh` pattern44- **Post-submission release ops**: `${CLAUDE_SKILL_DIR}/references/post_submission_release_ops.md` -- Zenodo DOI gating, tag-cleanup gates, reject-retarget versioning45- **Empirical peer-review lessons**: `${CLAUDE_SKILL_DIR}/references/empirical_lessons.md` -- 16 accumulated SR-MA peer-review / submission lessons (2026-05/06) that drive the Phase 4 extraction-form schema, Phase 4c QC, and Phase 8 submission gates. Load before designing the extraction form and before submission.4647### Built-in Templates (`${CLAUDE_SKILL_DIR}/templates/`)4849- **Extraction Form v2** (`templates/extraction_form_v2.md`) -- dual-extractor schema with `source_page_ref`, `source_verbatim_quote`, `cohort_source`, `overlap_flag_reviewer1/2`, `sample_n_dta_pool` vs `sample_n_prognostic_pool` columns. Required for SR-MA targeting high-impact radiology / medical AI journals.50- **Supplementary 8-file Checklist** (`templates/supplementary_8file_checklist.md`) -- S1-S8 mandatory package (PRISMA, PROSPERO, search strategy, exclusion list, extraction table, per-study x per-domain RoB, subgroup forests, sensitivity / publication bias) with a submission-gate bash check.5152### Built-in Scripts (`${CLAUDE_SKILL_DIR}/scripts/`)5354- **`screening_reconcile.py`** -- Phase 3f ID-set screening reconciliation.55- **`check_pool_consistency.py`** -- pool-composition / PRISMA count consistency.56- **`cohort_overlap_check.py`** -- shared-database cohort-overlap detection.57- **`extract_assist.py`** -- Phase 4 AI-assisted extraction *suggestions* (page ref + verbatim quote, `AI_SUGGESTED`/`needs_review`); human-confirm then `dta_extraction_qc.py`. Challenge card: `scripts/extract_assist_challenge/`.58- **`dta_extraction_qc.py`** -- 2x2 cell ↔ source sens/spec QC on the **confirmed** extraction CSV.5960---6162## Meta-Analysis Types6364| Type | RoB Tool | Statistical Model | Reporting Guideline |65|------|----------|-------------------|-------------------|66| **DTA** (diagnostic test accuracy) | **QUADAS-3** (QUADAS-2 for legacy reviews) | Bivariate / HSROC | PRISMA-DTA |67| **Intervention** (treatment effect) | RoB 2 (RCT) / ROBINS-I (NRSI) | Random-effects (DL/REML) | PRISMA 2020 |68| **Prognostic** (prediction model) | QUIPS / PROBAST | Random-effects | PRISMA 2020 |69| **Observational** (prevalence/association) | NOS / JBI | Random-effects | MOOSE |7071Auto-detect type from the research question or accept user specification.7273---7475## Workflow Phases7677### Phase 1: Protocol Development7879**Goal**: Produce a PROSPERO-ready protocol document.80811. **Structure the research question**:82 - DTA: PIRD (Population, Index test, Reference standard, Diagnosis)83 - Intervention: PICO (Population, Intervention, Comparator, Outcome)84852. **DTA only — do QUADAS-3 phases 1 and 2 now, not at risk-of-bias time**:86 QUADAS-3's first two phases are **review-level and belong in the protocol**:87 phase 1 states the **synthesis question(s)** (population, index test(s), target88 condition — a review may have more than one), and phase 2 defines the **ideal test89 accuracy trial** for each: objective, participants, index test(s), definition of the90 target condition, analysis. Every later risk-of-bias and applicability judgement is91 made against that trial.92 Write the review-specific guidance for answering each signalling question here too,93 with clinical **and** methodological input, and publish it as a web appendix.94 Defining the ideal trial after seeing the studies is not an assessment — it is a95 judgement fitted to the results. See `references/checklists/QUADAS3.md`.96973. **Define eligibility criteria**:98 - Study design (cross-sectional DTA, cohort, RCT, etc.)99 - Population characteristics100 - Index test / intervention specifics101 - Comparator / reference standard102 - Outcome measures (Se/Sp for DTA; effect size for intervention)103 - Exclusion criteria with justification1041054. **Plan the search**:106 - Minimum 3 databases: PubMed, Embase, and Cochrane CENTRAL (add Scopus, Web of Science as needed)107 - Draft Boolean search strategy using PIRD/PICO components108 - Grey literature plan (conference abstracts, trial registries)109 - Language restrictions (state explicitly)110 - Date range with justification1111125. **Plan RoB assessment**:113 - Select tool based on type (see table above)114 - State number of independent assessors (minimum 2)115 - Plan for disagreement resolution (consensus, third reviewer)1161176. **Plan synthesis**:118 - DTA: bivariate random-effects model (Reitsma) or HSROC (Rutter & Gatsonis)119 - Intervention: random-effects (DerSimonian-Laird or REML)120 - Heterogeneity assessment plan121 - Subgroup / sensitivity analysis plan122 - Publication bias assessment plan1231247. **Generate PROSPERO registration document**:125 - Read `${CLAUDE_SKILL_DIR}/references/PROSPERO_template.md` for field-by-field guidance126 - Generate all fields with word counts (stay within limits per field)127 - Structure: title, review question, PICO, searches, data collection, outcomes, synthesis, subgroups, stage, affiliation128 - **Registration-ID format gate.** A PROSPERO ID is `CRD42` + 9 digits (14 characters total), e.g. `CRD42024500001`. Validate any ID that appears in the manuscript or registration doc with `grep -oE 'CRD42[0-9]+'` and assert a 14-character length / `^CRD42\d{9}$` — a 15-character ID (a stray digit) is a transcription error a reviewer will check against the live record.129 - **Review-type selection.** Pick the *least-wrong* portal review type for the actual design and state any portal constraint in the protocol. A descriptive single-arm proportion synthesis is not an "Intervention review"; choosing "Intervention review" only to satisfy a portal field contradicts a later GRADE / effect-certainty statement. Whatever certainty language the protocol commits to (GRADE vs "evidence statements only") must match the manuscript verbatim — a guideline-style "we recommend" is not licensed by a descriptive review type.130 - For mixed designs (comparative + single-arm): explicitly address comparator for both arms131 - For RoB: map tool to study design (NOS for comparative, JBI for case series → select "Other" in form)132 - Output: Markdown + DOCX (via pandoc) for copy-paste into PROSPERO web form133 - Append Common Pitfalls Checklist (HTML entities, word limits, stage constraint)134 - Save to project `7_Submission/` or equivalent directory135136### Phase 2: Search Strategy137138**Goal**: Develop and validate reproducible search strategies.1391401. **Build search blocks** from PIRD/PICO:141 - Population block (MeSH + free text)142 - Index test / Intervention block143 - Comparator / Reference standard block (optional)144 - Study design filter (if applicable)1451462. **Combine with Boolean operators**:147 - Within blocks: OR148 - Between blocks: AND1491503. **Execute search per database** using `/search-lit`:151 - PubMed: MeSH + free text152 - Embase: Emtree + free text153 - Additional databases as specified in protocol1541554. **Report search per PRISMA-S** (Rethlefsen et al. 2021, PMID:33499930):156 Save search strategies as a structured document, one section per database,157 with date of search, number of results, and any limits applied.1581595. **Merge and deduplicate**: Combine all database results into a single spreadsheet.160 Deduplicate by DOI first, then PMID. Save raw counts for PRISMA flow.161162### Phase 3: Screening & Selection163164**Goal**: Systematic title/abstract and full-text screening with two independent reviewers.165166**3a. Round 1 — initial title/abstract screening (single reviewer).** Define the exclusion codes167from the protocol (E1=Not target population, E2=Not intervention, E3=Ineligible type, E4=Non-human,168E5=Duplicate). Mark every record INCLUDE / EXCLUDE / MAYBE with a reason code → `round1_{date}.tsv`.169170**3b. Round 2 — dual independent title/abstract screening.** A second independent reviewer (or AI171as a *documented* second-pass tool with human verification) re-screens all R1 records. Compute172Cohen's κ and report it in Methods. `round2_tag` = INCLUDE / EXCLUDE / MAYBE, where MAYBE means173disagreement **or** either reviewer flagged uncertainty → `round2_tag`, `round2_reason` columns.174175**3c. Round 3 — adjudication of disagreements (first reviewer).** Build the R3 sheet with all MAYBE176records first, then INCLUDE records for a brief confirmation pass. The first reviewer independently177adjudicates each row (`round3_decision`, plus `round3_reason` only when overturning R2). Optional178AI-assisted pre-screening can compress the effort — but **AI suggestions are not decisions**: the179reviewer independently confirms or overturns every one. Template, sort priority, and the required180Methods boilerplate are in the reference file.181182**3d. Round 4 — full-text screening.** Retrieve full texts for `round3_decision = INCLUDE` (use183`/fulltext-retrieval`), apply the full-text exclusion codes (F1=No extractable outcome, F2=No184comparative data, F3=Cannot separate target population, F4=Inadequate sample/follow-up,185F5=Full-text unavailable), with two independent reviewers, Cohen's κ, and consensus or a third186reviewer for disagreements. Flag comparative studies for priority extraction.187188**3e. PRISMA flow.** Track counts at every stage (R1 → R2 → R3 → R4 → final included); generate the189diagram with `/make-figures` once the numbers are final.190191**3f. Post-consensus count reconciliation gate (MANDATORY before Phase 5 write-up).** Reconcile the192counts from the **raw ID sets, never from prose summaries**, and record the canonical totals in one193source-of-truth file:194195```bash196python "${CLAUDE_SKILL_DIR}/scripts/screening_reconcile.py" \197 --screening 2_Screening/fulltext_screening.tsv \198 --consensus 2_Screening/consensus_decisions.tsv \199 --table1 6_Tables/table1_studies.csv \200 --output 2_Screening/screening_consensus.json201```202203Downstream stages consume `screening_consensus.json` for counts and ID sets; the Markdown consensus204document remains the human explanation. Three hard rules:2052061. **List the narrative-only IDs explicitly.** The highest-yield red flag is a numeric claim ("10207 narrative-only studies") that does not match the enumerable set `(A ∪ C) \ B \ T`.2082. **No "N → M" transition without ID receipts.** "k rose from 30 to 32 after FLAG consensus" must209 cite the added/removed IDs. A transition claim with no enumerable ID set is a **P0** and blocks210 the Phase 5 hand-off.2113. **`STAGE_TRANSFER_LOSS` is a P0.** Exit 1 when a record is included at screening but **absent212 from the consensus artifact altogether** — no adjudication was ever recorded. An exclusion is a213 decision; silence is a gap. Never let it settle into narrative-only (why: reference file).214215The set algebra, the reconciliation-table template, and the failure pattern it exists for (a216manuscript ships counts the ID sets do not support, with every downstream artifact echoing the same217unreconciled prose total) are in the reference file.218219**3f.5 Pool composition lock (MANDATORY at adjudication freeze).** Once 3f passes, freeze the pool220into a single source-of-truth YAML that every downstream artifact can be checked against:221222```bash223cp "${CLAUDE_SKILL_DIR}/templates/FINAL_POOL_LOCK.yaml.template" 2_Data/FINAL_POOL_LOCK.yaml224# fill counts + UID lists from 3f, compute the SHA-256 over the sorted UID list,225# and COMMIT THE LOCK before any Phase 4 extraction226```227228- **Never re-derive `k included` from the extraction TSV at manuscript build time** — always229 reference `final_pool_n` from the lock.230- **Aggregate patient/lesion totals are locked too**, not just study counts. Distinguish231 **arm-separable** from **both-arm** rows: a study contributing one arm must not have its232 full-cohort count folded into a pooled total. A hand-carried headline total that does not233 re-derive from the locked per-study values is a **P0**.234- A late post-freeze change to the pool is a **formal PROSPERO amendment**: file it, re-freeze as235 `FINAL_POOL_LOCK_v2.yaml`, and propagate to every artifact.236237**Read on demand:**238239| File | Read it when | Cost if read blindly |240|---|---|---|241| `references/phase3_screening_detail.md` | you are executing a screening round, using AI pre-screening, or a reconciliation/lock gate fired | ~3,600 tokens; the round procedures are needed one round at a time, not all at invocation |242### Phase 4: Data Extraction243244**Goal**: Create standardized extraction forms and extract 2x2 or effect-size data.245246**4.0 Entry gate (MANDATORY) — pool composition lock ↔ adjudication TSV.** Before any extraction247work begins, confirm the round-3 adjudication TSV and `FINAL_POOL_LOCK.yaml` (Phase 3f.5) agree on248which UIDs are included:249250```bash251python "${CLAUDE_SKILL_DIR}/scripts/check_pool_consistency.py" \252 --lock 2_Data/FINAL_POOL_LOCK.yaml \253 --adjudication-tsv 2_Screening/round3_adjudication.tsv \254 --decision-col round3_decision --uid-col uid \255 --include-labels "INCLUDE,INCLUDE_MIXED" \256 --out qc/pool_consistency.json257```258259**The gate fails closed: any UID disagreement blocks extraction.** Resolve by re-freezing the lock260with the corrected UID set (and propagating downstream) or by correcting a mis-labelled TSV row. Do261NOT proceed with a mismatch — the extraction matrix will not align with the locked pool, and the262drift surfaces as a fabrication-grade red flag at peer review.263264> **Failure-mode cross-ref** → `references/data_integrity_checklist.md` DI-1~DI-5 are mandatory265> during extraction (2x2 arm-swap, KM audit trail, methodology mismatch, PRISMA 5-way drift,266> single-source k).267268**Extraction form.** For an SR-MA targeting high-impact radiology / medical AI journals use269`${CLAUDE_SKILL_DIR}/templates/extraction_form_v2.md` — its dual-extractor, source-page-reference,270and verbatim-quote columns are what close the 2x2 cell-swap and cohort-overlap blind spots. The271DTA and intervention field lists are in the reference file.272273**AI-drafted starting document — treat as hallucination-suspect.** If a mentor or collaborator274shared an AI-drafted study list, 2x2 set, or effect estimates (*even* flagged "for reference275only"): save it with a `_DO_NOT_USE_VERBATIM` suffix and re-verify **every** N, denominator, event276count, OR/CI, and author/year against the source PDF. Trust hierarchy: **source PDF + own analysis277stdout > the mentor's direct text > the attached AI draft** — never promote a draft up that ladder.278Procedure and precedent: reference file.279280**4b. Special cases (KM reconstruction, composite exposure).** When studies report outcomes only as281Kaplan-Meier curves, or the intervention is a composite of techniques, load282`${CLAUDE_SKILL_DIR}/references/phase4_km_composite.md` for the WebPlotDigitizer → `IPDfromKM`283procedure (cite Guyot et al. 2012, doi:10.1186/1471-2288-12-9) and the 4-path composite-exposure284decision tree. Pre-specify a sensitivity analysis excluding composite-exposure studies.285286**Cross-verification (≥2 independent reviewers).** Report inter-reviewer agreement (% or Cohen's287κ) at title/abstract and full-text stages. Verify denominator consistency — **the denominator may288differ across outcomes within one study**, so for each outcome back-calculate `event ÷ denominator`289and confirm it reproduces the paper's reported percentage. Distinguish KM-curve estimates from raw290event counts and record the data source (Table / KM / text). Log every consensus decision in291`{project}/consensus_log.md`, then **lock the dataset**; later changes need a dated justification.292293**4c. Extraction QC & cohort overlap.** After dual-extractor consensus, run both before locking:294295```bash296# 2x2 cell integrity: validates TP/FN/TN/FP against source-reported sens/spec (catches arm-swap)297python3 "${CLAUDE_SKILL_DIR}/scripts/dta_extraction_qc.py" \298 --input 2_Extraction/extraction.csv --tolerance 0.02 \299 --out 2_Extraction/qc/dta_extraction_qc.tsv300301# cohort overlap: shared public DB / same institution+period / same first author ±2y302python3 "${CLAUDE_SKILL_DIR}/scripts/cohort_overlap_check.py" \303 --input 2_Extraction/studies.csv --enrich \304 --out 2_Extraction/qc/cohort_overlap.md305```306307Any `FLAG_SWAP` / `FLAG_MISMATCH` requires third-reviewer adjudication before Phase 6. **A308confirmed flag is not resolved until the extraction form itself is edited** — a flag corrected only309in a review note silently re-enters synthesis, so re-run the QC and confirm zero open flags before310locking. HIGH-confidence overlap pairs require a Limitations acknowledgment plus a sensitivity311analysis excluding one of the pair. Cross-links: `/peer-review` Phase 2A P1 + P2.312313**Read on demand:**314315| File | Read it when | Cost if read blindly |316|---|---|---|317| `references/phase4_extraction_detail.md` | building the extraction form, an AI draft was shared, you want the optional `extract_assist.py` scaffolding, or a QC flag fired | ~4,700 tokens; a clean dual-extraction with no AI draft needs none of it |318| `references/phase4_km_composite.md` | studies report only KM curves, or the exposure is composite | ~2,200 tokens |319### Phase 5: Risk of Bias Assessment320321**Goal**: Guide structured RoB assessment with the appropriate tool.322323**DTA**: this phase runs QUADAS-3 **phases 3–6** (flow diagram, identify the estimates to324assess, assess, overall judgement). Phases 1–2 — the synthesis question and the ideal test325accuracy trial — were written in Phase 1 above. If they were not, stop and write them before326judging anything; they are the comparator every judgement is made against.327328Select tool based on meta-analysis type (see table above), then read the corresponding checklist:329330| Tool | Checklist File |331|------|---------------|332| QUADAS-3 (DTA, current) | `${CLAUDE_SKILL_DIR}/references/checklists/QUADAS3.md` |333| QUADAS-2 (DTA, legacy) | `${CLAUDE_SKILL_DIR}/references/checklists/QUADAS2.md` |334| RoB 2 (RCT) | `${CLAUDE_SKILL_DIR}/references/checklists/RoB2.md` |335| ROBINS-I (NRSI) | `${CLAUDE_SKILL_DIR}/references/checklists/ROBINS_I.md` |336| PROBAST (Prediction) | `${CLAUDE_SKILL_DIR}/references/checklists/PROBAST.md` |337| NOS (Observational) | `${CLAUDE_SKILL_DIR}/references/checklists/NOS.md` |338| JBI (Case Series) | `${CLAUDE_SKILL_DIR}/references/checklists/JBI_Case_Series.md` |339340For AI/ML prediction models, also apply PROBAST+AI extensions.341342**Output**: Summary table + traffic light plot (use `/make-figures`).343344### Phase 6: Statistical Synthesis345346**Goal**: Execute meta-analysis and generate publication-ready outputs.347348> **Failure-mode cross-ref** → `references/data_integrity_checklist.md` DI-6/DI-7/DI-9 are the consistency gate (CSV ↔ script ↔ prose; single-source k; 3-way numeric reconciliation before Stage 4).349350**IMPORTANT**: Always use R for meta-analysis (packages: `meta`, `metafor`, `mada`).351See `${CLAUDE_SKILL_DIR}/references/r_templates.md` for full code templates.352353| Analysis family | Primary tool | Key output |354|-----------------|-------------|-----------|355| DTA | `mada::reitsma()` (bivariate) | Pooled Se/Sp + SROC with confidence/prediction regions |356| Intervention | `meta::metagen()` / `meta::metabin()` | Pooled OR/RR, I², Egger's test, leave-one-out |357| Dual (comparative + single-arm) | `metabin` + `metaprop` | PRIMARY vs SECONDARY per pre-specified protocol |358359**Load-on-demand**: Read `${CLAUDE_SKILL_DIR}/references/phase6_statistical_synthesis.md`360for the full R code templates, the dual-approach decision table (comparative vs361single-arm), practical cautions (method.tau, HK CI, zero-cell correction),362publication-bias test power, sensitivity-analysis menu, and error-handling rules.363364**Three checks before the pool is written up** — each is a Methods sentence, not only a365setting. R and detail in the same reference:3663671. **Is the event rare?** A pooled event rate < 1%, or any zero-event arm, moves the368 analysis off the inverse-variance default onto Peto / Mantel-Haenszel without a369 zero-cell correction / GLMM. Inverse-variance methods including DerSimonian-Laird are370 to be *avoided* for rare events, and so are 0.5 continuity corrections with them.3712. **Why this model?** Fixed vs random is a judgment about whether one common true effect372 exists — never derived from Cochran's Q or I². "A random-effects model was used373 because I² was 65%" is a reviewer catch, not a rationale.3743. **Does one study contribute several correlated effect sizes?** Multiple outcomes,375 readers, thresholds, or time points from the same participants need one pre-specified376 estimate per study, a multivariate model, or robust variance estimation — not377 independent pooling.378379### Phase 6b: Post-Analysis Source Fidelity Audit (MANDATORY)380381**Goal**: Catch numerical hallucinations that survived the forward pipeline (CSV → .R → manuscript).382383**The failure pattern** — treat this as a lived near-miss, not hypothetical:384> A safety outcome is reported with its arm-level events, and therefore its p-value,385> direction-reversed relative to what the primary-source Table actually recorded.386> The extraction CSV is correct; the R script's Fisher exact387> `matrix()` was hand-typed after a column in the source Table was misread. Internal388> consistency checks passed because every downstream artifact (Abstract, Discussion,389> Table, forest caption) echoed the same wrong number. The reversal was caught only on390> a second-pass audit with random extraction sampling against the primary paper.391392**Non-negotiable rules:**3933941. **No hand-typed numerical matrices when a CSV exists.**395 - Use `read.csv(...)` + subset / filter. Never copy a 2x2 table from a paper's Table into396 `matrix(c(...), ...)` by eye.397 - If hand entry is truly unavoidable (e.g., text-only extraction), the `matrix`, `c()`, or398 `data.frame` line MUST carry a comment citing the exact CSV row + column OR the exact399 primary-source Table/Page coordinate. Example:400 ```r401 # source: data_extraction_final.csv row <N> (<first-author> <year>), cols <event_arm1>=0, <event_arm2>=1402 # verified against primary source Table <X>, page <P>403 fisher.test(matrix(c(0, 45, 1, 55), nrow = 2, byrow = FALSE))404 ```4054062. **Comparative-arm subsets are a separate consensus-log row.**407 - When one study's arm-specific values (e.g., one arm of a multi-arm study) are used in a408 comparative analysis while the full cohort of that study appears elsewhere,409 `extraction_consensus_log.md` must carry an explicit row for the arm-specific values.410 Pooled totals and arm-specific values MUST NOT share a row.4114123. **Random 3-claim back-check before closing Phase 6.**413 - After the forest/funnel/subgroup outputs stabilize, randomly sample 3 numerical claims414 from the Results section of the draft manuscript and trace each back to (a) the R output415 log and (b) the original paper's Table/Figure.416 - Record the back-check as a small table in `peer_review_<vN>_internal.md`:417418 | Claim (manuscript line) | R output file:line | Primary source (paper, Table/Fig, page) | Match? |419 |---|---|---|---|420421 - A single mismatch is a P0 blocker — do not advance to Phase 7 until resolved.4224234. **Revision-introduced numbers must be tagged.**424 - Any new number added after v1 — including numbers produced by a new comparative / subgroup /425 sensitivity script — MUST be wrapped inline as `[VERIFY-CSV]` in the manuscript until the426 Phase 2.5a audit in `/self-review` clears it.4274285. **Sensitivity analyses must be recomputed on the modified data, not copied.**429 - When you add a sensitivity / leave-one-out / erosion / alternative-model analysis, every430 reported effect size (Cohen's dz/f, AUC, OR, HR, β, sens/spec, ICC) MUST be re-derived from431 the modified dataset. If a sensitivity-table effect size is **identical to the primary432 analysis to two decimals across ≥4 values**, the recomputation almost certainly did not run433 and the primary values were transcribed — re-run the script on the modified data.434 - The underlying means/SDs/counts will change even when the effect size looks similar; if the435 effect sizes are byte-identical while the inputs differ, that is the tell. Probability of ≥4436 independent values coinciding to 2 decimals by chance is ≈ (0.01)^4 — essentially zero.437 - The failure it catches: a sensitivity analysis reports a block of effect-size values438 byte-identical to the primary tables while the underlying means/SDs differ — the439 sensitivity analysis was never actually recomputed. Internal consistency cannot see it.4404416. **A "fixed" / "resolved" audit note requires re-run evidence, not a claim.**442 - When a prior audit note records a number as `fixed`, `resolved`, or `corrected`, that status is443 only valid if it carries the re-run evidence: a timestamp and the relevant stdout / output-file444 line showing the corrected value, or the commit that changed it. A bare "fixed in v10" with no445 re-run artifact does NOT clear the finding — re-run the script and attach the output.446 - The forward pipeline can echo a stale value through every artifact while an audit note claims it447 was fixed (e.g., a major-comparison N still reading the old total after a "fixed" note). The448 outcome-denominator cross-check (`/self-review` Phase 2.5b, the cohort-arithmetic / pool-lock449 assertions) must pass against the *current* outputs before any "fixed" status is accepted.450451**When this phase triggers:** every time Phase 6 outputs change (first draft, revision, reviewer-452requested re-analysis). Not optional on "minor" re-runs — the precedent reversal above453occurred inside a "minor" revision-era re-analysis.454455### Phase 7: GRADE / Certainty of Evidence456457**Goal**: Assess certainty of the body of evidence.458459For DTA meta-analysis, apply GRADE-DTA framework:4601. Risk of bias (from QUADAS-3, or QUADAS-2 for a legacy review)4612. Indirectness (applicability concerns)4623. Inconsistency (heterogeneity)4634. Imprecision (wide CIs, small sample)4645. Publication bias465466For intervention meta-analysis, apply standard GRADE.467468**Certainty is assessed per outcome, not once for the review.** The five domains resolve469differently for each outcome — an outcome pooled from 12 studies with narrow CIs and one470pooled from 3 with a wide CI do not share a rating, and a single review-level "moderate471certainty" sentence tells a reader nothing about the outcome they came for. Rate every472outcome carried into the Summary of Findings table, and state the reason for each473downgrade (which domain, why) rather than the resulting label alone.474475Output: Summary of Findings table — one row per outcome, carrying the pooled estimate476with its precision alongside the certainty rating (high / moderate / low / very low).477478### Phase 8: Reporting & Manuscript479480**Goal**: Generate PRISMA-compliant manuscript sections.481482> **Failure-mode cross-ref** → `references/submission_package_drift.md` — apply the `_build.sh` pattern + `DO_NOT_EDIT_HERE` gate when staging multi-journal submission folders.4834841. **Check reporting compliance**: Use `/check-reporting` with PRISMA-DTA or PRISMA 2020, then485 run it a second time over the **abstract** with PRISMA 2020 for Abstracts — 12 items, its own486 denominator. One run does not cover both.4872. **Write manuscript**: Use `/write-paper` with meta-analysis type selected4883. **Figures**: Use `/make-figures` for:489 - PRISMA flow diagram490 - Forest plots (paired for DTA)491 - SROC curve (DTA)492 - Funnel plot493 - RoB summary (traffic light plot)4944. **Tables**:495 - Characteristics of included studies496 - 2x2 data per study (DTA)497 - RoB assessment results498 - Summary of findings / GRADE table (one row per outcome — Phase 7)4995005. **The items published radiology SR/MA most often drop.** Park 2022 (Korean J Radiol;501 PMID:35213097) scored 24 SR/MAs against PRISMA 2020 and found 24 of 42 items reported502 by fewer than 80%. The checklist itself lives in `/check-reporting`; what follows is503 where drafts actually fail, so check these by hand before the compliance run rather504 than after it:505506 | PRISMA item | What is missing | Observed |507 |---|---|---|508 | **20a** | For **each** synthesis, a brief summary of the contributing studies' characteristics and risk of bias — not one global paragraph covering all pools | 0/24 |509 | **27** | Data availability: which of the extraction forms, extracted data, analysis dataset, and analytic code are public, and where | 0/24 |510 | **24a–c** | Registration number, where the protocol can be read, and any amendment — an explicit "not registered" satisfies 24a | 0/24 |511 | **22 / 15** | Certainty of evidence per outcome, and the method used to assess it | 9% |512 | **13f / 20d** | Sensitivity analysis: method and result | 28% |513 | **18** | Risk of bias **per study**, shown study-by-study rather than as a pooled proportion | 32% |514 | **13d** | Rationale for the synthesis model (see Phase 6 check 2) | 35% |515 | **16b** | Studies that look eligible but were excluded, cited individually with the reason | 25% |516 | Abstract **#3, #12** | Eligibility criteria and registration inside the structured abstract | 0/24 each |517518 The abstract items are the cheapest of these and the most reliably forgotten. PRISMA 2020519 devotes a **separate 12-item instrument** to the abstract — item 2 of the main checklist does520 nothing but defer to it — so a manuscript can satisfy all 42 main-text items and still fail521 most of the twelve. `/check-reporting` carries it as `PRISMA_2020_Abstracts.md`; run it as its522 own pass and report its score separately, because folding twelve items into a 42-item total is523 how they stay invisible.5245256. **Data availability statement**: name what is being shared (extraction template,526 locked dataset, analysis code, RoB judgments) and where — repository, DOI, or527 supplementary file. "Available from the corresponding author on reasonable request"528 satisfies few journals now and no longer satisfies item 27. If a Zenodo DOI is minted529 post-acceptance, `references/post_submission_release_ops.md` covers propagating it530 back into this statement.5315327. **Supplementary & analysis-code pre-submission gate** (run before Phase 9 circulation and before portal upload). Presence of the 8-file package (Empirical Lesson 5) is necessary but not sufficient — each item must also be reviewer-ready:533 - **De-scaffold**: strip internal-QC / tool artifacts before bundling — raw `/check-reporting` output ("Assessed by: <tool>", JSON blocks, "READY FOR SUBMISSION" verdicts, action-item lists), search-development planning docs (decision logs, expected-yield estimates, `[Check on execution]` placeholders, version-history dev notes), and stale version stamps. Ship a clean PRISMA 2020 checklist (27-item / 42-subitem table only) and an executed-method search-strategy doc, not the working drafts.534 - **Blind**: supplementary goes to reviewers — remove author names/initials and sibling-project cross-references ("Designed by: <name>", "identical to a sibling review"). Same standard as the blinded manuscript.535 - **Cross-consistency with the manuscript**: every supplementary number must match the main text — PRISMA counts, pool k/N, the Cochrane/CENTRAL search description, RoB counts. A supplement that says "Cochrane — NOT SEARCHED" while Methods report a confirmatory CENTRAL search is a contradiction reviewers catch.536 - **Submitted analysis code must reproduce and be self-contained**: run it from a clean copy of the bundle. It must (a) read the bundled locked dataset (not an out-of-bundle path) and write to the working directory, and (b) regenerate every pool reported in the results table. A hard-coded study-id subset that drifts from the manuscript (e.g., a pool computed over k=7 while the manuscript reports k=9) is a P0 — fix and re-run; never ship stale code or stale figures derived from it.537 - **Run a supplementary-only review pass** — the manuscript self-review/panel does not see the supplement; mirror `/self-review` Phase 2.5c–2.5d (reference + cross-reference QC) over the supplementary files.538539---540541### Phase 9: Co-author Circulation542543**Goal**: Standardized pre-submission circulation of the manuscript to co-authors and544senior methodologist / reviewer, with a bounded review window and a controlled attachment545scope.546547**Trigger**: Phase 8 is complete, and the draft has cleared Phase 6b source-fidelity548audit.549550**Summary**: Reply to the prior-version email thread to preserve `In-Reply-To` continuity551(v1 → v2 → v3 tracked in one place). Attach the manuscript body with figures inline and,552for v≥2, a change summary — exclude graphical abstract, cover letter, COI forms, and553supplementary until the target journal is confirmed. TO = corresponding author + one554senior methodologist; CC = remaining co-authors. Set a 7-day deadline (5 business days +555weekend). Ask the corresponding author for target-journal preference, reviewer candidates,556and cover-letter framing.557558**Load-on-demand procedural detail** (thread continuity, attachment scope rationale,559size-to-method table, journal-undetermined framing, response-tracking log):560`${CLAUDE_SKILL_DIR}/references/phase9_circulation.md`.561562> **Failure-mode cross-ref** → `references/review_orchestration.md` RO-1~RO-5 (dual-rating completeness, defensive-tone bias audit, response-matrix numeric tracking, 2nd-reviewer availability blocking).563564---565566### Phase 10: Self-Audit Recovery (v{N} → v{N+1} sprint)567568**Goal**: When an audit uncovers a structural data or protocol-application error,569withdraw the current version, rebuild, and re-circulate with a transparent audit trail.570Catching the error yourself before a journal reviewer does is the principal trust-building571move in this phase.572573**Trigger conditions (any one):**574575| # | Trigger | Source |576|---|---------|--------|577| T1 | Extraction CSV ↔ primary source disagreement for a cell feeding a pooled/subgroup estimate or reported proportion | Phase 6b audit |578| T2 | Included/excluded study violates the pre-specified criteria on re-read | Protocol review |579| T3 | Hand-typed numerical literal in the analysis script traces to a wrong value | Phase 6b audit |580| T4 | PROSPERO protocol ↔ delivered analysis disagreement on outcome, subgroup, or eligibility | Protocol ↔ analysis diff |581| T5 | Dual-reviewer consensus record ↔ locked dataset disagreement on inclusion | Consensus log diff |582583**Non-negotiable rule**: if the trigger fires after Phase 9 circulation but before584journal submission, withdraw the current version within 24 hours. Reviewer discovery is585a strictly worse failure mode than self-withdrawal.586587**Sprint outline (12 steps)**: (10.1) audit log at `qc/audit_vN_to_vNplus1.md` →588(10.2) CSV re-verification with `[VERIFY-CSV]` tagging → (10.3) fresh script re-run589(fixed seed, logged) → (10.4) manuscript auto-sync (grep for v{N} residue) → (10.5)590supplementary regeneration (consensus log, RoB, GRADE/SoF, PRISMA flow) → (10.6) figure591regeneration via `/make-figures` → (10.7) change summary with delta table → (10.8)592PROSPERO amendment (application correction, not criteria change) → (10.9) re-circulation593in the Phase 9 thread with the "On re-review" framing → (10.10) anti-patterns to avoid594(hide-and-submit, "minor revision" reframe, cover-letter-only disclosure) → (10.11) post-595submission escalation path → (10.12) post-recovery loop (Phase 9 restart; tighten Phase5966b if a second sprint is needed).597598**Load-on-demand procedural detail** (exact audit-log fields, delta-table template,599amendment language template, re-circulation paragraph template, anti-pattern rationale):600`${CLAUDE_SKILL_DIR}/references/phase10_recovery.md`.601602> **Failure-mode cross-ref** → `references/post_submission_release_ops.md` Gate 4 covers reject/revise Zenodo versioning, tag-cleanup gate, and re-target workflow (avoid "new version" misuse on re-target).603604---605606## Failure Modes (prior MA projects, anonymized)607608Failure patterns observed across three prior MA projects (anonymized). Each topical reference extends the phase it cross-references above — consult alongside phase procedural docs, not in isolation.609610| Domain | Phase span | Load-on-demand reference |611|---|---|---|612| Data integrity (2x2 arm-swap, KM audit, methodology mismatch, PRISMA 5-way drift, single-source k) | Phase 3 → 6 | `references/data_integrity_checklist.md` (DI-1~DI-9) |613| Review orchestration (2nd-reviewer blocking, dual-rating completeness, defensive-tone audit, response-matrix tracking) | Phase 9 circulation (extends `phase9_circulation.md`) | `references/review_orchestration.md` (RO-1~RO-5) |614| Submission package drift (multi-journal folder hygiene, `DO_NOT_EDIT_HERE` gate, build artifact vs master) | Phase 8 → submission | `references/submission_package_drift.md` |615| Post-submission release ops (Zenodo DOI timing, tag-cleanup gate, reject-retarget versioning) | Submission → Phase 10 | `references/post_submission_release_ops.md` |616617### Automation hooks (invoke at the phase listed)618619| When | Script | Gate |620|---|---|---|621| Phase 3f reconciliation (before Phase 5 write-up) | `python3 ${CLAUDE_SKILL_DIR}/scripts/check_exclusion_code_validity.py --protocol 0_Protocol/protocol.md --screening 2_Screening/*.tsv --strict` | validates each applied exclusion code against the *registered* eligibility criteria: `CODE_CONTRADICTS_ELIGIBILITY` (a code excludes a design the protocol includes — the bulk study-loss defect no arithmetic/inter-rater gate can see), `CODE_NOT_REGISTERED` (off-protocol code), `CODE_RENUMBERED` (same code, two meanings). Challenge card: `scripts/check_exclusion_code_validity_challenge/`. |622| Phase 4 kickoff (before first extraction row) | `python3 ${CLAUDE_SKILL_DIR}/../../scripts/extraction_consensus_log_init.py --output 2_Data/extraction_consensus_log623624…(truncated)