Self-Review Skill
You are helping a medical researcher check their own manuscript before journal submission. The goal is to anticipate reviewer comments by applying the same critical lens used in peer review across medical journals.
This is NOT about writing a review. It's about producing an actionable list of anticipated reviewer comments with specific fix suggestions, so the manuscript can be strengthened before reviewers ever see it.
Optional Flags
--fix: After generating the review report, automatically apply fixes for all issues wherefixable_by_aiis true. Edits the manuscript in place, then reports a diff summary. Does NOT fix issues markedfixable_by_ai: false(e.g., missing data, design flaws). Maximum 2 fix-and-re-review iterations.--json: Output the structured JSON block (see Phase 3c below) in addition to the markdown report. Default when called from/write-paperPhase 7.--panel: Run the multi-agent panel review (Phase 2.6) — several domain-expert reviewers in parallel plus an editor synthesis — instead of the single-pass review. Opt-in and off by default (a panel spawns N reviewer agents + 1 editor, so it costs several times more tokens). Reserve it for a high-stakes pre-submission final pass on a top-tier target. Do not combine with--fix: a panel diagnoses and prioritizes; run--fixas a separate follow-up pass once the author has triaged the panel's findings.
Severity Framing
When flagging issues, classify severity:
- Fatal: Fundamental design flaw that cannot be fixed with existing data (e.g., data leakage that invalidates all results, absence of any reference standard, label-feature circularity). The manuscript likely needs redesign. Submission would likely result in Reject.
- Fixable: Significant but addressable with existing data (e.g., missing calibration analysis, unclear exclusion criteria, absent CIs, incomplete reporting). These are the most actionable findings.
Most issues are Fixable. Reserve Fatal for true design-level problems.
Two Objectives: the Floor and the Ceiling
A submission-ready manuscript optimizes two things at once, and most of this skill (and the gate stack behind it) only optimizes the first:
- Floor — minimize rejection-for-cause. Fabricated citations, numbers that do not reconcile, overclaims, missing checklist items, leakage. Categories A–K and the deterministic gates (Phases 2.5–2.5f) do this, and they are right to. Many of them raise the floor by adding material: a hedge, a caveat, a disclosure, an audit trail, a checklist row.
- Ceiling — maximize editorial-championing. Will a handling editor read a confident narrative (problem → design → result → meaning) and want to send it out, or a defensive audit and bounce it? Nothing in the floor stack pushes here, and several floor gates push the other way. Iterated, a manuscript over-hardens: every individual gate finding is correct, yet the accumulated product reads as a rebuttal letter — over-hedged, audit-trail-heavy, Abstract buried under caveats, the strongest sensitivity result hidden in Limitations, too long.
These objectives can conflict, so the order matters: the floor gates run first and secure accuracy; then the ceiling pass (category L / Phase 2.5g) reads the accurate manuscript as a whole and recommends SUBTRACTION — REMOVE, MOVE, or TIGHTEN — so the same content is read confidently. The ceiling pass is advisory and never blocks; it cannot relax a floor gate. Without it, repeated self-review monotonically over-defends. Surface the ceiling findings as their own first-class output (Phase 3), not folded silently into the "add this" comments. Phase 2.5i (the loop controller) then reads the floor + ceiling state to declare when the loop is done — including a zero-edit PASS — so an accurate draft is not over-hardened by a pass it does not need.
Workflow
Phase 1: Intake
Get the manuscript -- PDF, Word doc, or pasted text.
Ask the user:
- Target journal? (affects reporting standards and scope expectations)
- Manuscript type? (original research / review / perspective / technical note / letter / meta-analysis / case report)
- Anything they're already worried about?
- Review depth? The default is a single-pass review. For a high-stakes pre-submission final pass, a multi-agent panel (
--panel, Phase 2.6) is available — several domain-expert reviewers run independently, then an editor consolidates them (more thorough, but it spawns several agents so it costs several times more tokens). On an interactive run, surface this option once in one line and offer it; then proceed with the single-pass review unless the user opts in. Do not surface or auto-apply the panel when invoked with--jsonor from/write-paper— those stay single-pass.
Read the full manuscript.
SSOT gate — confirm there is one manuscript, not several. Self-review reads a single input file, so a divergence between a legacy working copy and the live submission copy is structurally invisible to it. Before a
--panelrun (or any pre-submission pass), check for multiple copies and reconcile first:find . \( -path '*manuscript*' -o -path '*main_document*' \) -name '*.md' | grep -v node_modulesIf more than one manuscript-like file exists, confirm which is the SSOT and run
/sync-submission's divergence gate before reviewing — aSTALE_COPY(an SSOT numeric claim or heading that did not propagate to the other copy) is a P0 that must clear first:python3 "${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/sync-submission/scripts/detect_copy_divergence.py" \ --ssot <ssot>.md --copy <other-copy>.mdReview the SSOT copy; do not review a stale copy and pass it.
In
--panelmode this is a blocking precondition, not advice. A panel spawns N reviewer agents + an editor, so reviewing a stale copy wastes the whole pass (a prior panel's top finding was literally "you reviewed the wrong file"). If thefindabove returns more than one manuscript-like.mdand the SSOT is not pinned — noSSOT.yamlwithtruth.manuscript_mdand no explicit--ssot <path>argument — STOP before spawning any reviewer and have the user name the SSOT (and clear anySTALE_COPY). Do not auto-pick the longest/newest file. The single-pass review may proceed on the one file it was given, but the panel must not.
Phase 2: Systematic Check
Run the manuscript through each applicable category below. For each item, assess whether a reviewer would raise it as a Major or Minor comment. Use the Research-Type Adaptation table (below) to determine which categories apply fully, partially, or not at all.
The categories (A–L). The per-item check tables — what to look for under each — live
in references/phases/phase2_systematic_check.md; read it once you have the manuscript
and know its type, and work the categories the adaptation table marks as applicable.
| Category | What it asks | |
|---|---|---|
| A | Study Design & Data Integrity | patient-level splits, leakage, input-text contamination, analysis unit |
| B | Reference Standard & Ground Truth | definition specificity, timing, annotator independence |
| C | Validation & Statistical Reporting | CIs, calibration, comparator, effect size, power-aware nulls, equivalence margins, interaction anchoring |
| D | Clinical Framing & Importance | intended use, overclaiming, novelty, endpoint↔conclusion scope |
| E | Reproducibility | preprocessing, model detail, hardware/software, data & code availability |
| F | Reporting Completeness | abstract↔body consistency, flow diagram, ethics, missing data, word cap |
| G | Reporting Guideline Compliance | match the type to its checklist; /check-reporting does the item-level audit |
| H | Circularity | label–feature overlap, tautological prediction, circular validation |
| I | Protocol Heterogeneity | multi-site acquisition, harmonization, temporal protocol drift |
| J | Method Transparency | model provenance, fine-tuning, classical-style body conventions |
| K | Reviewer-team consistency | SR/MA only — dual-vs-single conjunction, LLM-as-reviewer (both fabrication-grade) |
| L | Editorial impression & defensiveness | advisory, never blocking — the ceiling category: REMOVE / MOVE / TIGHTEN |
Run the deterministic gates. These are greps and counts, so they belong in a gate rather than in eyeballing. Run them at Phase 2 entry, on every path:
# D. endpoint↔conclusion scope
python3 "${CLAUDE_SKILL_DIR}/scripts/check_scope_coherence.py" \
--manuscript manuscript.md --out qc/scope_coherence.json --strict
# J. classical-style body conventions
python3 "${CLAUDE_SKILL_DIR}/scripts/check_classical_style.py" \
--manuscript manuscript.md --out qc/classical_style.json --strict
# K. reviewer-team consistency (SR/MA only; pass the extraction JSON file or directory)
python "${CLAUDE_SKILL_DIR}/scripts/check_reviewer_team_consistency.py" \
--manuscript manuscript.md --prospero prospero/record.md \
--extraction-json extraction/ --out _audit_self/reviewer_team_consistency.md
# L. editorial impression (advisory; exits 0 even under --strict)
python3 "${CLAUDE_SKILL_DIR}/scripts/check_editorial_impression.py" \
--manuscript manuscript.md --out qc/editorial_impression.json
# J/D. Perspective structure (genre-gated: silent unless article_type is a Perspective).
# Pass the known type via --type; it also self-detects from the front-matter article_type.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_perspective_structure.py" \
--manuscript manuscript.md --type "${TYPE:-}" --out qc/perspective_structure.json
Verdict mapping: CROSS_SECTIONAL_PROGNOSTIC, SURROGATE_CARE_DIRECTIVE, SECTION_SYMBOL,
INBODY_AI_DISCLOSURE, and any reviewer-team hit (exit 1) are Anticipated Major Comments.
CROSS_SECTIONAL_YIELD_LANGUAGE, ELIGIBILITY_PROSE, DECIMAL_INCONSISTENCY,
EM_DASH_OVERUSE, PERSPECTIVE_HEADING_NOT_ASSERTION, PERSPECTIVE_ABSTRACT_NO_AUTHORIAL_MOVE,
and every check_editorial_impression verdict are Minor. The
per-verdict rationale and the resolution paths are in the reference file.
Read on demand:
Then check that every analysis you report was ever defined. The detectors in this skill ask whether a number is correct. None asks whether the analysis that produced it was defined — and that is the gap a reviewer walks straight into:
"The outcome (dependent variable) for the multivariable Cox model is not specified." … "The ground truth (reference standard) against which discrimination and calibration were assessed is not defined." … "This section is largely incomprehensible in its current form."
python3 "${CLAUDE_SKILL_DIR}/scripts/check_analysis_definitions.py" \
--manuscript manuscript.md --out qc/analysis_definitions.json --strict
MODEL_OUTCOME_UNDEFINED (a Cox / Fine–Gray / logistic model with no outcome named), MODEL_NOT_IN_METHODS, and REFERENCE_STANDARD_UNDEFINED (discrimination or calibration with nothing to score against) are Anticipated Major Comments. TIER_LABEL_UNDEFINED is Minor.
ANALYSIS_LOAD is informational and never a verdict. The reviewer who wrote "too many analyses have been performed and reported" also named the mechanism — "this appears to have contributed to omissions of critical information in the Materials and Methods section" — while a second reviewer of the same manuscript listed its sensitivity analyses as a strength. Load is the cause, not the crime. Do not cut analyses to satisfy this gate; restore the definitions the analyses crowded out. If load is genuinely high, move the defensive analyses to the supplement — same defence, far less reader burden and far less attack surface.
| File | Read it when | Cost if read blindly |
|---|---|---|
references/phases/phase2_systematic_check.md |
you are working the A–L manual pass and know the manuscript type | ~5,600 tokens — and a run that halts at Phase 1, or a panel-mode review, never reaches it |
Research-Type Adaptation
Not all categories apply equally to every study type. Use this routing table:
| Category | AI/ML | Observational | Educational | Meta-Analysis | Case Report | Surgical |
|---|---|---|---|---|---|---|
| A. Study Design | Full | Full | Partial | N/A | N/A | Full |
| B. Reference Standard | Full | Full | N/A | Per-study | Partial | Full |
| C. Validation & Stats | Full | Full | Full | Special* | Partial | Full |
| D. Clinical Framing | Full | Full | Full | Full | Full | Full |
| E. Reproducibility | Full | Partial | Partial | Partial | N/A | Full |
| F. Reporting | Full | Full | Full | Full | Full | Full |
| G. Guideline Compliance | Full | Full | Full | Full | Full | Full |
| H. Circularity | Full | Partial | N/A | N/A | N/A | Partial |
| I. Protocol Heterogeneity | Full | Full | N/A | Per-study | N/A | Full |
| J. Method Transparency | Full | Partial | Partial | N/A | N/A | Partial |
| K. Reviewer-team consistency | N/A | N/A | N/A | Full | N/A | N/A |
| L. Editorial impression | Full | Full | Full | Full | Full | Full |
*Meta-analysis: Replace C with heterogeneity assessment (I-squared, prediction intervals), publication bias (funnel plot, Egger), and sensitivity/subgroup analyses.
Type-Specific Additional Checks:
- Observational studies: Confounding assessment (DAG or adjustment strategy), selection bias, exposure measurement validity. Run Phase 2.5e (Confounding Completeness), then apply the O-probes in
references/domain-probes/observational_confounding.md— the two deterministic ones are O1 (a covariate imbalanced by exposure in Table 1 yet absent from the adjustment set) and O8 (records > subjects with the analysis unit undisclosed;check_cohort_arithmetic.py --id-col), and O7 is their opposite-direction twin (adjusting for a consequence/mediator of the outcome). If the manuscript develops or compares a clinical prediction model (TRIPOD / TRIPOD+AI, nested predictor-set comparison), also apply the CP-probes inreferences/domain-probes/clinical_prediction_model.md. The module is the single source for the probe list and its numbering; do not re-enumerate it here. - Educational studies: Learning outcome measurement validity, Kirkpatrick level, control group adequacy, curriculum fidelity
- Meta-analyses: Search comprehensiveness (2+ databases), screening reproducibility (2 reviewers), RoB assessment per study, GRADE certainty
- Case reports: Diagnostic reasoning transparency, timeline completeness, informed consent, generalizability disclaimer
- Surgical studies: Learning curve consideration, surgeon volume/experience, complication grading (Clavien-Dindo), operative detail completeness
Domain probe modules (load when the manuscript type matches):
These modules carry the same domain-specific critique probes used by /peer-review, vendored here so self-review reaches the same depth (in particular, survival/time-to-event manuscripts now get a dedicated probe set that the routing table above does not otherwise cover).
| Manuscript type / signal | Probe module |
|---|---|
| Systematic Review / Meta-Analysis | references/domain-probes/sr_ma.md (P0–P19) |
| Time-to-event / survival / prognostic model (Cox, Fine-Gray, DeepSurv, nomogram, risk-stratification cutoff) | references/domain-probes/survival_prognostic.md (S1–S9) |
| Radiomic feature reproducibility / acquisition-parameter sweep / reliability-based feature filtering | references/domain-probes/radiomics.md (R1–R4) |
| Cross-modality image synthesis (MRI→PET / MRI→CT / non-contrast→contrast / low-dose→full-dose) claiming functional/molecular information or target-modality substitution | references/domain-probes/image_synthesis.md (IS1–IS4) |
| Narrative / review article / primer / state-of-the-art | references/domain-probes/narrative_review.md (RV1–RV9) |
| Perspective / opinion / viewpoint (argumentative essay — npj DM long-essay, Lancet Comment, NEJM AI / RYAI short-structured) | references/domain-probes/narrative_review.md (RV1–RV9) + the deterministic check_perspective_structure.py gate above (IMRAD-heading + abstract authorial-move tells) |
| AI/ML primary study with a clinical claim (generalizable / outperforms clinicians / deployment-ready / can replace a reader) | references/domain-probes/ai_overclaiming.md (AO0–AO7) |
| Engineer-built medical-imaging model (segmentation / classification / detection; CNN / U-Net / nnU-Net / transformer) being validated — partition/leakage, seed & run variance, metric selection, reproducibility, reference-standard quality; plus saliency-map faithfulness, uncertainty/OOD/abstention, and deployment feasibility when a clinical-use claim is made | references/domain-probes/model_development.md (MD0–MD11) |
| LLM / MLLM evaluated on a clinical task (radiology report generation, visual question answering, clinical text extraction/classification; closed API or open weights) | references/domain-probes/mllm_evaluation.md (ME0–ME8) |
| Randomised controlled trial (parallel / crossover / cluster / stepped-wedge) | references/domain-probes/rct_trial.md (RC0–RC7) |
| Diagnostic test accuracy (DTA) primary study / multi-reader multi-case (MRMC) reader study (index test vs reference standard, AI-vs-reader, AI-assisted reading, modality comparison) | references/domain-probes/diagnostic_accuracy.md (D1–D12) |
| Case report / case series / single-patient clinical narrative (incl. adverse-event/pharmacovigilance and imaging-led radiology/nuclear-medicine/IR reports) | references/domain-probes/case_report.md (CR1–CR9) |
| AI/ML, prediction, or diagnostic study claiming cross-population performance (generalizable / deployment-ready / "works for patients"), or presenting subgroup analyses as a fairness/equity argument | references/domain-probes/equity_fairness.md (EQ0–EQ6) |
| Mendelian randomization (genetic variants as instrumental variables: two-sample summary-data, one-sample, multivariable MR, drug-target / cis-MR, non-linear MR) | references/domain-probes/mendelian_randomization.md (MR1–MR8) |
| Polygenic risk score / polygenic score (PRS / PGS) developed, validated, or applied as a predictor or risk-stratifier | references/domain-probes/polygenic_risk_score.md (PG1–PG8) |
| Network meta-analysis (≥3 interventions via direct + indirect evidence, treatment ranking, incl. component NMA) | references/domain-probes/network_meta_analysis.md (NM1–NM8) |
| Health economic evaluation (cost-effectiveness / cost-utility / cost-benefit / budget-impact; trial-based or decision-model-based — decision tree, Markov, DES) | references/domain-probes/health_economic_evaluation.md (HE1–HE8) |
| Observational study using routinely-collected health data (administrative claims / EHR / disease or population registry / health-checkup DB, linked or not) | references/domain-probes/record_routinely_collected_data.md (RD1–RD8) |
| Self-report survey / questionnaire study (KAP, physician/patient survey, cross-sectional questionnaire, web/e-survey) | references/domain-probes/survey_research.md (SV1–SV8) |
| Scoping review (maps the breadth/nature of evidence, clarifies concepts, identifies gaps; PCC framing, charting, optional appraisal — not a focused effectiveness/accuracy question) | references/domain-probes/scoping_review.md (SC1–SC8) |
| Qualitative study (interviews, focus groups, ethnography, grounded theory, phenomenology, document analysis; reflexivity, trustworthiness, thematic analysis — not quantitative validity) | references/domain-probes/qualitative_research.md (QL1–QL8) |
| Self-improving / self-evaluating system (an agent that critiques and rewrites its own output; training on model-generated data; an LLM used as the judge that scores the training signal; "self-evolving" clinical agents) | references/domain-probes/self_improving_system.md (SI1–SI7) + skills/peer-review/scripts/check_self_improvement_claims.py |
For a classifier / NLP / tabular ML manuscript, also run the deterministic feature-selection-leakage gate — a data-driven selection (feature selection, log-odds / univariate filtering, vocabulary construction, a threshold) fit on the FULL dataset before cross-validation inflates the CV metric:
python3 "${CLAUDE_SKILL_DIR}/scripts/check_cv_leakage.py" \
--manuscript manuscript.md --out qc/cv_leakage.json
CV_SELECTION_LEAKAGE (Major) fires when a selection token co-occurs with cross-validation and no fold-nesting is disclosed ("within each fold" / "nested CV" suppresses it). This is distinct from patient-vs-image split leakage (model-validation/check_split_leakage.py).
When the manuscript matches a row, read ${CLAUDE_SKILL_DIR}/references/domain-probes/<module>.md and apply each probe as an additional source of Anticipated Major / Minor Comments. The module severity words (MAJOR / MINOR) map to this skill's framing as follows: a conclusion-threatening or design-level finding becomes a Fatal Anticipated Major Comment, a reporting-level finding becomes a Fixable Anticipated Minor Comment, and each is tagged with the closest category letter (A–K). These probes complement categories A–K above; they do not replace them. (The modules are vendored byte-identical from /peer-review; do not edit one copy only — run python3 scripts/check_domain_probe_sync.py --sync.)
Phase 2.5: Numerical Cross-Verification (Internal)
Before generating the report, verify internal consistency:
- Abstract vs Body: Do all numbers in the Abstract match the Results section and Tables?
- Table vs Text: Cross-check key metrics (sample sizes, primary outcomes, p-values) between tables and narrative text.
- Figure vs Text: Do figure legends match the data described in Results?
- Percentage arithmetic: Verify that n/N percentages are calculated correctly (e.g., 23/150 = 15.3%, not 15.0%).
- CI plausibility: Do confidence intervals seem reasonable given sample sizes?
- Rate back-calculation: every reported rate must invert to its own numerator/denominator — an incidence rate ≈ events / person-years × scale (±rounding). A rate that does not recompute from the stated events and person-time (or that implies more events than the cohort can supply) is a Major, not a Minor.
- Exclusion-cascade and complete-case arithmetic (cohort/observational): the STROBE flow must balance — start N − Σ(exclusions) == final analytic N — and any complete-case statement must balance — total − missing == complete. A footnote N that does not equal the subtraction is a Major.
For cohort/observational manuscripts, run the deterministic gate instead of eyeballing it (it parses prose equations + GFM tables, and recomputes from a committed CSV when given one):
python3 "${CLAUDE_SKILL_DIR}/scripts/check_cohort_arithmetic.py" \
--manuscript manuscript.md --data analysis/cohort.csv --id-col mockid \
--out qc/cohort_arithmetic.json --strict
RATE_BACKCALC / CASCADE_SUM / PARTITION_OVERLAP rows are Anticipated Major Comments (category: A. Study Design & Data Integrity); the partition check is the Phase 2.5b cohort branch below. Pass --id-col (or let it auto-detect a subject-ID column) on health-screening / EMR / registry data so the gate also runs the analysis-unit check: when records > unique subjects and the manuscript states neither the analysis unit nor a one-record-per-subject sensitivity, it emits ANALYSIS_UNIT_UNDISCLOSED (Major — non-independent observations give anti-conservative CIs; probe O8). Flag any remaining internal-consistency discrepancies as Anticipated Minor Comments (category: F. Reporting Completeness).
Then recompute the three things a reviewer recomputes by hand. These are the arithmetic checks a careful reviewer does with a calculator on the train home, and the ones that end a submission when they fail:
# Every "n (%)" in a table, recomputed against its own denominator.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_table_percentages.py" \
--manuscript manuscript.md --out qc/table_percentages.json --strict
# Every reported P beside a 2×2 (or r×c) count, recomputed from the counts themselves.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_reported_p_from_counts.py" \
--manuscript manuscript.md --out qc/reported_p.json --strict
# Diagnostic-accuracy only: sensitivity/specificity against the reference-standard denominators.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_dta_denominators.py" \
--manuscript manuscript.md --out qc/dta_denominators.json --strict
PCT_MISMATCH, P_MISMATCH / P_IMPOSSIBLE, and DENOM_MISMATCH are P0 Major — a percentage
that does not follow from its own denominator, or a P value that does not follow from its own counts,
is not a rounding disagreement. It means one of the two numbers is wrong, and the reviewer who checks
will find it. Run the first two on every manuscript with a table; the third only on
diagnostic-accuracy work.
Phase 2.5a: Numerical Source-Fidelity Audit (External)
Internal consistency (Phase 2.5) is necessary but not sufficient. Numbers can be fully self-consistent across Abstract / Table / Text and still be wrong at the source — a single transcription error propagates cleanly through every downstream stage, and every internal check then confirms it. Only a traversal back to the primary source catches it.
Run the displayed-arithmetic gate first — a stated difference must equal the subtraction of its two displayed component values at the same precision:
python3 "${CLAUDE_SKILL_DIR}/scripts/check_rounded_delta.py" \
--manuscript manuscript.md --out qc/rounded_delta.json
ROUNDED_DELTA_MISMATCH (Minor) fires when AUCs shown as 0.70 and 0.73 (a displayed gap of
0.03) are reported with a between-arm difference of 0.02 — self-consistent only on the
unrounded values. A higher-precision component pair (0.703 vs 0.726) with a 2-dp delta is
the legitimate unrounded case and is not flagged.
When to run the external audit: MA revisions, submissions, or any review where the user says "check against the source", "verify extraction", or "random sample". Skip otherwise.
The audit, in one line: draw a stratified sample of 5 numerical claims — always including one
comparative-arm value and one revision-introduced number, the two highest-yield strata — and
trace each through three layers (manuscript → extraction CSV → primary-source page; plus analysis
script → CSV where a script produced it). Any mismatch is a Major Comment, and one that
reverses a direction or crosses a significance boundary is a P0 blocker. Every [VERIFY-CSV] tag
is a mandatory audit item regardless of sample size.
The traversal procedure, the recording table, the sampling strata, and the four prose-judgement
rules it also applies — hand-entered analysis-script inputs, prose↔table statistic-type
mismatches (a median in the text against a mean in Table 1), stale derived CSVs after a
model/adjustment-set change (the analytic n is the fastest tell, and the conflict can flip
significance), and the precedent direction-reversal that internal consistency could not see —
are in the reference file.
Read on demand:
| File | Read it when | Cost if read blindly |
|---|---|---|
references/phases/phase2_5a_source_fidelity.md |
you are running the external audit — tracing sampled claims back to primary sources | ~2,500 tokens; a first-draft review with no extraction CSV and no primary sources cannot use any of it |
Phase 2.5a-2: Design & Power Statistic Provenance
A design or power statistic is computed, not copied from a source, so the source-fidelity audit of Phase 2.5a cannot check it — it has to be re-derived from the manuscript's own inputs. This applies only when the manuscript states a sample-size calculation, a power figure, or a detectable-effect claim.
Read on demand:
| File | Read it when | Cost if read blindly |
|---|---|---|
references/phases/phase2_5a2_design_power.md |
the manuscript reports a sample-size / power / MDE calculation | ~1,050 tokens; a manuscript with no power statement needs none of it |
Load-on-demand: read ${CLAUDE_SKILL_DIR}/references/phases/phase2_5a2_design_power.md when the manuscript reports a sample-size / power / MDE calculation.
Phase 2.5b: Screening-Count Reconciliation from ID Sets (SR/MA + observational tier/stratum)
Internal consistency across Abstract/Methods/Results (Phase 2.5) and source fidelity of 2×2 and effect-size numbers (Phase 2.5a) do not cover study-count arithmetic. That is a separate failure mode: a prior-draft prose total ("30 → 32 after FLAG consensus") survives every downstream pass because Abstract, Methods, Results, Discussion, the Figure 1 caption, and even the supplementary consensus file all cite the same wrong number back to each other. The only thing that catches it is a recount from the ID sets.
When to run: any SR/MA manuscript revision, regardless of stage (run before Phase 3); or any observational manuscript presenting an ordinal tier / mutually-exclusive stratum split. Skip otherwise.
A. SR/MA — recount from the ID sets. Derive every study count from the screening TSV and the
consensus sheet rather than from prose, and list the narrative-only IDs explicitly — the
highest-yield cross-check, the one that turns "10 narrative-only studies" into "2 (IDs 120, 474)".
Any derived total that disagrees with Abstract, Methods, Results, the Figure 1 caption or
Limitations is a P0 Major, blocking submission, and any N → M transition claim not backed by
an enumerable ID addition/subtraction set is itself a Major — it is unverifiable by downstream
audit. The set definitions, the derivation formulas and the reconciliation-block template are in
the reference file.
B. Observational tier/stratum — the same set logic, as arithmetic. A partition claimed to be
disjoint must satisfy Σ(stratum N) == unique total and Σ(stratum events) == total events.
Denominators summing above the unique cohort double-count subjects; a table where every
stratum n equals the grand total is a mis-entry, not a partition. Confirm the reference
(baseline) row of any stratified hazard/odds table is present and labelled — without it the
other strata are uninterpretable.
python3 "${CLAUDE_SKILL_DIR}/scripts/check_cohort_arithmetic.py" \
--manuscript manuscript.md --data analysis/strata.csv --strict
C. Cross-script cut-point consistency — the root cause of stratum-N drift. When the same
cohort is re-stratified in more than one analysis script, the derived categorical must use one
identical cut definition (same breaks, same right= closure, same labels). Two scripts binning
one variable differently drift the per-stratum Ns while the grand total still reconciles — so a
manuscript-only check cannot localize it. The same gate covers the composite-indicator sibling
(a derived 0/1 criterion rebuilt in a second script with a clause dropped).
python3 "${CLAUDE_SKILL_DIR}/scripts/check_binning_consistency.py" \
--root analysis --root scripts --strict
PARTITION_OVERLAP, BINNING_DRIFT, and DERIVED_DEF_DRIFT are all P0 Major.
Read on demand:
| File | Read it when | Cost if read blindly |
|---|---|---|
references/phases/phase2_5b_screening_counts.md |
this is an SR/MA (ID-set recount) or a stratified cohort, and you are doing the recount | ~3,300 tokens — nothing in it applies to a single-cohort manuscript with no strata |
Phase 2.5c: Reference Scans (hallucination + adequacy)
Two scans run on the bibliography: 2.5c catches a citation that does not exist or whose first author is invented, and 2.5c-2 catches a claim that carries no citation at all. Both need a bibliography — a draft with no refs.bib and no reference list skips them entirely. Run /verify-refs --strict first; these scans read its audit rather than re-deriving it, then run the adequacy checker:
python3 "${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/self-review/scripts/check_reference_adequacy.py" \
--manuscript manuscript/manuscript.md --bib "$BIB" \
--article-type "$TYPE" ${CAP:+--journal-cap "$CAP"} \
--out qc/reference_adequacy.json --strict
Read on demand:
| File | Read it when | Cost if read blindly |
|---|---|---|
references/phases/phase2_5c_reference_scans.md |
the manuscript has a bibliography and you are auditing citations | ~1,950 tokens; a draft with no reference list cannot use any of it |
Load-on-demand: read ${CLAUDE_SKILL_DIR}/references/phases/phase2_5c_reference_scans.md when the manuscript has a bibliography and you are auditing citations.
Phase 2.5d: Cross-Reference QC (Manuscript ↔ rendered DOCX)
Reference-list integrity (Phase 2.5c) does not cover Table/Figure cross-references. That is a separate failure mode: an in-text citation ("Supplementary Table S4 reports a sensitivity analysis") resolves to a different caption in the rendered DOCX ("Supp Table S4 = a diagnostics table") because the build script carries its own legacy SSOT. Internal consistency (Phase 2.5) cannot see it — the prose and the build artifact each echo their own divergent truth cleanly.
Markdown stage (always). Every captioned Figure N. / Table N. must be cited at least once
elsewhere in the body:
python3 "${CLAUDE_SKILL_DIR}/scripts/check_figure_citation.py" \
--manuscript manuscript.md --out qc/figure_citation.json
FIGURE_ORPHAN / TABLE_ORPHAN (Minor) catch a newly-added float that has a legend but no
in-text citation — the early, no-build counterpart to check_xref's UNCITED.
DOCX stage (when a rendered DOCX exists — circulation drafts, post-build pre-submission checks. Skip on early drafts with no build):
python3 "${MEDSCI_SKILLS_ROOT:-$HOME/workspace/medsci-skills}/skills/manage-refs/scripts/check_xref.py" \
--md manuscript/manuscript.md --docx manuscript/manuscript_final.docx \
--out qc/xref_audit.json [--allow-separate-attachments]
Severity depends on the journal's figure/table submission policy. Many radiology and medical
journals (European Radiology, Radiology, AJR) accept figures and tables as separate
attachments rather than inline — pass --allow-separate-attachments there so a legitimate
attachment style is not read as a blocker.
| Status | Default policy | With --allow-separate-attachments |
|---|---|---|
MISSING_DOCX |
Major (P0) — cited Table/Figure absent from rendered output | Minor — separately attached per journal policy |
MISSING_BODY |
Major (P0) — no body caption definition | Major (P0) when the float IS in the rendered DOCX (SSOT drift). Minor when no --docx was supplied — excused without evidence, and reported as such |
MISMATCH |
Major (P0) — caption text disagrees between body and rendered DOCX | Major (P0) (no change) |
UNCITED |
Minor — orphan caption; cite it or remove it | Minor (no change) |
MISMATCH stays P0 under every policy. So does MISSING_BODY when the float is present in the
rendered DOCX — the build pipeline is then the only place that knows the caption text, which is SSOT
drift and not a style choice. MISSING_BODY with no --docx supplied is different: nothing was
checked, so under --allow-separate-attachments it is excused on the author's declaration and the run
says so in those words. Treat those rows as unverified, not as verified — re-run with --docx before
submission and read summary.downgraded_unchecked in the audit JSON.
Do NOT auto-fix cross-reference defects in --fix mode. Rewriting a caption in the body
without re-running the DOCX build merely moves the mismatch. Emit each P0 row as its own
M-numbered Major Comment with category: "F" and fixable_by_ai: false, and route the user to
/write-paper Step 7.6a for the pipeline-side fix.
Read on demand:
| File | Read it when | Cost if read blindly |
|---|---|---|
references/phases/phase2_5d_xref_qc.md |
the xref gate fired and you are writing up the reconciliation | ~2,400 tokens; an early draft with no DOCX build never reaches this stage |
Phase 2.5e: Confounding Completeness (observational only)
When to run: the manuscript is observational (cohort, case-control, cross-sectional, health-screening registry) and the central claim is an adjusted exposure–outcome association. Skip for RCTs, diagnostic-accuracy, SR/MA, and descriptive studies — which is why the full procedure is loaded on demand rather than carried inline.
The highest-yield, most mechanical observational finding — a covariate that is measured,
imbalanced across exposure groups in Table 1, and absent from the adjustment set
(residual confounding by a measured variable) — is invisible to a prose pass and only
exposed by joining the exposure-stratified Table 1 against the Methods adjustment set
(probe O1). Run the deterministic gate and treat each UNADJUSTED_IMBALANCED covariate as
an Anticipated Major Comment (category A. Study Design & Data Integrity):
python3 "${CLAUDE_SKILL_DIR}/scripts/check_confounding_completeness.py" \
--table1 table1_by_<exposure>.csv \
--adjusted-list "age, sex, BMI, hypertension, diabetes" \
--exposure-defining-list "body mass index, waist, fasting glucose, triglycerides, HDL cholesterol" \
--out qc/confounding_completeness.json --strict
When the manuscript is observational, load references/phases/confounding_completeness.md
for the full procedure: the precedent failure pattern; the --exposure-defining-list
over-adjustment exemption for guideline-defined exposures (MASLD / metabolic syndrome / CKM
/ sarcopenia / frailty); the SMD-from-mean ± SD fallback; the extended-adjustment
sensitivity model and its frame discipline (refit the unadjusted estimate on the reduced
complete-case frame, not the full frame); and the rest of the observational probe set
(O2–O10) from references/domain-probes/observational_confounding.md.
Phase 2.5f: Claim-vs-Artifact Cross-Check
Phases 2.5–2.5e check numbers and adjustment sets. This phase checks claims against the external artifacts they should trace to — the pre-registration, the protocol, the analysis outputs. These are the errors that survive a single-pass review because the manuscript prose is internally consistent yet disagrees with the registration or the analysis it reports: a primary re-designated after the results were known, an E-value that does not recompute from the estimate it is quoted against, an analysis promised in Methods that never reaches Results.
Run the gates (all deterministic; pass the supplement so the corpus is complete):
# 1. claims ↔ pre-registration/protocol: estimand provenance + E-value arithmetic
python3 "${CLAUDE_SKILL_DIR}/scripts/check_claim_artifact.py" \
--manuscript manuscript.md --prereg prereg.md \
--out qc/claim_artifact.json --strict
# 2. Methods ↔ Results ↔ disk coverage (both directions: promised-absent AND run-but-unreported)
python3 "${CLAUDE_SKILL_DIR}/scripts/check_artifact_coverage.py" \
--manuscript manuscript.md --supplement supplement.md --analysis-dir output/analysis \
--out qc/artifact_coverage.json --strict
# 3. reader-facing residue in EVERY rendered artifact, not just the body
python3 "${CLAUDE_SKILL_DIR}/scripts/check_supplement_hygiene.py" \
--supplement supplement.md --supplement tables.md --supplement captions.md \
--manuscript manuscript.md --out qc/supplement_hygiene.json --strict
# 4. float AND in-text reference-numb
…(truncated)