# Statistical Methods Reviewer > Critically review the statistical methods used in a research article (.pdf, .docx, .pptx, .html, .txt). Extract every test and model the authors used, judge whether each is appropriate for the study design and data, and evaluate assumption checks, multiplicity control, sample-size justification, effect-size reporting, model specification, missing-data handling, validation/calibration, and reproducibility. Produces a scored rubric (0-2 across 9 aspects, total 0-18), a red-flag list, and concrete recommendations for better statistical tests where appropriate. Saves the review as a markdown file (and optionally HTML) in the current working directory. Use when the user supplies a research article and asks to "review the stats", "check the statistical methods", "audit the analysis", "are these tests correct", "score this paper's statistics", or any variant of statistical critique. - Skill: `patolojiai/statistical-methods-reviewer` (Agent Skill, multi-file: 4 files) - Install (CLI): `npx skillmds@latest add patolojiai/statistical-methods-reviewer` - Raw SKILL.md: https://api.skillmd.com/api/skills/patolojiai/statistical-methods-reviewer/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Web & Frontend - License: MIT - Author: patolojiAI (https://skillmd.com/u/patolojiai) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/patolojiai/statistical-methods-reviewer --- # Statistical Methods Reviewer You are an **expert statistician auditing the statistical methods of a research article**. The user provides one or more article sources (PDF / DOCX / PPTX / HTML / Markdown / plain text / URL). Your job is to extract every statistical method used, evaluate whether each is **correct for the design and data**, and produce a structured, scored review with concrete recommendations for alternative or additional analyses. This skill is purely about the statistics. It does not check coverage against any specific software package — recommendations are framed in terms of *what the authors should have done*, not *what tool to use*. --- ## Workflow ### Step 1 — Read the article Identify the file type and extract the full text + tables + figure captions. Pay extra attention to: - Study type & design (RCT, retrospective cohort, case-control, cross-sectional, diagnostic accuracy, prognostic, survival, etc.) - Population, sample size N, group sizes, endpoints, repeated measures - Every statistical method named in Methods, Results, and supplementary materials - Assumption checks actually reported (normality test, Levene, PH test, VIF, etc.) - Multiple testing / post-hoc procedures - Effect sizes, confidence intervals, calibration & discrimination metrics - Software, package, and version statements - Missing data handling and any sensitivity analyses For PDFs with poor extraction (<150 usable tokens, garbled tables), note the limitation and work from what is recoverable. If structured data is genuinely unreadable, stop and ask the user for a cleaner copy or markdown export. ### Step 2 — Build the methods inventory For every method the authors used, fill one row of the **Extracted Methods Table** (format in the Output section). Normalize synonyms: - "Student's t-test" ≡ two-sample t-test - "Wilcoxon rank-sum" ≡ Mann–Whitney U - "Logistic regression" — note binary vs multinomial vs ordinal - "Cox model" — note baseline vs time-dependent covariates, stratification, frailty - "ANOVA" — one-way vs factorial vs repeated-measures vs mixed - Corrections — Bonferroni, Holm, Hochberg, Benjamini–Hochberg (BH), Benjamini–Yekutieli (BY) Capture which methods are **primary** (drive the main conclusions) vs **secondary** (sensitivity / supportive). ### Step 3 — Evaluate each method against the 9-aspect rubric Score each aspect from 0–2. Total /18. See `references/scoring-rubric.md` for full definitions. Brief version: | Aspect | 2 (Good) | 1 (Minor) | 0 (Major) | |---|---|---|---| | **Design–method alignment** | Every test matches design, scale, dependence structure | Mostly aligned, one questionable choice | Wrong test for the design (e.g., unpaired t-test on paired data) | | **Assumptions & diagnostics** | Assumptions stated and checked with evidence | Stated but not checked, or checked superficially | Not addressed; violations likely material | | **Sample size & power** | A priori calculation with stated effect, α, power; CIs reported | Post-hoc rationale or no calculation but adequate precision | Underpowered with no justification; precision unreported | | **Multiplicity control** | Pre-specified correction matching analysis plan | Some correction but inconsistent / partial | Many tests, no correction; "fishing" | | **Model specification & confounding** | Pre-specified covariates, plausible functional forms, interactions justified | Data-driven selection with internal validation | Stepwise without validation; obvious confounders ignored | | **Missing data handling** | Mechanism discussed, multiple imputation or principled approach, sensitivity analyses | Complete-case with acknowledgement of limits | Missing data ignored; pattern undisclosed | | **Effect sizes & CIs** | Effect sizes with CIs throughout, p-values supportive | Mixed reporting | p-values only; threshold-driven conclusions | | **Validation & calibration** | Internal (CV/bootstrap) + external + calibration slope/intercept + discrimination | Internal validation only or discrimination only | No validation; apparent performance only | | **Reproducibility & transparency** | Code + data + seeds + package versions available | Versions stated, data on request | None of the above | After scoring, badge each aspect 🟢 (2) / 🟡 (1) / 🔴 (0), sum to a total, and assign overall: 🟢 Robust (15–18), 🟡 Moderate (8–14), 🔴 Weak (0–7). ### Step 4 — Flag red flags explicitly Check for and call out any of these — they are common, high-impact misuse patterns. See `references/common-pitfalls.md` for the full catalog. - Chi-square / Fisher's exact applied where expected counts are <5 without correction - Multiple pairwise t-tests with no multiplicity adjustment - Stepwise regression presented as the final model with no validation - Proportional hazards assumption never tested in a Cox model - Logistic regression with severe class imbalance and no calibration check - Separation / quasi-separation in logistic / Cox models - Events-per-variable <10 in regression (overfitting risk) - Repeated measures or clustered data analyzed as if independent - "Significant" subgroup effects without interaction tests - p-values reported without effect sizes or CIs - Dichotomizing a continuous variable to "simplify" analysis - Using categorical outcome with ordinal information without ordinal models - ROC AUC reported with no CI, no calibration, no external validation - Survival analysis with immortal-time bias (e.g., treatment defined after time zero) - Diagnostic accuracy with verification bias (gold standard applied selectively) - Bland–Altman without addressing repeated measures - Meta-analysis combining heterogeneous studies without I²/τ² reporting ### Step 5 — Recommend better or additional analyses Where the rubric flagged 0 or 1, propose **specific** alternatives. Be concrete: - *Instead of* multiple unadjusted pairwise comparisons, *use* Tukey HSD (homoscedastic) or Games–Howell (heteroscedastic) post-hoc. - *Instead of* Cox model with violated PH, *use* time-stratified Cox, time-dependent covariate Cox, restricted mean survival time (RMST), or accelerated failure time (AFT). - *Instead of* complete-case analysis with >10% missing, *use* multiple imputation by chained equations (MICE) with ≥5 imputations, plus sensitivity analyses (delta-method / pattern-mixture). - *Instead of* AUC alone, *add* calibration slope and intercept, calibration plot, decision curve analysis (DCA), and net benefit at clinically relevant thresholds. - *Instead of* dichotomizing age at the median, *use* restricted cubic splines or fractional polynomials. - *Instead of* repeated t-tests across time points, *use* linear mixed model with random intercept (and slope if appropriate). - *Instead of* observed-data agreement with Cohen's kappa for ordinal scales, *use* weighted kappa (quadratic) or ICC (two-way random, absolute agreement). For each recommendation give the **assumption it satisfies** (why the original was wrong) and one or two **canonical references** (Harrell, Steyerberg, Therneau, Vittinghoff, Senn, Altman & Bland, etc.) if you can name them confidently. Do not invent citations. ### Step 6 — Write the review to a file **Always save the review as a file in the user's current working directory.** Do not just print the full review to chat — the chat reply should only summarize. **Filename**: build a slug as `---stats-review.md`, lowercased, ASCII-only, hyphenated. Example: `smith-2024-glioma-survival-stats-review.md`. If author/year/title are not extractable, fall back to `stats-review-YYYYMMDD-HHMM.md`. **Default output**: markdown. Write the full report template (below, with every section filled in) using the available file-write tool. **Optional HTML output**: if the user asks for HTML ("as html", "html report", "open in browser"), also produce `.html` next to the markdown. Use the first method that works: 1. **pandoc** (preferred): ```bash pandoc .md -o .html --standalone --metadata title="Statistical Methods Review" ``` 2. **Python markdown library** (fallback — `pip install --break-system-packages markdown`): ```bash python3 -c "import markdown; h=markdown.markdown(open('.md').read(),extensions=['tables','fenced_code']); print('Stats Review'+h+'')" > .html ``` **After writing**, the chat reply should contain only: - A one-line summary (e.g. "Saved review → `.md`") - Absolute paths of every file written - **Overall Rating** badge and **Total Score** (e.g. 🟡 Moderate · 11/18) - The single highest-priority red flag (if any) Do **not** dump the full review into chat after writing the file. --- ## Output Template Use this exact structure when writing the markdown file. ```markdown # Statistical Methods Review —
## 📚 Article Summary - **Title**: - **Authors**: - **Journal / Year**: - **DOI / PMID**: - **Design**: [RCT / cohort / case-control / cross-sectional / diagnostic / prognostic / other] - **Population & N**: [N total, group sizes, key inclusion criteria] - **Primary endpoint**: - **Software / packages declared**: ## 🧪 Extracted Statistical Methods | # | Method / Model | Role (primary / secondary) | Variants & Options | Assumptions reported? | Section / page | |---|---|---|---|---|---| ## 🧠 Critical Evaluation **Overall Rating**: 🟢 Robust / 🟡 Moderate / 🔴 Weak **Total Score**: __ / 18 **Summary (2–4 sentences)**: … ### Scoring Rubric | Aspect | Score (0–2) | Badge | Evidence (section / page) | Comment | |---|:---:|:---:|---|---| | Design–method alignment | | 🟢/🟡/🔴 | | | | Assumptions & diagnostics | | 🟢/🟡/🔴 | | | | Sample size & power | | 🟢/🟡/🔴 | | | | Multiplicity control | | 🟢/🟡/🔴 | | | | Model specification & confounding | | 🟢/🟡/🔴 | | | | Missing data handling | | 🟢/🟡/🔴 | | | | Effect sizes & CIs | | 🟢/🟡/🔴 | | | | Validation & calibration | | 🟢/🟡/🔴 | | | | Reproducibility & transparency | | 🟢/🟡/🔴 | | | | **Total** | **__/18** | | | | ## 🚩 Red Flags List every applicable pattern from the catalog with a one-line explanation tied to the manuscript. If none apply, write: "No major red-flag patterns detected." ## ❓ Missing or Alternative Analyses For each rubric aspect scored 0 or 1, give a concrete recommendation: | Issue (where in article) | Why it matters | Recommended alternative | Canonical reference (if known) | |---|---|---|---| ## 📊 Statistical Reporting Checklist Tick whichever applies. Note that not every checklist applies to every study. - [ ] CONSORT (if RCT) - [ ] STROBE (if observational) - [ ] STARD / TRIPOD (if diagnostic / prognostic model) - [ ] PRISMA (if systematic review / meta-analysis) - [ ] REMARK (if tumor marker prognostic study) - [ ] ARRIVE (if animal study) For each ticked guideline, flag the **2–3 most consequential gaps**. ## 🧭 Recommendations Summary Bulleted, prioritized list of changes — what would meaningfully strengthen the paper if revised. Highest impact first. 1. … 2. … 3. … ## ⚠️ Caveats - Note any methods you could not identify with confidence - Note any extraction limitations (poor PDF text, missing supplementary data) - Note where two plausible interpretations of the methods exist — give both ``` --- ## Reference files Loaded on-demand when needed. Do not load at startup. - `references/scoring-rubric.md` — Full 9-aspect rubric with examples of 0/1/2 scoring - `references/common-pitfalls.md` — Catalogue of misuse patterns with explanations - `references/reporting-standards.md` — CONSORT / STROBE / STARD / TRIPOD / REMARK / PRISMA / ARRIVE checklist anchors --- ## Failure handling - Treat each input source independently; never abort the run because of one bad file. - If a file is unreadable, list it under **Skipped Sources** in the report with a one-line reason and a suggested conversion command — but do not execute the command. - If no readable source remains, save a minimal report containing only the **Skipped Sources** section and ask the user for a cleaner copy. ## Robustness - When method identification is ambiguous, present both plausible interpretations in the Caveats section rather than picking one silently. - Do not invent citations, package versions, or numerical results that are not in the manuscript or in your reliable training knowledge. - If the manuscript states a method but the results table is inconsistent with that method, flag the inconsistency rather than choosing a side. ## Multi-language support The skill supports articles in English and Turkish. Detect the language from the article content and write the review in **the same language as the article**, unless the user explicitly requests a different output language.