# Statistical Method Selection

> Select appropriate statistical methods for experiment analysis

- Skill: `yogsoth-ai/statistical-method-selection` (Agent Skill)
- Install (CLI): `npx skillmds@latest add yogsoth-ai/statistical-method-selection`
- Raw SKILL.md: https://api.skillmd.com/api/skills/yogsoth-ai/statistical-method-selection/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: yogsoth-ai (https://skillmd.com/u/yogsoth-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/yogsoth-ai/statistical-method-selection

---


# Tactic: Statistical Method Selection

## Orchestration Pattern

1. **Assess Data Characteristics** → Determine distribution type, sample size, pairing structure
2. **metric-specification** → Ensure metrics are well-defined and measurable
3. **Select Test Family** → Choose between parametric, non-parametric, or Bayesian
4. **sample-size-estimation** → Power analysis for the selected test
5. **Define Analysis Pipeline** → Pre-register the complete analysis plan

## Decision Criteria

| Condition | Recommended Method |
|-----------|-------------------|
| Normal data, 2 groups, paired | Paired t-test |
| Normal data, 2 groups, unpaired | Welch's t-test |
| Normal data, 3+ groups | ANOVA + post-hoc (Tukey HSD) |
| Non-normal, 2 groups | Wilcoxon signed-rank / Mann-Whitney U |
| Non-normal, 3+ groups | Kruskal-Wallis + Dunn's test |
| Multiple datasets, multiple methods | Friedman + Nemenyi / critical difference |
| Want probability of superiority | Bayesian comparison (Benavoli 2017) |
| Small sample, no distributional assumptions | Permutation test |
| Variance estimation needed | Bootstrap confidence intervals |
| Multiple comparisons | Apply Holm-Bonferroni correction |

## Quality Checks

- Is the test appropriate for the data type (continuous, ordinal, categorical)?
- Are independence assumptions met? (If not, use paired/repeated-measures variants)
- Is the sample size sufficient for the chosen test's power requirements?
- Are multiple comparison corrections applied when testing multiple hypotheses?
- Is the effect size reported alongside p-values?
- Is the significance threshold pre-registered (not chosen post-hoc)?

<!-- BEGIN available-tables (generated) -->

## Available SOPs

Optional, no fixed order; the final leaf is always a sop.

| SOP | When to use |
| --- | --- |
| metric-specification | Define experiment metrics and significance standards |
| sample-size-estimation | SOP: power analysis and required experiment count estimation |

<!-- END available-tables (generated) -->