# Foundations R Hypothesis Testing And Nonparametrics

> Run exact/nonparametric hypothesis tests in R: binom.test, sign test, wilcox.test/wilcox.exact (Wilcoxon signed-rank and Mann-Whitney U), kruskal.test with Dunn post-hoc, Hodges-Lehmann CIs, and binomial power/sample-size functions. Use when a user asks to test proportions, compare paired or independent samples that are non-normal or small-n, run Mann-Whitney/Wilcoxon/Kruskal-Wallis in R, or compute power/sample size for a binomial test.

- Skill: `pavel-kravchenko/foundations-r-hypothesis-testing-and-nonparametrics` (Agent Skill)
- Install (CLI): `npx skillmds@latest add pavel-kravchenko/foundations-r-hypothesis-testing-and-nonparametrics`
- Raw SKILL.md: https://api.skillmd.com/api/skills/pavel-kravchenko/foundations-r-hypothesis-testing-and-nonparametrics/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: pavel-kravchenko (https://skillmd.com/u/pavel-kravchenko)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/pavel-kravchenko/foundations-r-hypothesis-testing-and-nonparametrics

---


# R Hypothesis Testing and Nonparametric Methods

## When to Use

- Testing a proportion against a fixed value (e.g., "42 of 120 patients responded — does that differ from a 40% historical rate?").
- Comparing paired measurements (before/after, treated/control on the same subject) with `n` too small or too skewed to trust a paired t-test.
- Comparing two independent groups (e.g., biological replicates, treatment arms) that are non-normal — Mann-Whitney U / Wilcoxon rank-sum instead of `t.test`.
- Comparing 3+ independent groups nonparametrically (Kruskal-Wallis) and finding which pairs differ (Dunn post-hoc).
- Computing power or required sample size for a binomial test before running an experiment.

## Version Compatibility

- R ≥ 4.0, base `stats` package (ships with R: `binom.test`, `wilcox.test`, `kruskal.test`, `p.adjust`).
- `exactRankTests` (CRAN, last updated for R ≥ 3.0) for exact Wilcoxon p-values (`wilcox.exact`) — needed when ties are present or `n` is small enough that `wilcox.test`'s exact method refuses.
- `PMCMRplus` (actively maintained; the older `PMCMR` is archived on CRAN) for Dunn's post-hoc test after Kruskal-Wallis.

## Prerequisites

- `install.packages(c("exactRankTests", "PMCMRplus"))`
- Familiarity with R's `d/p/q/r` distribution-function convention (below) and with reading a `data.frame`.
- Concept prerequisite: understand p-values, null/alternative hypotheses, and one- vs two-sided tests.

## R Distribution Function Convention

| Prefix | Returns | Example |
|---|---|---|
| `d` | Density/mass at x | `dnorm(x, mean, sd)` |
| `p` | CDF: P(X ≤ x) | `pnorm(q, mean, sd)` |
| `q` | Quantile (inverse CDF) | `qnorm(p, mean, sd)` |
| `r` | Random sample | `rnorm(n, mean, sd)` |

Suffixes: `norm`, `binom`, `t`, `chisq`, `f`, `pois`, `exp`, `unif`, `nbinom`.
`lower.tail=FALSE` gives P(X > x).

```r
# P(X > 54) for Binomial(100, 0.25)
pbinom(54, size = 100, prob = 0.25, lower.tail = FALSE)

# Normal: P(X > 252) where mean=262.5, sd=12
pnorm(252, mean = 262.5, sd = 12, lower.tail = FALSE)

# Sample size for a 99.5% CI half-width <= 0.5, sigma = 12
ceiling((12 / 0.5 * qnorm(0.995))^2)
```

## Exact Binomial Test + Power

**Goal:** decide whether an observed proportion differs from a fixed `p0`, and (before running the study) determine the sample size needed to detect a given effect.
**Approach:** `binom.test` gives the exact p-value from the binomial CDF; a hand-rolled power function built on `qbinom`/`pbinom` lets you scan power across a range of true proportions without extra packages.

```r
# H0: p = 0.5 (e.g. 701 of 1002 patients prefer treatment A). H1: p > 0.5 (right-sided).
binomial_test_report <- function(successes, n, p0, alternative = "two.sided") {
  # Wraps binom.test and also reports the normal-approximation p-value for comparison.
  exact <- binom.test(successes, n, p0, alternative = alternative)
  z <- (successes - n * p0) / sqrt(n * p0 * (1 - p0))
  approx_p <- switch(alternative,
    "greater"   = pnorm(z, lower.tail = FALSE),
    "less"      = pnorm(z, lower.tail = TRUE),
    "two.sided" = 2 * pnorm(-abs(z))
  )
  list(exact_p = exact$p.value, approx_p = approx_p, statistic_z = z)
}

binomial_test_report(701, 1002, 0.5, alternative = "greater")

# Power function for a left-sided test: H0: p=0.3, n=50, alpha=0.05
n <- 50; p_0 <- 0.3
qb <- qbinom(0.05, n, p_0, lower.tail = TRUE)   # critical boundary + 1
power <- function(p) pbinom(qb - 1, n, p, lower.tail = TRUE)
power(0.25)                    # power to detect true p = 0.25
cat("Type II error:", 1 - power(0.25), "\n")

# Required sample size for alpha=0.05, beta=0.025, true p1
required_n <- function(p1, p0 = 0.3) {
  ceiling(((qnorm(0.975) * sqrt(p1 * (1 - p1)) - sqrt(p0 * (1 - p0)) * qnorm(0.05)) / (p0 - p1))^2
}
required_n(0.25)
```

## Sign Test and Wilcoxon Signed-Rank (Paired Data)

**Goal:** test whether the median of paired differences is zero (e.g., weight before vs. after a diet).
**Approach:** the sign test only counts which member of each pair is larger (equivalent to `binom.test(p=0.5)`); the Wilcoxon signed-rank test additionally ranks the magnitude of each difference, so it is strictly more powerful when differences are symmetric.

```r
weight_before <- c(89.4, 92.1, 78.3, 101.5, 85.0, 95.2, 88.9, 90.1)
weight_after  <- c(87.1, 90.0, 79.0,  98.2, 83.5, 93.0, 86.2, 88.5)

# Sign test: H1 = weight decreases after diet (left-sided)
b <- sum(weight_after > weight_before)   # count of positive (increase) differences
n <- length(weight_before)
binom.test(b, n, p = 0.5, alternative = "less")

# Wilcoxon signed-rank test — uses magnitude as well as sign
library(exactRankTests)
wilcox.exact(weight_after, weight_before, paired = TRUE, alternative = "less", conf.int = TRUE)

# Manual signed-rank calculation (what wilcox.exact does internally)
manual_signed_rank <- function(x, y) {
  # x, y: paired vectors (e.g. after, before). Returns W+ and W-.
  d <- x - y
  rk <- rank(abs(d))
  signed_rk <- rk * sign(d)
  list(W_plus = sum(signed_rk[signed_rk > 0]), W_minus = sum(-signed_rk[signed_rk < 0]))
}
manual_signed_rank(weight_after, weight_before)
```

## Wilcoxon Rank-Sum / Mann-Whitney U (Independent Samples)

**Goal:** compare two independent, non-normal (or small-n) samples without assuming equal variance/normality.
**Approach:** `wilcox.test(paired=FALSE)` (or `wilcox.exact` for exact p-values with ties); add `conf.int=TRUE` to get the robust Hodges-Lehmann location estimate alongside the test.

```r
female <- c(118, 122, 130, 115, 128, 121, 119, 124)
male   <- c(135, 128, 140, 132, 138, 130, 142, 136)

# Two-sided Mann-Whitney U / Wilcoxon rank-sum test
wilcox.test(female, male, paired = FALSE, alternative = "two.sided")

# Hodges-Lehmann estimate (median of all pairwise differences) with 95% CI
library(exactRankTests)
wilcox.exact(female, male, paired = FALSE, alternative = "two.sided", conf.int = TRUE)
```

## Kruskal-Wallis Test + Dunn Post-Hoc (3+ Groups)

**Goal:** nonparametric analogue of one-way ANOVA — test whether ≥3 independent groups share the same distribution, then locate which pairs differ.
**Approach:** `kruskal.test` on a list of numeric vectors (or `formula` + `data.frame`); if significant, run `PMCMRplus::kwAllPairsDunnTest` with a multiple-testing correction.

```r
soil_type_1 <- c(23.1, 25.4, 22.8, 24.0, 26.1)
soil_type_2 <- c(28.5, 30.2, 27.9, 29.4, 31.0)
soil_type_3 <- c(21.0, 20.5, 22.2, 19.8, 21.5)

kruskal.test(list(soil_type_1, soil_type_2, soil_type_3))

# Dunn's post-hoc test with Benjamini-Hochberg correction
library(PMCMRplus)
crop_df <- data.frame(
  yield = c(soil_type_1, soil_type_2, soil_type_3),
  soil  = factor(rep(c("type1", "type2", "type3"), each = 5))
)
kwAllPairsDunnTest(yield ~ soil, data = crop_df, p.adjust.method = "BH")
```

## Test Selection Guide

| Data | Groups | Parametric | Nonparametric |
|---|---|---|---|
| Proportions | 1 | `binom.test` | — |
| Paired continuous | 2 | `t.test(paired=TRUE)` | `wilcox.test(paired=TRUE)` |
| Independent continuous | 2 | `t.test` | `wilcox.test` |
| Independent continuous | ≥3 | `aov` + `TukeyHSD` | `kruskal.test` + Dunn |

## Pitfalls

- **`wilcox.test` naming:** With two unpaired samples it performs Mann-Whitney U. With `paired=TRUE` it performs Wilcoxon signed-rank. The name is ambiguous — always specify `paired=`.
- **`p.adjust` method names:** Use `"BH"` for Benjamini-Hochberg FDR. Passing `"fdr"` fails silently (not a valid method name).
- **`t.test`/`wilcox.test` default is two-sided:** Use `alternative="greater"` or `alternative="less"` only when you have a pre-specified directional hypothesis.
- **Exact vs asymptotic Wilcoxon:** `wilcox.test`'s built-in `exact=TRUE` breaks down with ties (falls back to a warning + normal approximation); use `exactRankTests::wilcox.exact` when ties are present and you still need an exact p-value.
- **Kruskal-Wallis ≠ pairwise differences:** A significant `kruskal.test` only means at least one group differs. Follow up with Dunn's test (or `pairwise.wilcox.test`) plus a correction to find which pairs.
- **Hodges-Lehmann is a location estimate, not a mean:** It's the median of pairwise averages/differences — robust to outliers (a 10x outlier barely moves it, unlike `mean()`), but report it, don't confuse it with the sample mean.
- **Assumption violations:** Check normality (`shapiro.test`, Q-Q plot) and homoscedasticity before parametric tests; switch to the nonparametric analogue when assumptions fail.

## See Also

- `foundations-r-regression-correlation-and-diagnostics` — parametric follow-up (t-tests, ANOVA, regression assumptions).
- `foundations-probability` — underlying distribution theory behind `d/p/q/r` functions.
- `bio-experimental-design-power-analysis` — general power/sample-size calculations beyond the binomial case.
- `bio-experimental-design-multiple-testing` — correction methods (`p.adjust`) for many simultaneous tests.