Hypotheses
A hypothesis is a statement the world could contradict, paired with the observation that
would contradict it. Everything else is a theme. The work here is turning a direction into
statements of that kind, and designing the smallest experiment that tells them apart.
Rules
- Falsifiable, specific, mechanistic. "Method A improves accuracy" is a theme. "Under
label noise above 20%, A's margin over B grows with noise because A's loss saturates on
mislabeled examples" is a hypothesis: direction, condition, mechanism, and a prediction
that could fail.
- Always at least two. The hypothesis and the best competing explanation for the same
observation (confound, artifact, simpler mechanism). An experiment that cannot separate
them is not yet designed.
- Predictions before data. For each hypothesis, the observable pattern if it is true
and if it is false, written down before the experiment. Pre-specify the primary outcome,
the analysis and the decision rule.
- Power the study. Seeds, samples or replicates chosen from the expected effect size
and variance, not from habit. Three seeds detect only large effects; say what the study
can and cannot detect.
- Control what could explain the result otherwise. Randomize assignment, block on
known nuisance factors, include the positive and negative controls that show the
measurement works.
- Cheap discriminating experiments first. Order experiments by information per unit
cost; the first one should be able to kill at least one hypothesis.
Workflow
Hypotheses. Use references/quality-criteria.md as the bar: testable, specific,
grounded in a mechanism or prior evidence, scoped to conditions, and different from its
competitors in at least one prediction. Name the variables: independent, dependent,
controlled, and the confounds you cannot control.
Design. Pick the design from references/design-patterns.md (between-subjects,
within-subjects, factorial, crossover, ablation ladder, dose-response, natural
experiment). Randomize what can be randomized; block or stratify what cannot; hold the
evaluation fixed across arms. For computational studies: same data and splits for every
arm, seeds fixed and varied together, the budget equalized, the metric defined once.
Sample size. Estimate the effect size you care about and the variance from a pilot,
prior work or a tracked run. Compute n or the seed count for the power you want; a quick
simulation in Python is more honest than a formula applied to the wrong test. When a
library skill helps, statistical-power and experimental-design in the library carry
the formulas and package calls. State the minimum detectable effect in the plan.
Analysis plan. The primary outcome and its test (paired or unpaired, correction for
multiple comparisons, the model if regression), secondary outcomes labelled as such, how
outliers and missing data are handled, and the exact criterion for each hypothesis's
verdict. Written before any result is seen.
Deliverable
## Question
## Known
## Hypotheses
H1: <statement>. Mechanism: <...>. Predicts: <if true> / <if false>.
H2 (competing): ...
H0: ...
## Discriminating experiment
Design, arms, controls, randomization, fixed evaluation.
n / seeds: <value>, detects effects ≥ <MDE> at power <p>.
Primary outcome and test; decision rule per hypothesis.
## Order of experiments and cost
## Risks: confounds not controlled, measurement validity, external validity
Before you hand it over
- Each hypothesis has a prediction that would falsify it.
- At least one experiment separates every pair of hypotheses that matter.
- n and the minimum detectable effect are stated with their basis.
- The analysis plan is complete enough that a second person would reach the same verdict
from the same data.
1---2name: hypotheses3description: Turns a research direction into testable hypotheses with predictions, competing explanations and the experiments that discriminate between them, including the design (controls, randomization, blocking), sample size or seed count, the pre-specified analysis, and the result that would falsify each hypothesis. Use when a question needs to become a study, before data is collected or runs are launched, or when an observation needs candidate explanations. For generating directions use brainstorming; for running the loop use autoresearch.4summary: "Turn a direction into falsifiable hypotheses and the discriminating experiment, powered."5category: core6role: workflow7allowed-tools: [Read, Write, glob, grep, python, webfetch]8license: MIT9version: 1.0.010author: Synthetic Sciences11metadata:12 source: https://skillmd.com/skills/synthetic-sciences/hypotheses13 upstream: K-Dense-AI/claude-scientific-writer14 upstream-url: https://github.com/K-Dense-AI/claude-scientific-writer15 upstream-license: MIT16 upstream-relationship: references and scripts derived17 adapted-by: Synthetic Sciences18 skill-author: Synthetic Sciences19 adapted-from: K-Dense hypothesis-generation and scientific-critical-thinking (MIT)20---2122# Hypotheses2324A hypothesis is a statement the world could contradict, paired with the observation that25would contradict it. Everything else is a theme. The work here is turning a direction into26statements of that kind, and designing the smallest experiment that tells them apart.2728## Rules29301. **Falsifiable, specific, mechanistic.** "Method A improves accuracy" is a theme. "Under31 label noise above 20%, A's margin over B grows with noise because A's loss saturates on32 mislabeled examples" is a hypothesis: direction, condition, mechanism, and a prediction33 that could fail.342. **Always at least two.** The hypothesis and the best competing explanation for the same35 observation (confound, artifact, simpler mechanism). An experiment that cannot separate36 them is not yet designed.373. **Predictions before data.** For each hypothesis, the observable pattern if it is true38 and if it is false, written down before the experiment. Pre-specify the primary outcome,39 the analysis and the decision rule.404. **Power the study.** Seeds, samples or replicates chosen from the expected effect size41 and variance, not from habit. Three seeds detect only large effects; say what the study42 can and cannot detect.435. **Control what could explain the result otherwise.** Randomize assignment, block on44 known nuisance factors, include the positive and negative controls that show the45 measurement works.466. **Cheap discriminating experiments first.** Order experiments by information per unit47 cost; the first one should be able to kill at least one hypothesis.4849## Workflow5051- [ ] State the observation or question and what is already known about it.52- [ ] Write two to four hypotheses with mechanisms; include the null and the confound.53- [ ] For each, the predictions if true and if false; mark which predictions differ.54- [ ] Design the discriminating experiment; check it against `references/experimental-design.md`.55- [ ] Choose n; write the analysis plan; define the decision rule.56- [ ] Hand over as a study plan, or start it with autoresearch.5758**Hypotheses.** Use `references/quality-criteria.md` as the bar: testable, specific,59grounded in a mechanism or prior evidence, scoped to conditions, and different from its60competitors in at least one prediction. Name the variables: independent, dependent,61controlled, and the confounds you cannot control.6263**Design.** Pick the design from `references/design-patterns.md` (between-subjects,64within-subjects, factorial, crossover, ablation ladder, dose-response, natural65experiment). Randomize what can be randomized; block or stratify what cannot; hold the66evaluation fixed across arms. For computational studies: same data and splits for every67arm, seeds fixed and varied together, the budget equalized, the metric defined once.6869**Sample size.** Estimate the effect size you care about and the variance from a pilot,70prior work or a tracked run. Compute n or the seed count for the power you want; a quick71simulation in Python is more honest than a formula applied to the wrong test. When a72library skill helps, `statistical-power` and `experimental-design` in the library carry73the formulas and package calls. State the minimum detectable effect in the plan.7475**Analysis plan.** The primary outcome and its test (paired or unpaired, correction for76multiple comparisons, the model if regression), secondary outcomes labelled as such, how77outliers and missing data are handled, and the exact criterion for each hypothesis's78verdict. Written before any result is seen.7980## Deliverable8182```markdown83## Question84## Known85## Hypotheses86H1: <statement>. Mechanism: <...>. Predicts: <if true> / <if false>.87H2 (competing): ...88H0: ...89## Discriminating experiment90Design, arms, controls, randomization, fixed evaluation.91n / seeds: <value>, detects effects ≥ <MDE> at power <p>.92Primary outcome and test; decision rule per hypothesis.93## Order of experiments and cost94## Risks: confounds not controlled, measurement validity, external validity95```9697## Before you hand it over9899- Each hypothesis has a prediction that would falsify it.100- At least one experiment separates every pair of hypotheses that matter.101- n and the minimum detectable effect are stated with their basis.102- The analysis plan is complete enough that a second person would reach the same verdict103 from the same data.