Experiment Execution

Experiment Execution

Use this when a task moves from paper planning into executable experiments, metrics, or reproducibility checks.

Workflow

1. Define hypothesis, baseline, treatment, dataset, metric, resource budget, and success criteria.
2. Record exact run configuration before execution: code revision, command, environment, seeds, data split, and output directory.
3. Prefer dry runs or small smoke tests before full runs.
4. Monitor logs and intermediate artifacts. Starting a run is not completion.
5. Interpret results with uncertainty:
   • compare against baseline;
   • check variance or confidence intervals;
   • inspect failure cases;
   • avoid overclaiming from a single run.
6. Package reproducibility evidence: command, config, artifact path, metric table, and known caveats.

Failure Handling

• If a run fails, preserve the error, config, and last valid artifact.
• If results are weak, report whether the issue is implementation, data, metric mismatch, or hypothesis weakness.
• If a method requires expensive compute, state the smallest meaningful validation first.

Hand Off

• Use statistical-analysis or bayesian-ppl-diagnostics for model-level statistical reasoning.
• Use scientific-visualization for paper-ready result figures.
• Use paper-workflow-orchestrator when results change the manuscript claim.