woosook0127 Experiment Design
Use this to create an executable experiment plan or spec. Do not run the experiment here.
Procedure
- Confirm repo mapping is known. If not, use
repo-auditorframework-mappingfirst. - Define one primary hypothesis.
- Define the protected comparison surface:
- preserved baseline
- copied variant package if code changes are required
- dataset and split protocol
- checkpoint selection rule
- metric policy
- Define baseline, ablations, and variants as explicit deltas.
- Specify execution path: entrypoint, config/spec, model package, expected artifact root.
- Specify smoke validation before large runs.
- Mark unsupported metrics, data, or runtime assumptions honestly.
Output
- experiment proposal or spec
- baseline / variant / ablation plan
- metrics and artifacts
- reproducibility metadata
- smoke validation plan
- blockers and next owner
Guardrails
- A design without a baseline is incomplete.
- A design that cannot map to current code must say
unsupportedoradapter required. - Do not mutate preserved baselines in place.