Reproducible ML Experiments
- Convert the approved experiment design into explicit configuration files and runnable commands.
- Pin code revision, dependencies, model/tokenizer versions, dataset snapshot, preprocessing, seeds, and hardware details where feasible.
- Validate the data path, split logic, labels, batch shapes, loss, and metric implementation before long runs.
- Run a smoke test on a small sample; confirm checkpointing, logging, and resume behavior.
- Log configuration, environment, run ID, random seeds, resource use, metrics, and artifact locations.
- Compare models under the same data, split, metric, and budget. Run multiple seeds when variance matters.
- Keep validation-driven choices separate from final test evaluation.
- Report failed, interrupted, and incomplete runs; preserve enough state to reproduce results.
Never invent metrics or present simulated values as measured. A checkpoint existing does not prove a run completed. Do not launch expensive or remote jobs without the user's approval and a clear budget/resource plan.