Execution Hygiene

Runs computations so they finish, reproduce and fit the machine that will re-execute them, detaching long jobs through compute_job instead of blocking or sleeping shells, checkpointing so reruns resume, fixing seeds and capping BLAS and OpenMP threads, streaming large tables with explicit dtypes, writing outputs atomically, logging every command with its exit code and rerunning the final pipeline from scratch under the target limits before finishing. Use for any analysis or pipeline that runs longer than a minute, reads files larger than memory, or will be re-executed on a machine with tighter CPU, memory, wall-clock or network limits than this one; a one-off shell command whose output is read immediately does not need it.

synthetic-sciences Updated 4.4k repo stars
An OpenScience session comparing EGFR inhibitor potency: the trace shows the search, the written files, the analysis run and the answer with a results table

File contents

synthetic-sciences/openscience commit 0848a606a2

Frequently asked questions

npx get-skill add synthetic-sciences/execution-hygiene