Freeze Eval Splits Before Tuning

Use when an eval set is also used to choose prompts, thresholds, or hyperparameters. Locks a held-out split and its hash so tuning cannot leak into the reported number.

jiayaoqijia Updated 81 repo stars

File contents

jiayaoqijia/cryptoskill commit 28ee9e0d60

Frequently asked questions

npx get-skill add jiayaoqijia/freeze-eval-splits-before-tuning