Stratify Eval Scores By Slice

Use when reporting an eval accuracy or win rate. Splits the pooled number into slices and surfaces the worst slice instead of hiding it in the mean.

jiayaoqijia Updated 81 repo stars

File contents

jiayaoqijia/cryptoskill commit 16f3e9dd69

Frequently asked questions

npx get-skill add jiayaoqijia/stratify-eval-scores-by-slice