Data Labeling QA

Audit an untrusted labeled training set before fine-tuning. Find mislabeled examples with four complementary techniques — provenance/integrity checks (off-by-one, schema drift), cleanlab confident learning (random noise), high-loss monitoring (hard cases), and LLM-as-judge on the flagged subset (systematic confusion). Use before spending GPU on any fine-tune where you don't fully trust the labelers. Works with any classification dataset reshaped to `data` and `untrustworthy_label` columns.

brojonat Updated 1 repo stars

File contents

brojonat/llmsrules commit f2173bc1ff

Frequently asked questions

npx get-skill add brojonat/data-labeling-qa