1---2name: dataset-profiler-23description: Use to generate safe aggregate-only profiles for CSV, JSONL, text, or image datasets without exporting company raw samples.4---5# Dataset Profiler67## Inputs8- A local dataset path on a trusted machine.9- Allowed output folder for aggregate statistics.10- Optional task label: classification, retrieval, reranker, VLM, OCR, QA, or multimodal.1112## Workflow131. Inspect only aggregate metadata: row counts, columns, missing rates, label counts, length percentiles, file extensions, image dimensions when available.142. Write a `DATA_CARD.md` with safe summaries.153. Flag leakage risks, imbalance, duplicate IDs, missing modalities, and split mismatch.164. For company server work, ask Wyatt to run the script there and paste only the generated aggregate card or screenshot.175. Link the result to a `RUN_CARD` or `ERROR_CARD` when used in an experiment.1819## Hard rules20- Never export raw examples, raw text, image pixels, private identifiers, cookies, tokens, or credentials.21- Default to aggregate counts and percentages only.22- If a field name itself is sensitive, redact it before writing the card.23- Keep profiler scripts dependency-light so they can run on locked-down servers.24
WyattJY