Hf Datasets

Load, stream, process, and publish datasets with the HuggingFace datasets library. Covers Apache Arrow-backed streaming for large datasets, map/filter operations, train/val/test splitting, interleaving, concatenation, and pushing to the Hub.

mkurman Updated 327 repo stars
Picture from the README

File contents

mkurman/zorai commit 2f35db256f Runs: mkurman/embedding-analysis

Frequently asked questions

npx get-skill add mkurman/hf-datasets-2