# Prepare Dataset

> Process and validate datasets for training. Use when setting up data pipelines.

- Skill: `majiayu000/prepare-dataset` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add majiayu000/prepare-dataset`
- Raw SKILL.md: https://api.skillmd.com/api/skills/majiayu000/prepare-dataset/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: majiayu000 (https://skillmd.com/u/majiayu000)
- Updated: 2026-09-09
- Page: https://skillmd.com/skills/majiayu000/prepare-dataset

---


# Prepare Dataset

Load, preprocess, and validate datasets for machine learning model training including normalization and augmentation.

## When to Use

- Setting up data pipelines for training
- Normalizing and cleaning raw data
- Splitting into train/validation/test sets
- Applying data augmentation

## Quick Reference

```python
# Dataset preparation pipeline
class DatasetLoader:
    def load(self, path: str) -> Tuple[ndarray, ndarray]:
        # Load raw data
        pass

    def normalize(self, data: ndarray) -> ndarray:
        # Normalize to [0, 1] or standardize
        pass

    def split(self, data: ndarray, ratios: Tuple[float, float, float]):
        # Split into train/val/test
        pass

    def augment(self, data: ndarray) -> ndarray:
        # Apply transformations if needed
        pass
```

## Workflow

1. **Load raw data**: Read dataset from file (CSV, HDF5, NumPy)
2. **Validate data**: Check shape, dtype, missing values
3. **Preprocess**: Normalize, standardize, encode categorical features
4. **Split sets**: Create train/validation/test splits
5. **Augment data**: Apply transformations if needed (rotation, flip, etc.)

## Output Format

Dataset preparation report:

- Raw data shape and statistics
- Data validation results (missing values, outliers)
- Preprocessing applied (normalization, encoding)
- Train/val/test split sizes
- Final dataset shape and statistics
- Augmentation transformations applied

## References

- See `extract-hyperparameters` skill for data preprocessing config
- See `evaluate-model` skill for test set evaluation
- See `/notes/review/mojo-ml-patterns.md` for Mojo data loading