# Training Pipeline

> Use when building reproducible ML training workflows, orchestrating training jobs, or versioning training artifacts

- Skill: `kienbui1995/training-pipeline` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kienbui1995/training-pipeline`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kienbui1995/training-pipeline/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: kienbui1995 (https://skillmd.com/u/kienbui1995)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kienbui1995/training-pipeline

---


# Training Pipeline

## When to Use
When building the end-to-end system that takes raw data and produces a validated, registered model — reproducibly.

## Core Jobs

### 1. Define Pipeline Stages
```
Data → Validate → Preprocess → Train → Evaluate → Register
```
Each stage:
- Takes versioned inputs
- Produces versioned outputs
- Is independently re-runnable (idempotent)
- Has pass/fail criteria

### 2. Orchestrate the Pipeline
Tools: Airflow, Prefect, Kubeflow Pipelines, Metaflow, SageMaker Pipelines
- Each step = one container or task
- Pass artifacts between steps via shared storage (S3, GCS)
- Log metadata to experiment tracker at each step

### 3. Version Everything
- Data: DVC or dataset versioning in the registry
- Code: git commit hash logged with each run
- Environment: Docker image tag (pinned, not `latest`)
- Model: versioned in model registry with lineage to data + code

### 4. Automated Evaluation Gate
Before registering, automatically check:
- [ ] Performance > baseline (current prod model)
- [ ] Performance on recent data slice (last 30 days)
- [ ] No data leakage detected (temporal split verified)
- [ ] Latency within SLA
If any check fails: pipeline fails, Slack alert, no model registered.

### 5. CI/CD for ML
- PR → run unit tests on pipeline code
- Merge → trigger training run on subset (smoke test)
- Tag/release → full training run → deploy if gates pass

## Key Outputs
- Pipeline DAG definition
- Evaluation gate configuration
- Model registry entry (with lineage)
- Pipeline monitoring dashboard

## Anti-Patterns
- Manual steps in the pipeline (not reproducible)
- No evaluation gate — any model can go to prod
- `latest` tags on Docker images or data
- No lineage — can't trace a prod model back to its data