Contract
- Input: data assets, reproducibility requirements, collaboration model.
- Output: versioning strategy + branching workflow + reproducibility plan.
- Scope: designs versioning; execution requires approval.
- Rule: designs versioning; execution requires approval.
- Rule: documented standards override defaults; explicit project rules take precedence.
- Rule: if blocked by missing context or dependencies, surface the blocker before proceeding.
Provenance
| Question |
Answer |
| What is the source of truth? |
The user's request, originating spec/issue, and the skill's declared outputs |
| What is in scope? |
Work covered by the skill's acceptance criteria and completion rules |
| What is explicitly out of scope? |
Files, behaviors, and decisions outside the skill's declared boundary |
| Who or what consumes this artifact afterward? |
The next skill in the workflow or the user |
| What evidence proves it is done? |
Completion criteria met, artifact saved, validation passed |
| What risk remains? |
Subjective judgment calls, missing context, or external dependency failures |
Artifact
Emit DataVersioningArtifact as both:
- JSON:
.agents/skills/platform/artifacts/data-versioning/{identifier}.json
- Markdown view: same filename with
.md extension
Data Versioning
Design data versioning — DVC, lakeFS, delta tables, snapshotting, and reproducibility.
Process
1. Assess versioning needs
- Datasets: raw, processed, feature, model.
- Frequency of change: static, nightly, streaming.
- Team collaboration: single team, cross-team, external.
- Compliance: audit trail, retention, immutability.
Completion criterion: requirements and constraints saved.
2. Tool selection
- DVC: Git-like versioning for files and ML pipelines; works with any storage.
- lakeFS: open-source Git-like lakehouse; branch/commit/merge for data lakes.
- Delta Lake: ACID transactions, time travel, schema enforcement on lakes.
- Iceberg / Hudi: alternative table formats with versioning.
Completion criterion: tool selection saved with justification.
3. Versioning strategy
- Granularity: file-level, table-level, snapshot-level.
- Naming: semantic versioning, timestamped, hash-based.
- Metadata: commit messages, author, timestamp, diff summary.
- Storage: object store (S3, GCS, ADLS) with versioning enabled.
Completion criterion: versioning strategy saved.
4. Branching and merging
- Branching: dev / staging / prod; experiment branches for ML.
- Merging: conflict resolution, merge commits, cherry-pick.
- CI/CD: validate branches before merge (schema, quality, tests).
Completion criterion: branching workflow saved.
5. Snapshotting and reproducibility
- Snapshots: point-in-time captures of datasets and code.
- Provenance: link data version to code version, config, environment.
- Rollback: revert to previous snapshot with validation.
Completion criterion: snapshot and rollback plan saved.
6. Integration with ML pipelines
- Experiment tracking: data version → model version → metrics.
- Feature reproducibility: pin feature dataset version.
- Deployment: promote approved data versions to production.
Completion criterion: ML integration design saved.
Completion
- the skill's completion criteria are explicitly checked
- the artifact is saved and validated
- any blockers or skipped validations are documented
- the next consumer is identified or the work is handed off
@include .agents/skills/platform/contract-base.xml
1---2name: "data-versioning"3category: "data"4maturity: "stable"5version: "1"6description: "Data versioning (DVC, lakeFS, delta tables, snapshotting, reproducibility)."7license: "MIT"8capabilities:9 - apply data versioning methods within explicit data boundaries10 - analyze data versioning (dvc, lakefs, delta tables, snapshotting, reproducibility) with assumptions, constraints, and failure modes11 - produce and validate a data artifact with evidence, decisions, and next steps12outputs: "versioning strategy + branching workflow + reproducibility plan."13sideEffects: []14dependencies: []15stopCondition: "Versioning strategy saved; branching workflow defined; reproducibility plan documented."16risk: "medium"17trustTier: "3"18maxIterations: "6"19promptVersion: "2.0"20artifactType: "data"21modelTier: "reasoning"22evaluators: ["behavioral", "regression"]23fixturesPath: .agents/skills/platform/fixtures/behavioral/data-versioning.json24diataxis: "how-to"25tags: ["data"]26compatibility: []27approvalRequired: "true"28approvalFor: []29metadata:30 source: https://skillmd.com/skills/quantumquirkxyz/data-versioning31---3233## Contract3435- Input: data assets, reproducibility requirements, collaboration model.36- Output: versioning strategy + branching workflow + reproducibility plan.37- Scope: designs versioning; execution requires approval.38- Rule: designs versioning; execution requires approval.39- Rule: documented standards override defaults; explicit project rules take precedence.40- Rule: if blocked by missing context or dependencies, surface the blocker before proceeding.4142## Provenance4344| Question | Answer |45|---|---|46| What is the source of truth? | The user's request, originating spec/issue, and the skill's declared outputs |47| What is in scope? | Work covered by the skill's acceptance criteria and completion rules |48| What is explicitly out of scope? | Files, behaviors, and decisions outside the skill's declared boundary |49| Who or what consumes this artifact afterward? | The next skill in the workflow or the user |50| What evidence proves it is done? | Completion criteria met, artifact saved, validation passed |51| What risk remains? | Subjective judgment calls, missing context, or external dependency failures525354## Artifact5556Emit `DataVersioningArtifact` as both:57- JSON: `.agents/skills/platform/artifacts/data-versioning/{identifier}.json`58- Markdown view: same filename with `.md` extension596061# 62# 63# Data Versioning6465Design **data versioning** — DVC, lakeFS, delta tables, snapshotting, and reproducibility.6667## Process6869### 1. Assess versioning needs70- Datasets: raw, processed, feature, model.71- Frequency of change: static, nightly, streaming.72- Team collaboration: single team, cross-team, external.73- Compliance: audit trail, retention, immutability.7475**Completion criterion:** requirements and constraints saved.7677### 2. Tool selection78- DVC: Git-like versioning for files and ML pipelines; works with any storage.79- lakeFS: open-source Git-like lakehouse; branch/commit/merge for data lakes.80- Delta Lake: ACID transactions, time travel, schema enforcement on lakes.81- Iceberg / Hudi: alternative table formats with versioning.8283**Completion criterion:** tool selection saved with justification.8485### 3. Versioning strategy86- Granularity: file-level, table-level, snapshot-level.87- Naming: semantic versioning, timestamped, hash-based.88- Metadata: commit messages, author, timestamp, diff summary.89- Storage: object store (S3, GCS, ADLS) with versioning enabled.9091**Completion criterion:** versioning strategy saved.9293### 4. Branching and merging94- Branching: dev / staging / prod; experiment branches for ML.95- Merging: conflict resolution, merge commits, cherry-pick.96- CI/CD: validate branches before merge (schema, quality, tests).9798**Completion criterion:** branching workflow saved.99100### 5. Snapshotting and reproducibility101- Snapshots: point-in-time captures of datasets and code.102- Provenance: link data version to code version, config, environment.103- Rollback: revert to previous snapshot with validation.104105**Completion criterion:** snapshot and rollback plan saved.106107### 6. Integration with ML pipelines108- Experiment tracking: data version → model version → metrics.109- Feature reproducibility: pin feature dataset version.110- Deployment: promote approved data versions to production.111112**Completion criterion:** ML integration design saved.113114## Completion115116- the skill's completion criteria are explicitly checked117- the artifact is saved and validated118- any blockers or skipped validations are documented119- the next consumer is identified or the work is handed off120---121@include .agents/skills/platform/contract-base.xml