Contract
- Input: data inventory, producer/consumer map, SLAs, freshness requirements.
- Output: data quality framework + check suite + observability + data contract.
- Scope: designs quality framework; does not execute data pipelines or modify production data unless explicitly instructed.
- Rule: designs quality framework; does not execute data pipelines or modify production data unless explicitly instructed.
- Rule: documented standards override defaults; explicit project rules take precedence.
- Rule: if blocked by missing context or dependencies, surface the blocker before proceeding.
Provenance
| Question |
Answer |
| What is the source of truth? |
The user's request, originating spec/issue, and the skill's declared outputs |
| What is in scope? |
Work covered by the skill's acceptance criteria and completion rules |
| What is explicitly out of scope? |
Files, behaviors, and decisions outside the skill's declared boundary |
| Who or what consumes this artifact afterward? |
The next skill in the workflow or the user |
| What evidence proves it is done? |
Completion criteria met, artifact saved, validation passed |
| What risk remains? |
Subjective judgment calls, missing context, or external dependency failures |
Artifact
Emit DataQualityArtifact as both:
- JSON:
.agents/skills/platform/artifacts/data-quality/{identifier}.json
- Markdown view: same filename with
.md extension
Data Quality
Design a data quality framework — Great Expectations, Soda, checks, observability, contracts — with explicit SLAs, freshness rules, and incident response for bad data.
Process
1. Inventory and ownership
- Data assets (tables, streams, files, APIs) with owners.
- Producer/consumer map (who writes, who reads, downstream dependencies).
- Criticality tiers (P0: financial / safety; P1: operational; P2: analytical).
- SLA definitions (freshness, completeness, accuracy).
Completion criterion: data inventory with owners and criticality tiers documented.
2. Quality dimensions
- Freshness (staleness threshold, lag alerts).
- Volume (row count, file size, expected range).
- Distribution (value ranges, null percentages, duplicates, cardinality).
- Referential integrity (foreign keys, joins, orphaned records).
- Uniqueness (primary keys, business keys).
- Validity (schema conformance, type checks, regex patterns).
Completion criterion: quality dimensions mapped to assets with thresholds.
3. Check suite design
- Expectation types (row count, column null %, unique, value set, regex, cross-column).
- Thresholds and tolerances (hard fail, warning, drift detection).
- Anomaly detection (seasonality-aware, change point detection).
- Check placement (ingestion, transformation, warehouse load, API response).
Completion criterion: check suite with thresholds and placement defined.
4. Tooling and implementation
- Tool selection (Great Expectations, Soda, Monte Carlo, custom SQL).
- Integration points (Airflow, dbt, Spark, Kafka, CI/CD).
- Check execution (scheduled, event-driven, on-demand).
- Result storage and versioning (expectation suite as code).
Completion criterion: tool selection and integration plan defined.
5. Observability and alerting
- Metrics (pass rate, failure count, time-to-detect, time-to-resolve).
- Dashboards (per-table health, SLA compliance, trend over time).
- Alert routing (on-call, data owner, platform team).
- Incident response (runbook, severity, escalation for data incidents).
Completion criterion: observability dashboard and alerting rules defined.
6. Data contracts
- Schema and semantics (column names, types, units, allowed values).
- Producer guarantees (freshness, completeness, format version).
- Consumer obligations (handling nulls, handling schema changes, deprecation notice).
- Versioning and breaking change policy.
Completion criterion: data contract specification with versioning policy defined.
Rules
- Rule: treat data quality as a product feature, not an afterthought; assign owners.
- Rule: define SLAs in terms of business impact, not just technical thresholds.
- Rule: run checks as close to the source as possible; fail fast and loud.
- Rule: version expectation suites alongside data models and pipeline code.
- Rule: require data contracts for any cross-team or cross-service data exchange.
Completion
- the skill's completion criteria are explicitly checked
- the artifact is saved and validated
- any blockers or skipped validations are documented
- the next consumer is identified or the work is handed off
@include .agents/skills/platform/contract-base.xml
1---2name: "data-quality"3category: "data"4maturity: "stable"5version: "1"6description: "Design data quality frameworks — Great Expectations, Soda, checks, observability, contracts — with explicit SLAs, freshness rules, and incident response for bad data."7license: "MIT"8capabilities:9 - apply data quality methods within explicit data boundaries10 - analyze design data quality frameworks — great expectations, soda, checks, observability, contracts — with explicit slas, freshness rules, and incident response for bad data with assumptions, constraints, and failure modes11 - produce and validate a data artifact with evidence, decisions, and next steps12outputs: "data quality framework + check suite + observability + data contract."13sideEffects: []14dependencies: []15stopCondition: "Framework defined; check suite configured; observability and contract rules documented."16risk: "medium"17trustTier: "3"18maxIterations: "6"19promptVersion: "2.0"20artifactType: "data"21modelTier: "reasoning"22evaluators: ["behavioral", "regression"]23fixturesPath: .agents/skills/platform/fixtures/behavioral/data-quality.json24diataxis: "how-to"25tags: ["data"]26compatibility: []27approvalRequired: "true"28approvalFor: []29metadata:30 source: https://skillmd.com/skills/quantumquirkxyz/data-quality31---3233## Contract3435- Input: data inventory, producer/consumer map, SLAs, freshness requirements.36- Output: data quality framework + check suite + observability + data contract.37- Scope: designs quality framework; does not execute data pipelines or modify production data unless explicitly instructed.38- Rule: designs quality framework; does not execute data pipelines or modify production data unless explicitly instructed.39- Rule: documented standards override defaults; explicit project rules take precedence.40- Rule: if blocked by missing context or dependencies, surface the blocker before proceeding.4142## Provenance4344| Question | Answer |45|---|---|46| What is the source of truth? | The user's request, originating spec/issue, and the skill's declared outputs |47| What is in scope? | Work covered by the skill's acceptance criteria and completion rules |48| What is explicitly out of scope? | Files, behaviors, and decisions outside the skill's declared boundary |49| Who or what consumes this artifact afterward? | The next skill in the workflow or the user |50| What evidence proves it is done? | Completion criteria met, artifact saved, validation passed |51| What risk remains? | Subjective judgment calls, missing context, or external dependency failures525354## Artifact5556Emit `DataQualityArtifact` as both:57- JSON: `.agents/skills/platform/artifacts/data-quality/{identifier}.json`58- Markdown view: same filename with `.md` extension596061# 62# 63# Data Quality6465Design a **data quality framework** — Great Expectations, Soda, checks, observability, contracts — with explicit SLAs, freshness rules, and incident response for bad data.6667## Process6869### 1. Inventory and ownership70- Data assets (tables, streams, files, APIs) with owners.71- Producer/consumer map (who writes, who reads, downstream dependencies).72- Criticality tiers (P0: financial / safety; P1: operational; P2: analytical).73- SLA definitions (freshness, completeness, accuracy).7475**Completion criterion:** data inventory with owners and criticality tiers documented.7677### 2. Quality dimensions78- Freshness (staleness threshold, lag alerts).79- Volume (row count, file size, expected range).80- Distribution (value ranges, null percentages, duplicates, cardinality).81- Referential integrity (foreign keys, joins, orphaned records).82- Uniqueness (primary keys, business keys).83- Validity (schema conformance, type checks, regex patterns).8485**Completion criterion:** quality dimensions mapped to assets with thresholds.8687### 3. Check suite design88- Expectation types (row count, column null %, unique, value set, regex, cross-column).89- Thresholds and tolerances (hard fail, warning, drift detection).90- Anomaly detection (seasonality-aware, change point detection).91- Check placement (ingestion, transformation, warehouse load, API response).9293**Completion criterion:** check suite with thresholds and placement defined.9495### 4. Tooling and implementation96- Tool selection (Great Expectations, Soda, Monte Carlo, custom SQL).97- Integration points (Airflow, dbt, Spark, Kafka, CI/CD).98- Check execution (scheduled, event-driven, on-demand).99- Result storage and versioning (expectation suite as code).100101**Completion criterion:** tool selection and integration plan defined.102103### 5. Observability and alerting104- Metrics (pass rate, failure count, time-to-detect, time-to-resolve).105- Dashboards (per-table health, SLA compliance, trend over time).106- Alert routing (on-call, data owner, platform team).107- Incident response (runbook, severity, escalation for data incidents).108109**Completion criterion:** observability dashboard and alerting rules defined.110111### 6. Data contracts112- Schema and semantics (column names, types, units, allowed values).113- Producer guarantees (freshness, completeness, format version).114- Consumer obligations (handling nulls, handling schema changes, deprecation notice).115- Versioning and breaking change policy.116117**Completion criterion:** data contract specification with versioning policy defined.118119## Rules120121- Rule: treat data quality as a product feature, not an afterthought; assign owners.122- Rule: define SLAs in terms of business impact, not just technical thresholds.123- Rule: run checks as close to the source as possible; fail fast and loud.124- Rule: version expectation suites alongside data models and pipeline code.125- Rule: require data contracts for any cross-team or cross-service data exchange.126127## Completion128129- the skill's completion criteria are explicitly checked130- the artifact is saved and validated131- any blockers or skipped validations are documented132- the next consumer is identified or the work is handed off133---134@include .agents/skills/platform/contract-base.xml