Data Quality

Data Quality

The expensive failure is not the job that crashes. It is the run that succeeds and writes plausible garbage, which is then queried, charted, and acted on. Every check here answers one question: if this broke silently, what would tell us?

Standing reference: references/data-quality-checklist.md.

Process

1. Start with freshness and volume. They are cheap and they catch most real incidents. Max timestamp against a stated SLA, row count against a trailing median band. Add them before anything more sophisticated, even to a table that has no other tests.

2. Alert on data, not on task exit. A green pipeline that produced zero rows is the failure that costs a week, and task-failure alerting is blind to it by construction.

3. Test the grain. Count against count-distinct on the declared key. If the table has no declared key, that is the first finding, not a reason to stop testing.

4. Then completeness and validity. Null rate on the columns people filter by, accepted values on enums, ranges on numerics, referential integrity on join keys.

5. Reconcile against an independent source. The upstream total, the billing system, a hand count for one day. Two systems agreeing is worth more than any single check.

6. Place tests at ownership boundaries. Where data enters your ownership and where it leaves it. Tests on every intermediate step become noise, and noise trains people to ignore the channel that matters.

7. Prove each check fails. Feed it deliberately bad data: hold the pipeline back, drop rows, inject a duplicate, and watch the check go red. A quality check never seen red is decoration.

8. Give every check an owner and a threshold in the alert text. Which table, which check, observed value, threshold. An alert that says "data quality issue" gets muted.

Triage: a number looks wrong

Work in this order. Most wrong numbers resolve in the first two steps.

1. Is the source table fresh for the period? A pipeline gap looks exactly like a real drop.
2. Is row volume for the period in the normal band? A metric that fell in step with row count is an incident, not a business event.
3. Did the definition change? Check the model, the dashboard filter, the deploy history.
4. Did the grain change? Count against count-distinct on the key for that period.
5. Did the upstream change? A new enum value, a renamed column, a source-side backfill.
6. Only then look for a real change in behaviour.

Output

## Quality: <table or model>

- **Freshness:** SLA <target>, observed <value>, alert <name> to <owner>
- **Volume:** band <range> from trailing <n> periods, observed <value>
- **Uniqueness:** key <cols>, count <n> vs distinct <n>
- **Completeness:** <col> null rate <n>% (threshold <n>%)
- **Validity:** <accepted values / ranges / referential checks>
- **Reconciliation:** <this> <n> vs <independent source> <n>, delta <n>
- **Negative test:** <what was corrupted>, check <name> went red — output quoted

Verification

• Every check was run against corrupted input and failed as designed; quote one.
• Freshness alerting was triggered by withholding data, not assumed to work.
• Each check names an owner expected to act, not only a channel.
• Checks that fire routinely were fixed or deleted before this was called done.

Red flags