timescale Validate Pipeline
End-to-end validation of the UNS pipeline. Queries TimescaleDB to verify that
readings collected by the clients landed correctly in tag_history.
Usage
/validate-pipeline— query TimescaleDB and report on the data that landed
Prerequisites
Before running, ensure:
- Dependencies installed:
uv sync - The schema is applied and
uns_namespaceis seeded - The collection clients have run, so
tag_historyholds data
To collect without real sources, start the simulator first (a separate
script). If tag_history is empty, the validation script warns and exits.
Steps
- Run
uv run scripts/validate_pipeline.py(no arguments — it validates whatever data is already intag_history) - Interpret the 6 checks:
| Check | What it validates | Failure means |
|---|---|---|
| Tag coverage | Every tag in tag_database.csv has rows | Client dropped a tag |
| Timestamp sanity | Timestamps are recent, UTC, not future | Timezone bug or clock skew |
| Foreign key integrity | All tag_ids resolve to uns_namespace | Client wrote an unresolved tag |
| Dead letter | No rejected readings | Client emitting invalid data |
| Value plausibility | Booleans are 0/1, temps in range | Scale factor or type error |
| Cross-area drift | Areas have aligned timestamps | Naive-timestamp leak |
- Report the client health summary (rows per area, latest timestamp)
- For any FAIL: cross-reference client code and config to diagnose root cause
Report Format
## Pipeline Validation Report
### Checks
- Tag coverage: {N}/{total} — {PASS|FAIL}
- Timestamp sanity: {PASS|FAIL}
- Foreign key integrity: {PASS|FAIL}
- Dead letter: {N} rejected — {PASS|FAIL}
- Value plausibility: {PASS|FAIL}
- Cross-area drift: {PASS|FAIL}
### Client Health
- area_01: {N} tags, {M} rows
- area_02: {N} tags, {M} rows
### Diagnosis
{Root cause analysis for any failures}