Asteria Fleet Data Quality Hub Reconciliation
Work through every fleet audit task using the same reusable pipeline: discover
the data landscape, load and reconcile records across overlapping sources,
assign internal control codes, and produce a JSON answer matching the supplied
answer template exactly.
Core Workflow
Execute these phases in order; each phase depends on the previous one.
Phase 1 — Read the case scope
Start by reading two files the task always provides:
case_scope.json — collection ID, cutoff, focus items, decision panel IDs, thresholds
answer_template.json — the output contract (JSON Schema or field-contract doc)
Extract exactly these items from the case scope and hold them in memory:
| Field |
Purpose |
collection_id |
Which collection to query |
cutoff_at / business_cutoff / as_of |
Temporal boundary for records |
business_period (if present) |
Additional time window for scoping |
focus_* arrays |
Which entities/assets/clusters need individual reports |
*_decision_ids arrays |
Which public IDs need control-code panels |
status_thresholds / certification_gate |
How to map quality metrics to PASS/HOLD |
status_action_map (if present) |
Explicit mapping; otherwise infer from standard rules |
*_ranking_limit |
How many ranked items to return |
Phase 2 — Discover the data landscape
Use the HTTP API at <TASK_ENV_BASE_URL>. The environment_access.md file
provides the base URL and authentication. All endpoints are read-only GET or
POST.
Execute these calls and keep the full responses for the rest of the task:
GET /api/catalog/collections — Find the target collection by
collection_id. Record its source_systems and any snapshot_policy.
GET /api/catalog/schema?collection_id=<id> — Learn field names, types,
and which fields represent timestamps, amounts, categories, and identifiers.
GET /api/source-snapshots?collection_id=<id> — List every snapshot. Each
snapshot has a snapshot_id, a status (CERTIFIED, PROVISIONAL, STALE), a
row_count, and a loaded_at timestamp.
Determine the authoritative snapshot:
- If exactly one snapshot has status CERTIFIED, it is authoritative.
- If multiple are CERTIFIED, pick the one with the latest
loaded_at.
- If no CERTIFIED exists, pick the PROVISIONAL with the latest
loaded_at.
- Record its
snapshot_id as authoritative_snapshot_id.
Phase 3 — Load all records
The primary data endpoint for a collection is a direct GET on
/api/<collection_kind>/<collection_id> or paginated via POST /api/query.
The task prompt and catalog response tell you which collection kind to use.
Pagination: Collections are often larger than a single response page. Use
POST /api/query for paginated retrieval:
{
"collection_kind": "<kind>",
"collection_id": "<id>",
"snapshot_ids": ["<snapshot_id>", ...],
"limit": 500,
"offset": 0
}
Loop with increasing offset until fewer than limit rows return or the
response is empty. Collect all rows into a single working set.
Always load all snapshots from /api/source-snapshots. You need the full
multi-snapshot picture to detect duplicates and assign source-basis codes.
Tag each row with its source snapshot_id as you accumulate.
Phase 4 — Reconcile records
4a. Row-level deduplication
When the same logical record (same primary ID) appears in multiple snapshots:
- Count every physical row as
raw_row_count.
- A logical record is a distinct primary ID (
transaction_id, event_id,
charge_id, or the row's own stable identifier like PAR-C*, FIE-C*).
logical_count = distinct primary IDs across all snapshots.
duplicate_raw_count = raw_row_count - logical_count.
For each logical record with duplicates, retain the occurrence from the
authoritative snapshot. When the task asks for retained_snapshot_id,
report the authoritative snapshot ID.
For contact-oriented tasks: the primary ID is the row identifier. Clustering
into canonical people uses the hub's cluster_id field — look for it in the
schema or /api/contacts response. A cluster with multiple member rows is a
duplicate cluster.
4b. Validation and quarantine
Validate every logical record against these rules:
- Missing or unparseable timestamp: Reject if the primary timestamp is null,
empty, or cannot be parsed as ISO-8601.
- Invalid timestamp: Reject if after the cutoff or outside the business period.
- Negative or zero quantity: Reject if a physical measure (volume_l,
weight_kg, distance_km, odometer_km, labor_hours) is null, zero, or negative.
- Invalid range: Reject if a value exceeds reasonable bounds (extreme_labor
check: labor_hours well beyond the expected distribution).
- Unresolvable category: A description/alias mapping to zero or more than
one canonical category.
Records failing validation go into the quarantine set. They are excluded
from normalized totals but still contribute to counts like raw_row_count,
logical_count, quarantine_rate, and exception rankings.
4c. Category/class recognition
For fuel tasks: Map free-text description to canonical fuel types. The answer
template lists valid types: BIODIESEL, DIESEL, ELECTRIC_CHARGE,
PREMIUM_UNLEADED, UNLEADED. A description with no match is unrecognized;
a description matching multiple types is ambiguous.
For freight tasks: Use GET /api/reference/aliases?collection_id=<id>. Each
alias_id maps to a canonical service_class (EXPRESS, HAZMAT, OVERSIZE,
REFRIGERATED, STANDARD). No mapping → unrecognized_alias. Multiple
mappings for the same alias → ambiguous_alias.
4d. Contact-specific reconciliation
For contact/roster tasks:
Survivor selection: Within each cluster, pick the survivor row by source
precedence. For partner onboarding: Compliance Master > Partner Portal > CRM.
For field service: HR Directory > Identity Registry > Dispatch. Within equal
source precedence, prefer the highest lexicographic row ID.
Canonical field selection: Apply source-system precedence per field. Use
the value from the highest-precedence source that has a non-null, valid
value for that field. Fall through to lower-precedence sources.
Quarantine for contacts: A row is quarantined when it has no usable email
AND no usable phone. "Usable" means non-null, non-empty, and well-formed.
Readiness: An entity is eligibility-eligible when ACTIVE AND has at least
one usable contact channel. Among eligible entities:
both = usable email AND usable phone
email_only = usable email only
phone_only = usable phone only
not_ready = active but no usable channel
- Inactive entities are excluded from readiness entirely.
4e. Odometer regression (maintenance only)
- Sort valid events by
(asset_id, event_timestamp).
- For each asset, track the last valid odometer. If a later event has a lower
odometer, flag it as a regression.
- Regression events stay in
valid_event_count but are reported separately in
regression_event_ids and regression_asset_ids.
- Corrected total distance: for each asset, compute
last_reliable_odometer - first_reliable_odometer, skipping regressed
readings.
Phase 5 — Compute normalized totals
For financial audits (fuel, freight):
- Apply unit conversions via
GET /api/reference/conversions to bring all
measures into the canonical unit (L, KG, KM).
- Apply FX rates via
GET /api/reference/fx to convert spend into the base
currency (always USD in training data).
- Sum across all valid, non-quarantined logical records.
- Group by recognized category/class and sum within each group.
- Round all numeric results to 2 decimal places unless the template specifies
otherwise.
Phase 6 — Focus-item reports
For every item in focus_assets, focus_clusters, focus_people, etc.,
produce the per-item breakdown required by the answer template:
- Count logical, valid, mismatch, quarantine, and exception records for that
item.
- Report canonical values for that focus item (survivor row, email, phone, city
for contacts; volume/spend for assets).
- Follow the exact field contract from the answer template.
Phase 7 — Assign control codes
The hub's internal code systems encode policy decisions about each record.
Allowed code values are always listed in the answer template's enum fields.
See the detailed reference at code_systems.md for
the complete decision tables. Here is the summary:
- Identity codes (IC-*): IC-25 (single-source, uncontested), IC-40
(quarantined), IC-70 (multi-source field-merged), IC-90 (multi-source
corroborated).
- Outreach codes (OR-*): OR-15 (inactive), OR-35 (dispatchable), OR-60
(no usable contact), OR-80 (consent-blocked).
- Field-provenance codes (FP-*): FP-20 (single-source), FP-55 (multi-source
merged), FP-75 (quarantined).
- Source-basis codes (SB-*): SB-24 (certified only), SB-61 (both, certified
retained), SB-79 (provisional only).
- Ledger-disposition codes (LD-*): LD-14 (unrecognized category), LD-31
(valid, category mismatch), LD-53 (matched, certified-only), LD-72 (matched,
provisional-only), LD-88 (ambiguous category).
- Reference-policy codes (RB-*): RB-17 (clean one-to-one alias mapping),
RB-42 (contested alias mapping, recognized but non-straightforward), RB-83
(unrecognized alias, no reference entry).
- Maintenance-source codes (MS-*): MS-12 (provisional only), MS-47
(certified only), MS-86 (multi-snapshot).
- History-route codes (HR-*): HR-19 (odometer regression), HR-33 (clean,
single-source), HR-74 (clean, multi-source).
Code assignment priority: Quarantine overrides all. Mismatch/unrecognized
category overrides source-basis. Source multiplicity determines source-basis
before category-match determines ledger-disposition.
Phase 8 — Certification decision
Map the computed quality metrics to a certification status using the thresholds
from the case scope.
Standard thresholds (when status_thresholds is present):
quarantine_rate == 0.0 → PASS → RELEASE
0.0 < quarantine_rate <= pass_with_exceptions_max → PASS_WITH_EXCEPTIONS → REVIEW_EXCEPTIONS
quarantine_rate > pass_with_exceptions_max → HOLD → BLOCK_AND_REMEDIATE
Gate-based thresholds (when certification_gate is present):
The gate specifies a single condition and consequence. For example, if
odometer_regression_status: HOLD, any odometer regression triggers HOLD.
If the case scope provides an explicit status_action_map, use it. Otherwise
use the standard mapping:
PASS → RELEASE
PASS_WITH_EXCEPTIONS → REVIEW_EXCEPTIONS
HOLD → BLOCK_AND_REMEDIATE
Match the answer template's exact field names for the certification object.
Different tasks use different names: action (fuel), routing (freight),
next_action (contacts). Copy the template's names verbatim.
Phase 9 — Produce the JSON answer
Re-read the answer template. Note every required field, its type, any enum
constraints, minItems/maxItems, pattern validations, and ordering rules
(often in x-ordering_rules or description text).
Ordering rules (apply to every list unless contradicted by the template):
- String ID lists: lexicographically ascending.
- Ranked lists: by the specified sort keys.
- Region/depot lists: lexicographically ascending by region/depot code.
- Focus items: by their ID, ascending.
- Decision panels: by the decision ID, ascending.
Numeric precision:
- Counts: exact integers.
- Rates: round to the decimal places specified in the schema (typically 4 for
rates, 2 for amounts). Round half-up.
Produce a single JSON object. Do not wrap in Markdown. Do not include
commentary. Use the exact field names from the answer template.
Reference files
- api_endpoints.md — Full API reference with
request and response shapes
- code_systems.md — Complete code assignment
tables with decision trees
Scripts
- query.py — Paginated query helper for
/api/query
1---2name: asteria-fleet-dq-hub-23description: Reconcile and certify fleet data-quality audits using the Asteria Fleet Data Quality Hub. Use this skill whenever the user mentions Asteria, Fleet Data Quality Hub, data reconciliation, fleet audits, cargo/fuel/freight/maintenance/contact/roster certification, partner onboarding, carrier accrual, fuel ledger, field-service roster, or any task that references an Asteria API with collections, snapshots, and an answer template. Even if the task is phrased generically, use this skill when the user points at a read-only API with catalog, schema, source-snapshots, and query endpoints.4metadata:5 source: https://skillmd.com/skills/prism-shadow/asteria-fleet-dq-hub-26---78# Asteria Fleet Data Quality Hub Reconciliation910Work through every fleet audit task using the same reusable pipeline: discover11the data landscape, load and reconcile records across overlapping sources,12assign internal control codes, and produce a JSON answer matching the supplied13answer template exactly.1415## Core Workflow1617Execute these phases in order; each phase depends on the previous one.1819### Phase 1 — Read the case scope2021Start by reading two files the task always provides:22- `case_scope.json` — collection ID, cutoff, focus items, decision panel IDs, thresholds23- `answer_template.json` — the output contract (JSON Schema or field-contract doc)2425Extract exactly these items from the case scope and hold them in memory:2627| Field | Purpose |28|---|---|29| `collection_id` | Which collection to query |30| `cutoff_at` / `business_cutoff` / `as_of` | Temporal boundary for records |31| `business_period` (if present) | Additional time window for scoping |32| `focus_*` arrays | Which entities/assets/clusters need individual reports |33| `*_decision_ids` arrays | Which public IDs need control-code panels |34| `status_thresholds` / `certification_gate` | How to map quality metrics to PASS/HOLD |35| `status_action_map` (if present) | Explicit mapping; otherwise infer from standard rules |36| `*_ranking_limit` | How many ranked items to return |3738### Phase 2 — Discover the data landscape3940Use the HTTP API at `<TASK_ENV_BASE_URL>`. The `environment_access.md` file41provides the base URL and authentication. All endpoints are read-only GET or42POST.4344Execute these calls and keep the full responses for the rest of the task:45461. `GET /api/catalog/collections` — Find the target collection by47 `collection_id`. Record its `source_systems` and any `snapshot_policy`.48492. `GET /api/catalog/schema?collection_id=<id>` — Learn field names, types,50 and which fields represent timestamps, amounts, categories, and identifiers.51523. `GET /api/source-snapshots?collection_id=<id>` — List every snapshot. Each53 snapshot has a `snapshot_id`, a `status` (CERTIFIED, PROVISIONAL, STALE), a54 `row_count`, and a `loaded_at` timestamp.55564. Determine the **authoritative snapshot**:57 - If exactly one snapshot has status CERTIFIED, it is authoritative.58 - If multiple are CERTIFIED, pick the one with the latest `loaded_at`.59 - If no CERTIFIED exists, pick the PROVISIONAL with the latest `loaded_at`.60 - Record its `snapshot_id` as `authoritative_snapshot_id`.6162### Phase 3 — Load all records6364The primary data endpoint for a collection is a direct GET on65`/api/<collection_kind>/<collection_id>` or paginated via `POST /api/query`.66The task prompt and catalog response tell you which collection kind to use.6768**Pagination**: Collections are often larger than a single response page. Use69`POST /api/query` for paginated retrieval:7071```json72{73 "collection_kind": "<kind>",74 "collection_id": "<id>",75 "snapshot_ids": ["<snapshot_id>", ...],76 "limit": 500,77 "offset": 078}79```8081Loop with increasing `offset` until fewer than `limit` rows return or the82response is empty. Collect all rows into a single working set.8384**Always load all snapshots** from `/api/source-snapshots`. You need the full85multi-snapshot picture to detect duplicates and assign source-basis codes.86Tag each row with its source `snapshot_id` as you accumulate.8788### Phase 4 — Reconcile records8990#### 4a. Row-level deduplication9192When the same logical record (same primary ID) appears in multiple snapshots:9394- Count every physical row as `raw_row_count`.95- A logical record is a distinct primary ID (`transaction_id`, `event_id`,96 `charge_id`, or the row's own stable identifier like `PAR-C*`, `FIE-C*`).97- `logical_count` = distinct primary IDs across all snapshots.98- `duplicate_raw_count` = `raw_row_count` - `logical_count`.99100For each logical record with duplicates, retain the occurrence from the101**authoritative snapshot**. When the task asks for `retained_snapshot_id`,102report the authoritative snapshot ID.103104For contact-oriented tasks: the primary ID is the row identifier. Clustering105into canonical people uses the hub's `cluster_id` field — look for it in the106schema or `/api/contacts` response. A cluster with multiple member rows is a107**duplicate cluster**.108109#### 4b. Validation and quarantine110111Validate every logical record against these rules:112113- **Missing or unparseable timestamp**: Reject if the primary timestamp is null,114 empty, or cannot be parsed as ISO-8601.115- **Invalid timestamp**: Reject if after the cutoff or outside the business period.116- **Negative or zero quantity**: Reject if a physical measure (volume_l,117 weight_kg, distance_km, odometer_km, labor_hours) is null, zero, or negative.118- **Invalid range**: Reject if a value exceeds reasonable bounds (extreme_labor119 check: labor_hours well beyond the expected distribution).120- **Unresolvable category**: A description/alias mapping to zero or more than121 one canonical category.122123Records failing validation go into the **quarantine set**. They are excluded124from normalized totals but still contribute to counts like `raw_row_count`,125`logical_count`, `quarantine_rate`, and exception rankings.126127#### 4c. Category/class recognition128129For fuel tasks: Map free-text `description` to canonical fuel types. The answer130template lists valid types: `BIODIESEL`, `DIESEL`, `ELECTRIC_CHARGE`,131`PREMIUM_UNLEADED`, `UNLEADED`. A description with no match is `unrecognized`;132a description matching multiple types is `ambiguous`.133134For freight tasks: Use `GET /api/reference/aliases?collection_id=<id>`. Each135`alias_id` maps to a canonical `service_class` (`EXPRESS`, `HAZMAT`, `OVERSIZE`,136`REFRIGERATED`, `STANDARD`). No mapping → `unrecognized_alias`. Multiple137mappings for the same alias → `ambiguous_alias`.138139#### 4d. Contact-specific reconciliation140141For contact/roster tasks:1421431. **Survivor selection**: Within each cluster, pick the survivor row by source144 precedence. For partner onboarding: Compliance Master > Partner Portal > CRM.145 For field service: HR Directory > Identity Registry > Dispatch. Within equal146 source precedence, prefer the highest lexicographic row ID.1471482. **Canonical field selection**: Apply source-system precedence per field. Use149 the value from the highest-precedence source that has a non-null, valid150 value for that field. Fall through to lower-precedence sources.1511523. **Quarantine for contacts**: A row is quarantined when it has no usable email153 AND no usable phone. "Usable" means non-null, non-empty, and well-formed.1541554. **Readiness**: An entity is eligibility-eligible when ACTIVE AND has at least156 one usable contact channel. Among eligible entities:157 - `both` = usable email AND usable phone158 - `email_only` = usable email only159 - `phone_only` = usable phone only160 - `not_ready` = active but no usable channel161 - Inactive entities are excluded from readiness entirely.162163#### 4e. Odometer regression (maintenance only)1641651. Sort valid events by `(asset_id, event_timestamp)`.1662. For each asset, track the last valid odometer. If a later event has a lower167 odometer, flag it as a regression.1683. Regression events stay in `valid_event_count` but are reported separately in169 `regression_event_ids` and `regression_asset_ids`.1704. Corrected total distance: for each asset, compute171 `last_reliable_odometer - first_reliable_odometer`, skipping regressed172 readings.173174### Phase 5 — Compute normalized totals175176For financial audits (fuel, freight):1771781. Apply unit conversions via `GET /api/reference/conversions` to bring all179 measures into the canonical unit (L, KG, KM).1802. Apply FX rates via `GET /api/reference/fx` to convert spend into the base181 currency (always USD in training data).1823. Sum across all valid, non-quarantined logical records.1834. Group by recognized category/class and sum within each group.1845. Round all numeric results to 2 decimal places unless the template specifies185 otherwise.186187### Phase 6 — Focus-item reports188189For every item in `focus_assets`, `focus_clusters`, `focus_people`, etc.,190produce the per-item breakdown required by the answer template:191192- Count logical, valid, mismatch, quarantine, and exception records for that193 item.194- Report canonical values for that focus item (survivor row, email, phone, city195 for contacts; volume/spend for assets).196- Follow the exact field contract from the answer template.197198### Phase 7 — Assign control codes199200The hub's internal code systems encode policy decisions about each record.201Allowed code values are always listed in the answer template's enum fields.202203See the detailed reference at [code_systems.md](references/code_systems.md) for204the complete decision tables. Here is the summary:205206- **Identity codes (IC-*)**: IC-25 (single-source, uncontested), IC-40207 (quarantined), IC-70 (multi-source field-merged), IC-90 (multi-source208 corroborated).209- **Outreach codes (OR-*)**: OR-15 (inactive), OR-35 (dispatchable), OR-60210 (no usable contact), OR-80 (consent-blocked).211- **Field-provenance codes (FP-*)**: FP-20 (single-source), FP-55 (multi-source212 merged), FP-75 (quarantined).213- **Source-basis codes (SB-*)**: SB-24 (certified only), SB-61 (both, certified214 retained), SB-79 (provisional only).215- **Ledger-disposition codes (LD-*)**: LD-14 (unrecognized category), LD-31216 (valid, category mismatch), LD-53 (matched, certified-only), LD-72 (matched,217 provisional-only), LD-88 (ambiguous category).218- **Reference-policy codes (RB-*)**: RB-17 (clean one-to-one alias mapping),219 RB-42 (contested alias mapping, recognized but non-straightforward), RB-83220 (unrecognized alias, no reference entry).221- **Maintenance-source codes (MS-*)**: MS-12 (provisional only), MS-47222 (certified only), MS-86 (multi-snapshot).223- **History-route codes (HR-*)**: HR-19 (odometer regression), HR-33 (clean,224 single-source), HR-74 (clean, multi-source).225226**Code assignment priority**: Quarantine overrides all. Mismatch/unrecognized227category overrides source-basis. Source multiplicity determines source-basis228before category-match determines ledger-disposition.229230### Phase 8 — Certification decision231232Map the computed quality metrics to a certification status using the thresholds233from the case scope.234235**Standard thresholds** (when `status_thresholds` is present):236```237quarantine_rate == 0.0 → PASS → RELEASE2380.0 < quarantine_rate <= pass_with_exceptions_max → PASS_WITH_EXCEPTIONS → REVIEW_EXCEPTIONS239quarantine_rate > pass_with_exceptions_max → HOLD → BLOCK_AND_REMEDIATE240```241242**Gate-based thresholds** (when `certification_gate` is present):243The gate specifies a single condition and consequence. For example, if244`odometer_regression_status: HOLD`, any odometer regression triggers HOLD.245246If the case scope provides an explicit `status_action_map`, use it. Otherwise247use the standard mapping:248- `PASS` → `RELEASE`249- `PASS_WITH_EXCEPTIONS` → `REVIEW_EXCEPTIONS`250- `HOLD` → `BLOCK_AND_REMEDIATE`251252Match the answer template's exact field names for the certification object.253Different tasks use different names: `action` (fuel), `routing` (freight),254`next_action` (contacts). Copy the template's names verbatim.255256### Phase 9 — Produce the JSON answer2572581. Re-read the answer template. Note every required field, its type, any `enum`259 constraints, `minItems`/`maxItems`, `pattern` validations, and ordering rules260 (often in `x-ordering_rules` or `description` text).2612622. **Ordering rules** (apply to every list unless contradicted by the template):263 - String ID lists: lexicographically ascending.264 - Ranked lists: by the specified sort keys.265 - Region/depot lists: lexicographically ascending by region/depot code.266 - Focus items: by their ID, ascending.267 - Decision panels: by the decision ID, ascending.2682693. **Numeric precision**:270 - Counts: exact integers.271 - Rates: round to the decimal places specified in the schema (typically 4 for272 rates, 2 for amounts). Round half-up.2732744. Produce a single JSON object. Do not wrap in Markdown. Do not include275 commentary. Use the exact field names from the answer template.276277## Reference files278279- [api_endpoints.md](references/api_endpoints.md) — Full API reference with280 request and response shapes281- [code_systems.md](references/code_systems.md) — Complete code assignment282 tables with decision trees283284## Scripts285286- [query.py](scripts/query.py) — Paginated query helper for `/api/query`