# Verification Sweep

> Post-orchestration verification sweep — runs after epic-team or ticket-pipeline completes to verify dashboard endpoints return HTTP 200 with real data, database tables contain expected rows, and dod_evidence rendered_output items have passing receipts. Non-blocking (flags only, does not halt orchestration).

- Skill: `omninode-ai/verification-sweep` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add omninode-ai/verification-sweep`
- Raw SKILL.md: https://api.skillmd.com/api/skills/omninode-ai/verification-sweep/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: OmniNode-ai (https://skillmd.com/u/omninode-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/omninode-ai/verification-sweep

---


# Verification Sweep

**Skill ID**: `onex:verification_sweep`
**Version**: 1.0.0
**Owner**: omniclaude
---

## Purpose

Post-orchestration verification sweep that runs as the final step after `epic-team` or
`ticket-pipeline` completes. It verifies that the work actually produced visible, correct
results — not just that PRs merged and tests passed.

Three verification phases:

1. **Dashboard endpoints** — HTTP GET against endpoints referenced by the work; assert
   HTTP 200 with non-null, non-default response data
2. **Database tables** — query tables touched by the work; assert rows exist matching
   expected schemas
3. **DoD evidence** — check `dod_evidence` items of type `rendered_output` have passing
   receipts

This skill is **non-blocking**: failures produce receipts and optional Linear comments
but do not halt orchestration. The orchestrator reads the receipt and decides whether
to proceed.

---

## Usage

```
/verification-sweep --ticket TICKET-ID
/verification-sweep --tickets TICKET-ID,TICKET-ID-2
/verification-sweep --epic EPIC-ID
/verification-sweep --dry-run --epic EPIC-ID
/verification-sweep --ticket TICKET-ID --skip-dashboard
```

---

## Announce

"I'm using the verification-sweep skill to verify post-orchestration results for {tickets}."

---

## Phase 1 — Dashboard Endpoint Verification

For each ticket, extract dashboard endpoints from:
1. The ticket's `ModelTicketContract.dod_evidence` entries where `surface == "DASHBOARD"` or
   `type == "rendered_output"`
2. The ticket description (scan for `localhost:3000` or route references)
3. Known endpoint mappings from `omnidash/topics.yaml` for topics touched by the ticket

For each endpoint:

```bash
# HTTP check — expect 200 with non-empty body
curl -s -o /tmp/verify_response.json -w "%{http_code}" http://localhost:3000{route}

# Verify response is not null/empty/default
# Check: HTTP status == 200
# Check: response body is not empty, not "null", not "{}", not "[]"
# Check: if JSON, contains at least one non-null data field
```

Classify each endpoint:
- `PASS`: HTTP 200, response contains real data
- `FAIL_HTTP`: non-200 status code
- `FAIL_EMPTY`: HTTP 200 but response is null/empty/default
- `FAIL_DEFAULT`: HTTP 200 but data matches known default/placeholder patterns
- `SKIP`: endpoint not reachable (service not running) — non-blocking

---

## Phase 2 — Database Table Verification

For each ticket, extract database tables from:
1. The ticket's `ModelTicketContract.interfaces_touched` where surface is `DB`
2. Migration files referenced in the PR diff
3. Projection handlers mapped from `omnidash/topics.yaml`

For each table:

```bash
source ~/.omnibase/.env

# Check table exists and has rows
psql -h localhost -p 5436 -U postgres -d omnidash_analytics \
  -c "SELECT count(*) as row_count FROM {table_name};" 2>/dev/null

# Check schema matches expected columns (if contract specifies them)
psql -h localhost -p 5436 -U postgres -d omnidash_analytics \
  -c "SELECT column_name, data_type FROM information_schema.columns WHERE table_name = '{table_name}';" 2>/dev/null
```

Classify each table:
- `PASS`: table exists, has rows, schema matches expectations
- `FAIL_MISSING`: table does not exist
- `FAIL_EMPTY`: table exists but has 0 rows (when rows are expected)
- `FAIL_SCHEMA`: table exists but columns don't match expected schema
- `SKIP`: database not reachable — non-blocking

---

## Phase 3 — DoD Evidence Verification

For each ticket, fetch the `ModelTicketContract` from the Linear ticket description and
check `dod_evidence` items:

1. Items with `type: rendered_output`:
   - Must have a corresponding receipt in `.onex_state/verification-receipts/`
   - Receipt must show `status: pass`
   - If no receipt exists: `FAIL_NO_RECEIPT`

2. Items with `type: integration_test`:
   - Check CI status on the merged PR — all required checks must be green
   - If PR not merged: `SKIP`

3. Items with `type: playwright_behavioral`:
   - Check for Playwright test results in CI artifacts or local test runs
   - If no results found: `FAIL_NO_EVIDENCE`

Classify each evidence item:
- `PASS`: evidence exists and validates
- `FAIL_NO_RECEIPT`: rendered_output with no receipt
- `FAIL_NO_EVIDENCE`: evidence type with no supporting artifacts
- `FAIL_STALE`: receipt exists but is older than the most recent merge
- `SKIP`: evidence type not applicable or infrastructure unavailable

---

## Verification Receipt Schema

Written to `.onex_state/verification-receipts/{ticket-id}.yaml`:

```yaml
# Verification Receipt
ticket_id: "TICKET-ID"
sweep_timestamp: "2026-04-02T10:30:00Z"
overall_status: pass  # pass | fail | partial
phases:
  dashboard:
    status: pass
    endpoints_checked: 2
    results:
      - endpoint: "/api/intelligence/patterns"
        status: pass
        http_code: 200
        evidence: "Response contains 15 pattern records"
      - endpoint: "/api/platform/registry"
        status: pass
        http_code: 200
        evidence: "Response contains 8 registered nodes"
  database:
    status: pass
    tables_checked: 1
    results:
      - table: "pattern_learning_artifacts"
        status: pass
        row_count: 42
        evidence: "42 rows, schema matches expected columns"
  dod_evidence:
    status: pass
    items_checked: 1
    results:
      - type: "rendered_output"
        status: pass
        evidence: "Receipt exists, status=pass, timestamp within 24h of merge"
idempotent: true  # Running again produces the same receipt if state unchanged
```

---

## Failure Receipt Schema

Written to `.onex_state/verification-failures/{ticket-id}.yaml` when `overall_status != pass`:

```yaml
ticket_id: "TICKET-ID"
sweep_timestamp: "2026-04-02T10:30:00Z"
overall_status: fail
failure_summary: "Dashboard endpoint /api/platform/registry returned empty data"
phases:
  dashboard:
    status: fail
    endpoints_checked: 2
    results:
      - endpoint: "/api/intelligence/patterns"
        status: pass
        http_code: 200
        evidence: "Response contains 15 pattern records"
      - endpoint: "/api/platform/registry"
        status: fail_empty
        http_code: 200
        evidence: "Response body is '[]' — no registered nodes"
  database:
    status: pass
    tables_checked: 1
    results:
      - table: "pattern_learning_artifacts"
        status: pass
        row_count: 42
        evidence: "42 rows, schema matches expected columns"
  dod_evidence:
    status: skip
    items_checked: 0
    results: []
linear_comment_posted: true
```

---

## Linear Comment on Failure

When `overall_status` is `fail` and `--dry-run` is NOT set, post a Linear comment:

```
**Verification Sweep — FAIL**

Ticket: {ticket_id}
Sweep time: {timestamp}

**Failed checks:**
- Dashboard: `/api/platform/registry` returned empty data (HTTP 200, body `[]`)

**Passing checks:**
- Dashboard: `/api/intelligence/patterns` — 15 records
- Database: `pattern_learning_artifacts` — 42 rows

Receipt: `.onex_state/verification-failures/TICKET-ID.yaml`
```

---

## Non-Blocking Behavior

This skill is explicitly **non-blocking**:

- It writes receipts to `.onex_state/verification-receipts/` or `.onex_state/verification-failures/`
- It optionally posts Linear comments on failure
- It does NOT halt orchestration, block merges, or prevent epic completion
- The calling orchestrator (epic-team, ticket-pipeline) reads the receipt and decides
  how to handle failures

---

## Idempotency

Running the sweep twice on the same ticket with the same underlying state produces the
same receipt. The receipt file is overwritten (not appended). Receipt timestamps reflect
the most recent sweep run.

---

## Pre-Merge Mode (`--pr`)

When invoked with `--pr owner/repo#number`, verification_sweep runs in **pre-merge mode**
as the per-PR check driven by `merge_sweep --verify`. In this mode there is no ticket
contract to read; verification targets are selected from the PR diff using a
deterministic changed-file-to-target mapping.

### Target mapping

Applied first-match-wins against `gh pr diff --name-only`:

| Changed-file pattern | Verification target | Check |
|---|---|---|
| `src/**/runtime/auto_wiring/**`, `src/**/runtime/service_kernel.py`, `**/handlers/handler_*.py` | `runtime-health` | `docker inspect` reports `RestartCount == 0`; last 200 runtime container log lines include `Auto-wiring complete`, report `failed=0`, and do not contain `Auto-wiring failed` in the last successful boot |
| `src/**/projection*.py`, `src/**/projector*.py` | Projection table for that module | Table exists, `row_count > 0`, sample row has all non-null required columns matching the Drizzle schema |
| `src/**/handler*.py` (Kafka consumer) | Projection sink + summary endpoint consuming it | Endpoint returns HTTP 2xx with structurally valid JSON matching expected response shape |
| `src/**/route*.py`, `src/**/api*.py`, `pages/api/**` | The modified API route(s) | Endpoint returns HTTP 2xx with non-error payload; response body contains expected top-level keys |
| `drizzle/**`, `migrations/**` | All projection tables in the affected database | Tables exist, migrations applied without error, row schema matches Drizzle definition |
| `topics.yaml`, `contract.yaml` (event bus) | Kafka topic exists + consumer group lag | Topic in `rpk topic list`, consumer group lag via `rpk group describe` |
| No pattern match | Skip verification for this PR | Exit `skipped_no_mapping`, status=skip |

### Exit status (pre-merge mode)

Maps onto the 7 merge_sweep categories. verification_sweep emits exactly one terminal
status per `--pr` invocation:

| Status | Meaning |
|---|---|
| `merged` | All selected checks passed, or no mapping matched and sweep exited cleanly |
| `verification_failed` | One or more concrete check failures — receipt written under `.onex_state/verification-failures/` with expected vs actual and target mapping used |
| `verification_unavailable` | Verification target unreachable (service down, DB offline) — neutral skip |
| `verification_timeout` | Run exceeded `--timeout-seconds` — neutral skip |
| `verification_tool_error` | verification_sweep itself errored (exception, missing binary) — neutral skip |
| `skipped_no_mapping` | No changed-file pattern matched — normal skip |

Pre-merge mode is non-blocking for the caller: merge_sweep reads the status and decides
whether to enable auto-merge on that PR. `verification_failed` is the only status that
blocks auto-merge for the individual PR; all other non-`merged` statuses are neutral
skips that do not halt the sweep batch.

### PR comment on failure

In pre-merge mode, when `overall_status == verification_failed` and `--dry-run` is not
set, a single GitHub PR comment is posted with:

1. Which check failed (target kind + identifier)
2. Expected vs actual (row count, HTTP code, schema columns, ...)
3. The target mapping entry that selected this check
4. A link to the receipt under `.onex_state/verification-failures/`

---

## Integration Points

- **merge-sweep** (`--verify`): invokes verification-sweep in pre-merge mode
  (`--pr ...`) after CI passes but before enabling auto-merge; consumes the 7-category
  status to decide whether the PR proceeds to auto-merge
- **epic-team**: dispatches verification-sweep after all waves complete, before DoD gate
- **ticket-pipeline**: dispatches verification-sweep after auto-merge, before marking Done
- **integration-sweep**: complementary — integration-sweep checks contracts and surfaces;
  verification-sweep checks live data and rendered output
- **data-flow-sweep**: complementary — data-flow-sweep checks full pipeline topology;
  verification-sweep checks per-ticket verification
- **dod-verify**: verification-sweep checks dod_evidence items; dod-verify runs the full
  DoD compliance check

---

## Dispatch Rules

- ALL work dispatched through Agent Teams or Headless
- NEVER edit files directly from orchestrator context
- `--dry-run` produces zero side effects (no receipts, no Linear comments)

---

## Summary Output

```
VERIFICATION SWEEP — {ticket_ids}
====================================

| Ticket    | Phase      | Check                      | Status     | Evidence                                |
|-----------|------------|----------------------------|------------|-----------------------------------------|
| TICKET-ID | Dashboard  | /api/intelligence/patterns | PASS       | 15 pattern records                      |
| TICKET-ID | Dashboard  | /api/platform/registry     | FAIL_EMPTY | Response body is '[]'                   |
| TICKET-ID | Database   | pattern_learning_artifacts | PASS       | 42 rows, schema matches                 |
| TICKET-ID | DoD        | rendered_output            | PASS       | Receipt exists, status=pass             |

Summary: 3 PASS, 1 FAIL, 0 SKIP (4 total)
Overall: FAIL
Receipt: .onex_state/verification-failures/TICKET-ID.yaml
```