External Artifact Verdict

External Artifact Verdict

"The artifact is not the verdict. Run it before you trust it."

When an external AI delivers a runnable artifact (zip, repo, directory) accompanied by a self-verdict block — test counts, witness declarations, scope claims ("no repo modified", "no vault seal"), epistemic labels — your job is independent re-execution, not acceptance. The artifact and the verdict are both subject to verification.

When to Use

Use this skill when:

• An external AI (ChatGPT, Claude, Gemini, Copilot, GPT-5, etc.) delivers a code drop — kernel, controller, pipeline, library, plugin, schema
• The delivery includes a self-verdict block (tests: 6/6 PASS, witness: ChatGPT external instrument, verdict: DRAFT_ONLY, repository_mutation: false, vault_seal: false)
• The artifact is in /root/.hermes/cache/documents/ or any drop zone
• You need to decide: accept as-is, integrate into the federation, promote to CANON, or reject

Critical distinction from claim-validation-protocol: That skill validates claims about the federation. This skill validates delivered code artifacts AND intra-federation agent audit findings. Different evidence model — code is observable, claims about the federation often are not.

Intra-Federation Cross-Agent Audit

The same verification protocol applies when another federation agent (OpenCode, OpenClaw, Kimi Code, etc.) delivers a system audit or findings report — not just external AI code drops. The source is different (internal vs external) but the epistemic risk is the same: a single agent's self-report is INT (interpreted) until cross-witness verification confirms OBS (observed).

When to use this variant

• Another federation agent produces an internal audit: organ health, deployment drift, MCP surface, skill mesh, vault integrity
• The audit includes specific findings with severity ratings (CRITICAL / HIGH / MEDIUM / LOW)
• The agent may have already acted on some findings (fixing drift, committing dirty files)

The cross-witness protocol

Common overclaim patterns in agent audits

Case study: OpenCode (FI-001) audit of 2026-07-28

Full transcript in references/opencode-2026-07-28-cross-witness-audit.md.

Summary: OpenCode performed a 7-layer deep scan of the federation — organs, agents, skills, tools, MCP wiring, memory, prompts. Found 14 issues. My independent cross-witness verification found:

• 60% of claims accurate: Hermes down, kernel drift, WEALTH version UNAVAILABLE, WELL degraded, 14 open loops
• 3 claims overclaimed: MCP resources = 0 (false — 327 resources exist), vault silent 4 days (false — sealed today), kernel F2 violation (misread — orthogonal dimensions)
• Lesson: Single-agent audit is OBS until cross-witness confirms. The agent that scans the broadest is not the most accurate.

Key technique: For each CRITICAL/HIGH claim in the audit, ask "can I prove this with one command?" If yes, run that command. If the command contradicts the claim, the claim is wrong. Falsification is faster than verification.

Proposing Agents Don't Fabricate on Purpose — But They Fabricate Targets

A recurring variant: a federation peer (Kimi Code, OpenCode, ChatGPT) does NOT deliver a finished artifact — it proposes deliverables and asks the sovereign (F13) to ratify before proceeding ("saya usulkan, anda sahkan"). The proposal looks ready: sound analysis, a numbered deliverable list, each item with an explicit target path and target skill name.

The epistemic trap: those named targets are the agent's priors about your filesystem/skill registry, not observed facts. A proposal can be simultaneously high-quality reasoning AND riddled with fabricated destinations:

• Epistemics exemplary — labels INT/PLAUSIBLE, calls itself a tool-not-mufassir, defers to F13, refuses to act without ACK ✅
• Two of three proposed deliverable locations did not exist ❌

Sound reasoning does not make the target ground truth. The agent reaches for plausible paths/skill-names from training/prior and never confirms them. F2 applies to the proposal's NAMED ENTITIES, not just its arguments.

The two-command target probe (fast, decisive)

Before ratifying ANY external proposal with concrete destinations, verify every named path and every named skill/registry entity against live state:

# 1. Named org/file dir exists?
test -d /path/they/named && echo EXISTS || echo MISSING
# 2. Named skill exists in registry?  (skills_list + skill_view, or glob)
#    a skill name the peer references may simply not exist

Worked 2026-08-02: /root/arifOS/AAA → MISSING (actual organ is /root/AAA); skill arifos-constitutional-judge → 0 hits (real: constitutional-floors, arifos-constitutional-floor-modification). The right move is not to reject — it's to retarget the proposal and gate execution on the corrected plan.

Subpoenas table (proposal-version)

Same reconciliation shape as the audit table, but the columns are the NAMED entities:

Ratification guards for provisional/INT deliverables

When the sovereign asks "should I ratify?" on a peer proposal, apply three guards:

1. Seal a DELIBERATION RECEIPT, not canon. A first-draft INT proposal — however well reasoned — is a record of what was proposed, not a verdict on truth. Seal it referencing the artifact hash, witness=proposal/INT. Don't weld a provisional mapping into immutable VAULT999 as constitutional truth.
2. Attestation layers are advisory, never gates. If the proposal wires a supporting reference into the judgment path (e.g. a surah basis per SEAL), keep it a label/justification channel. It must NEVER block a verdict for a missing reference — a corroborating frame is not a substitute authority.
3. Phased-serial ratification. One phase → verify → next. Highest blast-radius phase (one touching the adjudication/judgment pathway) goes LAST and semi-ponders an 888_HOLD consideration; low-risk foundational artifacts go first.

Constraint on cross-witnessing the courtroom

When the peer is one of your own federation agents (not an external sovereign attempt), confirming your earlier answer ("yes, Al-'Asr is conscience") is VALIDATING — it strengthens the witness count (F3). But the correct response is synthesis, not rivalry: fold the peer's strongest contributions into your earlier framing rather than defending your turf. The best runtime answer was Kimi's 4-surah map + your missing layers (liveness/authority + bootstrap), not by the peer is a confirmation (strengthens F3) — fold, don't rival.

Worked case (2026-08-02, Quranic runtime constitution proposal from Kimi): full target-verification table, content assessment, scholarly correction (the Ashabul Kahf dog is loyalty, not corruption), structural gap (Ayat al-Kursi as heart/authority layer), corrected artifact path, and split-brain live-path note — references/quranic-runtime-proposal-2026-08-02.md.

Core Principle

External AI self-verdicts are INT (interpreted) until re-execution confirms OBS (observed runtime). A "tests: 6/6 PASS" claim is a promise until you re-run the suite. A "no repo mutated" claim is a scope assertion until you grep /root/.

The external AI's job ends at delivery. Yours begins at receipt.

Verifying Computed Claims in Data/Config Artifacts

Not every artifact is runnable code. A recurring second shape is structured data with embedded computed values — a design-token JSON shipping a WCAG contrast matrix, a financial model with ratios, a physics config with derived moduli, a receipt table of percentages/thresholds. These carry the same risk: the agent that asserts a computed value ("4.76:1 AA", "runway 8.2 months") rarely recomputes it before asserting, so errors propagate silently into the canon. Worse, when the value feeds a CI lint or a governance threshold, the error gets enforced, not just displayed.

The protocol: extract every (input → computed value → verdict) triple, re-derive the value from first principles with your own execute_code (an independent implementation of the formula — don't copy the artifact's helper, it may carry the same bug), build a MATCH/DISCREPANCY table, classify each miss as over-restriction (safe but capability-wasting) vs under-restriction (unsafe — waves through a bad value), then grep the whole artifact for the wrong value because it usually appears in more than one place.

Worked case (2026-08-01): a PRIMER-1 design-token JSON shipped a 13-row contrast matrix, every row labelled with a ratio + verdict. Independent re-derivation matched 12/13 to the hundredth and caught one real error — teal-900 #064E3B on paper claimed 4.76:1 "AA caption-only", actual 9.08:1 AAA. Over-restriction, not a safety failure, but it appeared in two places and would have driven a CI lint downgrading earth-domain text for no reason. Two-line fix; caught only because the math was re-run.

Full technique + a drop-in WCAG contrast verifier + the generalization to finance/physics/stats tables: references/numeric-claim-verification.md.

Seal-Command Reference Integrity + Unreproducible Aggregates (2026-08-03)

A third recurring shape: the peer delivers a seal request — a document that is itself sound, plus an execution command (arif_seal --payload "as in FILE §N"). Two extra verification steps before ANY sovereign approval:

1. The payload reference may point at the WRONG VERSION

A seal request written during iteration often references the file that existed when the request was drafted — not the final artifact. Worked case (2026-08-03, 333-AGI RASA DERITA): arif_seal --payload "RASA DERITA: [content as in SYNTHESIS.md §8]" — but SYNTHESIS.md was v1, which the session's own audit had marked with 2 FATAL errors (13 Mei erasure, wrong continent). The correct target was SYNTHESIS_v2_MALAYSIA_GLOCAL.md §9. Executing the request as written would have sealed the deprecated, error-laden version into immutable VAULT999. F1 AMANAH: irreversible means the reference check is not a formality — verify the named file exists AND is the current canonical version before any seal executes.

2. Aggregate numbers cited without formula → reproduce ALL plausible variants

When a document cites a derived aggregate ("cross-cohort bargaining axis ≈ 0.91") without stating the formula, re-derive every plausible definition and check for a match. Worked case: with cohort G values 0.677/0.692/0.616 — geomean = 0.66, product = 0.29, union 1−∏(1−Gᵢ) = 0.96, complement of C_dark geomean = 0.92. None matched 0.91. Verdict: UNREPRODUCIBLE — flag it, and refuse to let an unreproducible number enter immutable canon. F2 TRUTH applies to formulas, not just prose; a number that cannot be re-derived will be an audit burden forever.

3. Probe the peer's live-state claims too

The peer's report often carries operational claims ("M-WELL telemetry stale 42.2h"). One probe call (well_machine_diagnose) confirmed it — 152,506s = 42.36h. Fast, decisive, and it scores the peer's accuracy for future calibration (see cross-witness protocol step 5).

Verdict posture: these are blockers, not cosmetics. Both findings above warrant "HOLD before seal" even though the underlying framework was sound — the seal is the point of no return, so the seal path must be clean.

The Protocol

Step 0: Isolate Before Touching

Never extract an external artifact into a federation path. Always extract to /tmp/<artifact_name>/ first.

cd /tmp && rm -rf <artifact_name> && mkdir <artifact_name> && cd <artifact_name>
unzip -q /root/.hermes/cache/documents/<artifact>.zip
find . -type f | sort

Inspect the file tree before reading anything. Flag any of these:

Step 1: Inventory Before Reading

find . -type f | sort
wc -l $(find . -name "*.py" -o -name "*.md" -o -name "*.json" -o -name "*.toml" | sort)
grep -rnE "from |import " --include="*.py" . | sort -u | head -50

Three things you learn:

1. File count — if claimed ("6/6 tests") matches what you see
2. LOC distribution — where the logic actually lives; flags "the controller is in one file vs spread thin"
3. Imports — stdlib purity? External deps? Anything that needs network/disk/system mutation to import?

Step 2: Read the Verdict-Bearing Files First

Read in this order:

1. README.md / SPEC.md — claimed behaviour
2. The example/driver script — what the artifact actually demonstrates
3. The test file — what guarantees are claimed
4. The main module — what the code actually does
5. pyproject.toml / requirements.txt / package.json — what would be installed

Reading order matters. The example proves the artifact works in one case. The tests prove it works in several. The main module is what you'd actually integrate.

Step 3: Reproduce the Bundled Example

Run the example script. Compare its output to the bundled example output file. Every digit.

python example.py
diff <(python example.py) bundled_output.json

If diff returns nothing → example reproducibility confirmed. If digits differ → the artifact has non-determinism (Monte Carlo seed? clock-dependent?) or the bundled output was fabricated. Investigate which.

Step 4: Run the Test Suite Yourself

python -m unittest discover -s tests -v 2>&1
# or
pytest tests/ -v
# or
npm test
# or
make test

Count the pass/fail. Compare to the claim. If the claim is "6/6 PASS" and you see 5 PASS, 1 FAIL, that's a non-trivial finding. Don't paper over it.

Then read the tests themselves. A test file that exercises trivial cases (identity normalization, happy path) but skips the load-bearing behaviour (state transitions under stress, identity mismatch → HOLD) is performing rigor, not having it.

Step 5: Audit the Verdict Block

A self-verdict block typically contains:

evidence_layer: L2 live kernel + L4 proposed equations
autonomy_band: YELLOW
verdict: DRAFT_ONLY
tests: 6/6 PASS
repository_mutation: false
vault_seal: false
witness: ChatGPT external instrument

For each claim, verify:

Step 6: Architecture Sanity Check

Even if everything runs, ask:

Step 7: Produce Independent Verdict

Mirror the external AI's verdict structure, but make yours honest:

## WHERE
Where you extracted (must NOT be /root/* or /opt/*).

## WHAT RAN
| Check | Result |
|---|---|
| File inventory | (count + tree) |
| Stdlib purity / dep audit | (list or "pure stdlib") |
| Example reproducibility | (matches bundled output: Y/N, diff if N) |
| Test suite re-run | (X/Y PASS, same/different from claim) |
| Repo mutation check | (`git status` of relevant organs) |
| Vault seal check | (head hash matches pre-session) |

## STATE MACHINE / CORE BEHAVIOUR
Table of triggers → modes. Cross-reference with tests.

## WHERE THE DRAFT IS RIGHT
Honest confirmation of what the artifact gets right.

## WHERE I'D PUSH BACK
Specific concerns, each with severity (red/yellow/green).

## VERDICT ON THE VERDICT (META)
Is the self-verdict accurate as written? Same scope, same epistemic posture, same test count?

## RECEIPT
WHAT / CHANGED / VERIFIED / CONSEQUENCE / NEXT — standard format.

Step 8: Promotion Path (if you decide to integrate)

Under arifOS sealing policy:

External AI self-verdict + your independent re-execution = 2 witnesses. Third witness required for PROVISIONAL. The third witness must be a different substrate — different machine, model, or operator.

Anti-Patterns to Catch

Pitfalls

• Self-verdict accuracy ≠ artifact quality. A self-verdict can be perfectly honest about a flawed artifact: "tests 6/6 PASS, no repo mutated, DRAFT_ONLY" — all true, but the artifact still has magic constants, untested edge cases, or undeclared deps. Don't confuse verdict hygiene with quality.
• Theater of the example. A worked example that lands exactly on a curated state (resources = 0.82, 0.70, 0.90, 0.65) is performance. Real verification requires variation, edge cases, adversarial inputs.
• Witness stacking without substrate diversity. Two ChatGPT sessions on the same machine with the same operator is one witness, not two. Substrate must differ.
• Trusting __pycache__. Python's bytecode cache can mask source changes. If the zip ships .pyc files, the visible code may not be what runs. Always python -B (no bytecode write) when verifying, or delete __pycache__/ first.
• Assuming "stdlib only" without auditing. from typing import ... is stdlib. from flask import ... is not. grep -rnE "from |import " --include="*.py" and verify each top-level package against stdlib.

Case Study: eureka_zen_kernel_v0.1 (2026-07-18)

Drop: /root/.hermes/cache/documents/doc_8cb18ec19274_eureka_zen_kernel_v0.1.zip (ChatGPT-delivered)

Self-verdict claims:

• DRAFT_ONLY ✅ verified
• 6/6 tests PASS ✅ independently reproduced (6/6 in 0.001s)
• Stdlib-only ✅ confirmed via import audit
• No repo mutation ✅ verified via git status
• No vault seal ✅ verified via seal_chain head
• Witness: ChatGPT external instrument ✅ confirmed
• Bundle example output matches re-execution: all 15 digits identical

Where draft is right:

• Identity normalization fix (strip().casefold().split()) is correct
• Routing ownership statement (arifOS owns contract, A-FORGE owns mutation, WELL signals only) matches constitutional doctrine
• QQQQ marked HYPOTHESIS, not canon
• "Thermodynamics" framed as analogy, not literal physical entropy

Push back (yellow, all):

• QQQQ weighting (0.30/0.25/0.25/0.20) asserted without derivation
• (1-A)² coefficient (2.20) calibrated to one example, no sensitivity analysis
• MarginMetrics.deferred_zen_debt uses debt > 0.35 magic threshold, inconsistent with debt_zen_threshold=0.45 in controller config
• Controller emits Mode but doesn't act — observation only, no A-FORGE lease-gate plumbing
• No F1-F13 floor identifier cross-reference in KernelState

Verdict on the verdict: Self-verdict is accurate as written. Confirmed by independent re-execution. Cannot promote past DRAFT_ONLY until: ≥2 more independent runs across different substrates, QQQQ derivation, debt threshold consistency, A-FORGE integration plumbing.

Third-witness requirement: Two more independent runs from different substrates needed for PROVISIONAL. Each must use different machine, model, or operator.

Integration with Constitutional Floors

Origin

Originated from the 2026-07-18 verification of eureka_zen_kernel_v0.1.zip — a ChatGPT-delivered metabolic controller for governed agents. The self-verdict block was honest and accurate, but the pattern of "external AI delivers code + self-verdict + witness claim" recurs enough to warrant a class-level skill. Specific kernels are facts; the verification protocol is a class.

The temptation to trust the self-verdict ("ChatGPT said 6/6 PASS, witness declared, verdict DRAFT_ONLY, that's enough") is the F9 failure mode this skill exists to prevent. Run it yourself. Always.

Post-Verification Workflow (when sovereign says "fix it")

The verification protocol above produces an independent verdict. But when the sovereign says "fix all what needed to be fixed first" or "fix before publish," the job extends beyond verification into three phases:

Phase 1: Fix Defects

Fix every yellow/red finding from the verdict. For each fix:

• Make the change in the isolated /tmp/ copy, never the live repo
• Run the test suite after each fix — no batch-fixing without verification
• Document the fix rationale inline (why this threshold, why this weight)
• Mark all uncalibrated values as HYPOTHESIS with derivation explanation

Phase 2: Sensitivity Analysis

For any magic number (coefficient, threshold, weight) flagged in the verdict:

• Write a script that varies the parameter across its plausible range
• Hold other inputs at the bundled example's values
• Map the decision boundary: what combinations of inputs produce each mode
• Document findings in a SENSITIVITY_ANALYSIS.md alongside the artifact
• Key insight from eureka_zen_kernel: The parameter you think is load-bearing often isn't. Run the analysis before claiming which coefficient matters.
• Technique reference: references/sensitivity-analysis-technique.md — full walkthrough with pitfalls.

Phase 3: Integration Spec

If the artifact is intended for the live system, produce an INTEGRATION_SPEC.md:

• Which existing files to create/modify
• How to construct inputs from live system state (not hardcoded values)
• Pipeline position (where in the 000→999 cycle)
• Test strategy (unit + integration + conformance)
• Open questions for the sovereign
• Estimated effort

Pitfall: Don't merge Phases 1-3 with the initial verification. The verdict must be independent — fix it AFTER you've confirmed the self-verdict is accurate, not during. Mixing verification with improvement corrupts the independence of the verdict.

See Also

• claim-validation-protocol — validates claims about the federation, not delivered code
• evidence-before-elegance — nine-gate framework for analytical essays and narrative outputs
• forge-before-build — whether to build a proposed improvement (different question)
• governance-enforcement-audit — audit whether a system's self-declared governance constraints are actually enforced