Research Integrity Audit

Research Integrity Audit

Audit without modifying the input. Treat block as a release stop, warn as a required review, and pass only as “no rule violation detected,” never as proof that a historical claim is true.

Run

From this Skill's own folder, so the command works both inside the repository and in a copy of the folder downloaded on its own:

python3 scripts/audit_case.py \
  path/to/research-case.json \
  --output path/to/integrity-audit.json \
  --pretty

Existing report paths are not overwritten. Exit codes are 0=pass, 1=warn, 2=block, and 3=input/output error.

Interpret

Read status, then substance, then checks and findings. Resolve every block before packaging. Review every warn, recording why it was resolved, accepted, or left unresolved. Preserve the report as an audit event and rerun after changing claims or evidence.

Vacuous versus substantive

status: pass alone never means a case earned anything. Read it together with substance:

A case with zero publishable claims is vacuous and is reported as block (VACUOUS_AUDIT_NO_PUBLISHABLE_CLAIMS). This gate exists to certify claims; certifying nothing is not a pass. An empty case and a validated case must never be readable as the same result.

Checks

Every rule is reported individually in checks, with its own finding_ids.

Definitions the checks rely on:

• Substantive claim. A publishable claim with at least one linked evidence record that exists in the case, is not HYP, is not registered under a HYP material, is not produced by a generated material, and is not rejected.
• Trusted support. Only verification_status: human-checked. This is an allowlist, never a denylist: support whose status is missing, non-string, misspelled, outside the schema enum, unverified, machine-checked, or conflicted counts as unadjudicated. Unknown provenance is untrusted, so the gate fails closed.
• Completion attempt. stage is packaging or complete, or extensions.stop_condition_met is true, or a decision with action: complete exists.
• Adjudication. For RIA-009, a claim resting only on untrusted support needs a decision carrying an adjudication object that names an accountable human — a non-empty reviewer_role that is not an automated actor, plus a non-empty rationale and reviewed_at — and whose affected_ids, evidence_ids, or evidence_assessments reach the claim or its supporting evidence. An empty or half-written adjudication object clears nothing and is itself reported as RIA-018.
• Automated actor. A reviewer role or evidence method naming AI, an agent, an LLM, or generated or automated production. An AI actor never satisfies a human gate, and an AI adjudication never clears RIA-009.
• Trusted transcription. For RIA-011, an evidence record that is human-checked, sits on the OBS or SRC layer, was not produced by an automated actor, and either reads as a transcription or ground truth in its own text or is designated by extensions.trusted_transcription.
• Reported metric figure. For RIA-011, an accuracy, CER, WER, precision, recall, or F1 word within forty characters of a digit, or a metric-named key with a numeric value. An honest negative sentence such as “these differences do not establish OCR accuracy” carries no digits and is not treated as a figure.
• Sticky blocker. A decision with action human_review, stop_unresolved, or rollback. It stays in force until a later continue, reroute, or complete decision names it in resolves_decision_id. Resuming without that reference is RIA-014, not a resolution: the blocker stays open and is still reported by RIA-013. Blockers accumulate — every unresolved blocker is tracked, so a second blocking decision never buries the first, and a resumption that names one blocker lifts only that one. RIA-013 reports each still-open blocker separately.
• Intact hash binding. extensions.latest_decision_state exists exactly when decisions exist, identifies the last decision, reproduces its digest, carries a SHA-256 input case hash, and never claims more records than the case holds. Marker counts that lag the case are ordinary staleness from a non-decision append and are accepted.
• Layer confusion. Evidence presented as OBS or SRC whose material is registered as DER, INF, or HYP, or whose material role is generated, or whose derived material sits on a different layer.

Report contract

The report envelope is laddered through report_schema_version, currently 1.1. Every field emitted by version 1.0 is still emitted with the same meaning; 1.1 only adds report_schema_version, claims_examined, substance, publishable, and extra summary counters.

skill_version deliberately stays 1.0.0. skills/rubbing-research-orchestrator/scripts/orchestrate.py pins that exact string, together with skill_id, case_id, input_case_sha256, input_unchanged, and matching status/gate_status. Ladder the report contract; do not repurpose those fields.

Limits

The audit does not fetch sources, authenticate artifacts, give legal clearance, detect records omitted from the input, or replace expert review. RIA-004 is a legacy reporting marker with no independent finding path; it is retained for report compatibility. RIA-018 checks that an adjudication names a reviewer, not that the named reviewer exists or is qualified; identity cannot be verified from the JSON alone. A pass means no implemented rule was violated by the supplied case, nothing more.