alenpjose Validate Release Evidence
Workflow
- Rewrite each requested outcome as one independently testable claim.
- Assign the minimum evidence level required by the claim.
- Record the evidence artifact, observed result, environment, and exact build identity.
- Run
scripts/validate_evidence.py --manifest <path>. - Report passed, missing, failed, and unsupported claims separately.
- Describe a release as validated only when every required claim passes at its required level.
Read references/evidence-manifest.md before creating or reviewing a manifest.
Evidence Levels
Use the weakest level only when it genuinely proves the claim:
static_inspection: source, configuration, or package structure review.build: successful compilation or packaging of an identified commit.simulated_runtime: mocked or synthetic execution.real_runtime: execution with the real application or integration binaries.user_acceptance: user confirmation from the intended environment.
Do not substitute a lower level for a higher requirement. Compilation cannot prove that a DLL loads. Installer creation cannot prove installation or uninstallation. Source inspection cannot prove source-instance isolation, output routing, persistence, or visual correctness.
Integrity Rules
- Tie evidence to a commit SHA, artifact digest, or local binary hash.
- Treat evidence from an older binary as applicable only when the tested runtime bytes are proven identical.
- Mark inaccessible or expired artifacts as unresolved delivery.
- Preserve negative evidence; do not omit failed attempts.
- Distinguish environment restrictions from product defects and retain the logs supporting that classification.
- Use
unsupportedwhen no available test can establish the claim; never silently downgrade it.
Output
Return a compact matrix with claim, required level, best observed level, status, and evidence location. End with one of:
validated: every required claim passed.not validated: one or more claims failed or lack sufficient evidence.partially validated: only when the user explicitly requests an interim status.