Audit Modeling Evidence

Audit Modeling Evidence

Audit without silently repairing, rerunning, or promoting unsupported claims. Return the smallest defensible repair and a clear status: pass, warn, failed, or blocked.

Workflow

1. Identify source-of-truth files, active subquestions, evidence/data mode, ambition, workflow stage, result status, and audit scope.
2. Select only applicable lanes: intake/data; backend; method/experiment; candidate validation; simulator/decision trace; innovation/readiness; frozen evidence/manuscript.
3. Resolve SKILL_ROOT as this Skill directory and invoke scripts by absolute quoted path.
4. Run the smallest deterministic check first. Preserve input hashes, commands, environment, and artifact paths.
5. Distinguish observed, inferred, diagnostic-only, validated, frozen, and out-of-scope evidence. Rewriting a claim cannot turn failed evidence into a pass.
6. Report each issue with file, field/row, observed condition, required condition, affected downstream artifacts, and smallest repair.

Script routing

• Data/leakage: scripts/audit_dataset.py.
• MATLAB capability and numerical ownership: scripts/audit_backend_contract.py.
• Experiment budgets/runs: scripts/register_experiment.py, scripts/run_experiment.py.
• Candidate evidence/promotion: scripts/audit_candidate_evidence.py, scripts/promote_validated_candidate.py.
• Innovation and competition readiness: scripts/audit_innovation_portfolio.py.
• Exact/surrogate decisions: scripts/audit_decision_trace.py, scripts/check_transition_fidelity.py.
• Numeric MATLAB–Python round trip: scripts/preflight_matlab_python_bridge.py, only when numerical values cross languages.
• Whole project: scripts/audit_modeling_project.py.
• Optional registries: scripts/init_evidence_bundle.py, scripts/audit_evidence_bundle.py.
• Gate applicability: scripts/resolve_required_gates.py.

Conditional rules

• Accept equivalent official MATLAB MCP capability names; require a real call, recorded environment, and artifact. Local MATLAB is the next route; another backend needs a concrete auditable reason.
• Require challengers only when empirical or heuristic model comparison is meaningful. Analytical/exact work may use a bound, proof obligation, limiting case, or independent checker.
• Apply action masks only to constructive/policy search where illegal actions can enter the generated set.
• Choose exact/surrogate fidelity cases from horizon, action, branch, boundary, and rare-state coverage; no universal transition count is sufficient.
• Separate verification, reproducibility, sensitivity, robustness, and generalization. An independent rerun does not prove transfer.
• Audit Pareto claims on raw-objective dominance, span, coverage, stability, and selection rule.
• An innovation is not validated by a nonempty path string: evidence artifacts must exist inside the project. Negative candidates may be downgraded honestly.
• Freeze only validated claims with traceable hashes and artifacts. A self-review may approve internal freeze; user input is required only for a material interpretation choice or external submission.

Output

Return:

• source-of-truth lock and applicable gates;
• checks run and deterministic commands;
• pass/warn/failed/blocked findings with evidence paths;
• stale or affected downstream artifacts;
• validated versus diagnostic-only claims;
• smallest next repair and stop condition.

Read only references needed by the lane: references/data-and-reproducibility.md, references/matlab-native-workflow.md, references/matlab-mcp-setup.md, references/candidate-validation-contract.md, references/validation-playbook.md, references/experiment-budget-and-promotion.md, references/innovation-and-competition.md, references/exact-simulation-contract.md, references/discrete-event-scheduling.md, references/hard-constraint-action-mask.md, references/score-gap-analysis.md, or references/evidence-registry-contract.md.