02 Audit Report

Audit Report Skill

Use When

• Use when independently assessing controls or delivery evidence and issuing read-only findings, severity, scope limits, and corrective actions. Use risk-assessment for prospective risk analysis and compliance-documentation to author control artefacts.

Do Not Use When

• Do not use when a more specific upstream or downstream skill owns the task, or when the required project context has not been prepared.
• Do not use this skill to fabricate missing project facts, legal conclusions, test results, approvals, or certification claims.

Required Inputs

Capability and permission boundaries

Default to read-only. Read and search access to the supplied artefacts are required. Execute only non-mutating validation when authorised; editing remediation, publishing, signature, certification, production mutation, destructive action, spending, and risk acceptance require explicit authority.

Degraded Mode

When files, tools, network, rendering, fonts, execution, or evidence are unavailable, return the narrowest useful qualified draft or finding set. Name every unavailable check and its consequence; an unassessed check is never a pass. Preserve evidence already gathered and provide the exact next verification step.

Decision Rules

Workflow

1. Confirm the requested artefact, audience, scope, decision owner, and applicable baseline or version. Work read-only by default; source mutation, publication, signature, certification, production change, or risk acceptance requires explicit authority.
2. Inspect every required input and record missing, stale, conflicting, or inaccessible evidence. Stop claims that depend on an unresolved required input.
3. Apply the Decision Rules, then execute the existing Core Instructions below in order; preserve project terminology and trace each material statement to its source.
4. Test the draft against the output acceptance conditions and domain quality standards. If a check cannot run, mark it not assessed and never convert it into a pass.
5. On failure, recover by preserving completed evidence, identifying the narrowest corrective action and owner, and rerunning only the affected checks before handoff.
6. Produce the named artefact and evidence record; publish, sign, certify, mutate production, or accept risk only under explicit authority.

Outputs

Evidence Produced

Quality Standards

• Keep outputs grounded in source context, traceable to stated standards, and specific enough to review or verify.
• Use deterministic acceptance conditions and preserve traceability from source to decision and output.
• Separate facts, inferences, assumptions, and approvals; never present one as another.
• Apply 28-anti-ai-slop during authoring and 29-ai-slop-audit at major checkpoints and release.

Anti-Patterns

• Producing Audit Report from assumptions instead of named project sources. Fix: Cite the source or mark the item unverified.
• Treating a missing or inaccessible check as passed. Fix: Mark it not assessed, state impact, and block dependent claims.
• Using vague gates such as adequate, secure, or user-friendly. Fix: Replace each with an observable criterion, threshold, and evidence source.
• Copying a generic template without product, control, role, version, or jurisdiction detail. Fix: Ground every section in the supplied context and remove unused boilerplate.
• Publishing, signing, certifying, changing production, or accepting risk without authority. Fix: Prepare a draft and route the decision to the accountable owner.
• Listing evidence without provenance or an acceptance result. Fix: Record source, period, integrity check, mapping, and pass/fail/not-assessed status.

Worked Example

Example: if required evidence is missing or inaccessible, mark the check not assessed, state impact, and stop any pass decision. Record the evidence and result in the validation record; this avoids false assurance from an incomplete review.

References

• Generation logic: load when creating the complete artefact.
• Skill notes: consult for local examples and invocation context.
• Repository operating rules: apply the engine's routing, evidence, and release gates.

Overview

For an engine or produced-document audit, load the Kaizen product-audit skill and its evidence matrix. Report the raw diagnostic score separately from the required audit score min(raw_score, 65), then produce a named remediation plan targeting 95/100 with acceptance evidence and re-test criteria.

This skill generates a formal Verification and Validation (V&V) audit report that systematically evaluates all project documentation against IEEE 1012-2016 criteria. The report assesses completeness, consistency, correctness, and traceability, producing categorized findings with severity levels and actionable remediation recommendations. The audit concludes with a formal recommendation of Pass, Conditional Pass, or Fail.

When to Use This Skill

• After the traceability matrix has been generated and reviewed
• When preparing for external regulatory or client audits
• When a project milestone requires formal V&V sign-off
• When stakeholders request evidence of documentation quality
• When transitioning between major project phases
• When anomalies have been reported and require systematic investigation

Quick Reference

Input Files

Output Files

Core Instructions

1. The skill SHALL read projects/<ProjectName>/<phase>/<document>/Traceability_Matrix.md and projects/<ProjectName>/<phase>/<document>/SRS_Draft.md. If either file is missing, halt execution and report the error.

2. The skill SHALL define the audit scope, listing every document reviewed, the standards applied, and the methodology used (IEEE 1012-2016 V&V framework).

3. The skill SHALL assess Correctness by verifying that each requirement accurately reflects stakeholder intent as documented in upstream artifacts. Flag deviations as findings.

4. The skill SHALL assess Completeness by checking that every business goal has at least one corresponding requirement, every requirement has a test case, and no sections contain TBD or placeholder content.

5. The skill SHALL assess Consistency by cross-referencing terminology, data types, and logical structures across all reviewed documents. Flag contradictions or ambiguities.

6. The skill SHALL assess Traceability by analyzing the Traceability Matrix for coverage gaps, orphan items, and broken links.

7. The skill SHALL categorize every finding by severity:

   • Critical: Renders the system unsafe, non-compliant, or fundamentally broken
   • Major: Significant gap that must be resolved before release
   • Minor: Cosmetic or low-impact issue that should be addressed
   • Observation: Improvement suggestion with no compliance impact

8. The skill SHALL generate a Remediation Plan with specific corrective actions, responsible parties (where identifiable), and priority ordering.

9. The skill SHALL produce a Compliance Assessment Matrix mapping each IEEE 1012-2016 clause to its compliance status (Compliant/Partially Compliant/Non-Compliant).

10. The skill SHALL conclude with an Audit Summary and formal Recommendation:

    • Pass: All critical and major findings resolved; documentation meets standards
    • Conditional Pass: No critical findings; major findings have approved remediation plans
    • Fail: Critical findings present or excessive major findings without remediation

11. The skill SHALL tag any finding requiring immediate action with [V&V-FAIL] and reference the originating document and section.

Output Format Specification

The generated Audit_Report.md SHALL contain the following sections:

# Verification & Validation Audit Report
## 1. Document Information
## 2. Audit Scope & Methodology
## 3. Documents Reviewed
## 4. Compliance Assessment Matrix
## 5. Findings
### 5.1 Critical Findings
### 5.2 Major Findings
### 5.3 Minor Findings
### 5.4 Observations
## 6. Correctness Analysis
## 7. Completeness Analysis
## 8. Consistency Analysis
## 9. Traceability Analysis
## 10. Remediation Plan
## 11. Audit Summary & Recommendation
## 12. Revision History

Common Pitfalls

• Producing findings without actionable remediation steps
• Conflating severity levels (e.g., marking cosmetic issues as Critical)
• Omitting the formal Pass/Conditional Pass/Fail recommendation
• Auditing only the SRS without cross-referencing design and test artifacts
• Using subjective language ("seems incomplete") instead of specific evidence
• Failing to reference the exact document section where a finding originates

Verification Checklist

1. All four V&V dimensions (correctness, completeness, consistency, traceability) are assessed.
2. Every finding has a severity level, description, affected artifact, and remediation action.
3. The Compliance Assessment Matrix covers all applicable IEEE 1012-2016 clauses.
4. The audit concludes with a formal Pass/Conditional Pass/Fail recommendation.
5. [V&V-FAIL] tags are applied to all findings requiring immediate action.
6. The Documents Reviewed section lists every artifact examined during the audit.
7. Findings reference specific document sections, not just document names.
8. The Remediation Plan prioritizes corrective actions by severity.

Integration

• Upstream: Requires the traceability matrix from 01-traceability-matrix and all documentation artifacts from prior phases.
• Downstream: Feeds 03-compliance-documentation and 04-risk-assessment. Audit findings may trigger re-execution of upstream skills.

Standards Compliance

Resources

• IEEE 1012-2016: Standard for System, Software, and Hardware Verification and Validation
• IEEE 830-1998: Recommended Practice for Software Requirements Specifications
• CLAUDE.md: V&V Standard Operating Procedure and Failure Protocols