az9713 /generate-soc2-evidence
Aggregates comprehension gate artifacts into a SOC 2 CC8 Change Management evidence package. The evidence already exists — COMPREHENSION_ARTIFACT.md files are the review records. This skill's job is to organize that evidence, identify gaps, and produce a document auditors can inspect.
What SOC 2 CC8 requires and how this helps
SOC 2 Trust Service Criteria CC8 (Change Management) requires evidence that significant changes were tested, reviewed, and approved before deployment. Each COMPREHENSION_ARTIFACT.md documents that a review was performed before a significant change merged. The pre-commit hook (pre-commit-comprehension-check.ps1) enforces that commits of ≥50 lines require such an artifact — creating an audit trail automatically.
The gap detection step is the most important part: changes that merged without a gate review are not just missing from the package, they are potential control exceptions that must be disclosed to the auditor. Finding them now is better than an auditor finding them.
Arguments
- (none) — uses all artifacts, all time periods
--period YYYY-MM-DD YYYY-MM-DD— restrict to a specific audit period (start and end date)--control CC8— explicit control filter; currently only CC8 is supported
Phase 1: Discover evidence artifacts
Scan .claude/comprehension/ for all files matching COMPREHENSION_ARTIFACT*.md.
For each artifact, extract:
- Date — from the artifact's "Generated" or "Date" field, or file modification time as fallback
- Branch / change set — from "Branch" or "Change" field in the artifact
- Verdict — CLEAR / REVIEW REQUIRED / HOLD
- Findings summary — count by severity (HIGH/MEDIUM/LOW) from the findings table
- HOLD resolution — if verdict is HOLD, check whether a subsequent CLEAR artifact exists for the same branch (search for artifacts with matching branch name and later date)
If --period was provided, filter artifacts to only those dated within the period.
Build an evidence index from these extractions.
Phase 2: Identify control gaps
A gap is a significant change (≥50 lines, per the pre-commit hook threshold) that merged without a COMPREHENSION_ARTIFACT.md.
If git is available:
- Get all commits in the period:
git log --oneline --since=[start] --until=[end] - For each commit, check the diff size:
git show --stat [hash] - For commits ≥50 lines added+changed, check whether a COMPREHENSION_ARTIFACT.md exists for that branch or was created within 24 hours before the commit
Gap detection logic:
- Extract branch name from commit message (common patterns:
Merge branch 'X',Merge pull request ... from X) - Check for artifact with matching branch name in the evidence index
- If no match found → gap
Known legitimate gap sources (note these separately from unexplained gaps):
- Commits before the pre-commit hook was installed
- Commits with
SKIP_COMPREHENSION_GATE=1(check commit message or environment context) - Commits that stayed under 50 lines individually but were part of a coordinated change
If git is not available, note that gap analysis could not be performed and proceed with the artifacts found.
Phase 3: Interview about gaps
For each identified gap, ask in a single batched question to avoid interrupting the user repeatedly:
I found [N] changes that appear to have merged without a comprehension gate review:
[List: date, branch, approximate size in lines]
For each, I need to know:
- Was this change reviewed through another mechanism? (pair programming, synchronous code review, etc.)
- Is there alternate evidence that this change was reviewed?
- If no review occurred: is this change still deployed, was it rolled back, or did it cause an incident?
Record answers. Gaps with alternate evidence are "control gaps with explanation." Gaps with no alternate evidence are "control exceptions" — these must be disclosed to the auditor and remediated.
Phase 4: Write the evidence package
Write to docs/compliance/soc2-cc8-evidence-YYYY-MM-DD.md (where YYYY-MM-DD is today's date or the period end date if --period was specified). Create docs/compliance/ if it doesn't exist.
Use this structure exactly:
# SOC 2 Change Management Evidence (CC8)
Period: [start date] to [end date]
Generated: [today's date]
Status: DRAFT — verify with auditor before submission
> This evidence package was generated by aggregating COMPREHENSION_ARTIFACT.md files produced
> by the dark code suite comprehension gate. It covers the CC8 (Change Management) trust service
> criterion only. Other trust service criteria require separate evidence collection.
---
## Summary
| Metric | Value |
|--------|-------|
| Total significant changes in period | N |
| Changes with comprehension gate review | N (X%) |
| Changes without review (gaps) | N |
| Gaps with alternate evidence | N |
| Control exceptions requiring disclosure | N |
| HOLD verdicts resolved before merge | N / N total HOLDs |
---
## Evidence Index
| Date | Change / Branch | Verdict | Findings | Artifact |
|------|-----------------|---------|----------|---------|
| YYYY-MM-DD | [branch name] | CLEAR | 0 HIGH, 0 MED, 2 LOW | [relative path] |
| YYYY-MM-DD | [branch name] | REVIEW REQUIRED | 0 HIGH, 1 MED | [relative path] |
[If no artifacts: "No comprehension gate reviews found for this period. The gate may not have
been active, or all changes were below the 50-line threshold."]
---
## Control Gaps
Changes that merged without a comprehension gate review during this period.
| Date | Change / Branch | Lines | Alternate evidence | Status |
|------|-----------------|-------|--------------------|--------|
| | | | | |
[If none: "No control gaps identified in this period."]
---
## Control Exceptions
Gaps where no alternate evidence exists. These must be disclosed to the auditor and remediated.
| Change / Branch | Date | Lines | Risk assessment | Remediation status |
|-----------------|------|-------|-----------------|-------------------|
| | | | | |
[If none: "No unmitigated control exceptions in this period."]
---
## Scope Limitations
- **Pre-hook baseline:** The comprehension gate hook was installed on [date if determinable from git log — otherwise: "date unknown"]. Changes before that date will appear as gaps regardless of whether they were reviewed.
- **Line threshold:** Only changes ≥50 lines are subject to the gate. Smaller changes are not represented in this evidence package.
- **CC8 only:** This package covers Change Management. Availability (CC9), Confidentiality (CC11), Processing Integrity, and Privacy require separate evidence collection.
- **AI-generated evidence:** This package was assembled from files on disk and git history. Completeness depends on the comprehension gate having been consistently applied.
---
## Appendix: Artifact Details
[For each artifact in the Evidence Index — one subsection per artifact]
### [branch name] — [date]
**Verdict:** CLEAR / REVIEW REQUIRED / HOLD
**Findings:** [paste the findings table from the artifact]
**Artifact path:** [path]
After writing
Report:
- Number of artifacts included
- Coverage percentage (reviews / total significant changes)
- Number of control exceptions requiring auditor disclosure
- Path to the generated file
If control exceptions exist, say explicitly:
⚠ [N] control exception(s) found — changes with no review and no alternate evidence.
These must be disclosed to the SOC 2 auditor as control exceptions and remediated.
Recommended remediation: run /comprehension-gate retroactively on each exception and
document the retroactive review as supplemental evidence.