Exploratory Nightly Verification

Exploratory Nightly Verification

Run from a nightly CI job (e.g. anthropics/claude-code-action on a schedule) to explore the main branch in a 60-minute budget. Targets the library / app itself, not a specific PR.

This skill is a sibling to exploratory-pr-verification, but differs significantly:

Bundled scripts (run as-is)

Let SKILL_ROOT be the directory containing this SKILL.md (in Claude Code: ${CLAUDE_SKILL_DIR}). The deterministic parts of this workflow are implemented as scripts. Run them as-is — do not read them to re-derive their logic, do not rewrite them, and do not hand-roll replacements. They require bash, git, curl, and jq (jq for summary.sh / check-upstream.sh).

All of them accept --date YYYYMMDD (default: today). Capture date from the init-run.sh output and pass it explicitly to every later call — that keeps a run that crosses midnight in one directory.

0. Invariants (do not violate)

1. Workflow (PDCA compressed into 60 minutes)

1. Environment confirmation (~3 min)
   • Run "$SKILL_ROOT/scripts/init-run.sh" — note date / dir / commit from its JSON
   • git status --short for a dirty-tree sanity check
   • Note the current versions from gradle/libs.versions.toml or equivalent
2. Plan (~5 min)
   • Read references/categories.md, pick the cat1 → cat5 order
   • Note a per-cat time budget in TodoWrite or a scratch list
3. Sequential exploration (~50 min)
   • cat1 → cat2 → cat3 → cat4 → cat5, ~8–10 min each
   • For every finding, immediately create the issue file with scripts/new-issue.sh (hand-writing an issue file is forbidden), then fill in its body sections
   • One file written → next finding
4. Final formatting (50–58 min)
   • Re-read all issue files: deduplicate, polish titles, tighten the ## Detail body
   • Run "$SKILL_ROOT/scripts/summary.sh" --date <date> — it writes SUMMARY.md and reports format lint as JSON; fix any lint findings and re-run until "ok": true
5. Termination report (~60 min)
   • Final message: "wrote N issues, wrote SUMMARY.md". That's all.

2. Categories

Five categories run sequentially. Full details in references/categories.md. Summary:

• cat1 — Static analysis: KDoc / SoT / silent failure / type design / TODO grep
• cat2 — CI logs, public-API baseline, docs consistency: nightly warning grep, baseline drift, README ↔ docs consistency
• cat3 — Dynamic build / test: run the project's primary test task (and integration / docs variants if present); capture failures and warnings
• cat4 — Upstream release watching: run scripts/check-upstream.sh for the Kotlin / Compose Multiplatform / Gradle / AGP / AndroidX drift table; your only judgment is whether a drifted release announces breaking changes (P1) or not (P2)
• cat5 — Comparison / leftover angles: sibling-library comparison, residual TODO grep, semver / BCP review, sample-app launch sanity

Parallelism is off — single CI VM with a 60-minute budget cannot afford parallel build contention. Sequential is the safe default.

3. Issue Markdown format

Every issue file must be created via scripts/new-issue.sh — never hand-write one. The script owns the path, the atomic gapless numbering, the H1, and the metadata lines; you own only the body sections (## Reproduction, ## Detail, optional ## Fix proposal):

"$SKILL_ROOT/scripts/new-issue.sh" cat2 P1 bcv-baseline-drift "BCV baseline diverges" --date <date>
# → prints .local/tmp/exploratory-nightly-<date>/issues/<NN>-bcv-baseline-drift.md

Full spec in references/issue-format.md. Required fields:

• Path: .local/tmp/exploratory-nightly-<date>/issues/<NN>-<kebab-slug>.md
  • <NN>: zero-padded 2-digit sequential number, starting at 01
  • <kebab-slug>: lowercase + hyphens, ≤ 40 chars
  • <date>: YYYYMMDD (e.g. 20260513)
• Line 1: # <Short title> (≤ 60 chars; non-Latin OK)
• Then these metadata lines + headings:
  • **Category**: cat<N> (<name>)
  • **Severity**: P0 | P1 | P2 | P3
  • **Detected at commit**: <first 12 chars of git rev-parse HEAD>
  • ## Reproduction
  • ## Detail
  • ## Fix proposal (optional)

The title, the metadata lines, and the first few lines of ## Detail are the candidate excerpt for any downstream notification (Slack / Discord / email). Write them at a granularity that's meaningful without the surrounding context.

4. Severity reference

P0 / P1: write the issue immediately when found. P2 / P3: write if time allows.

5. Tool selection by category

• Read / Glob / Grep: static analysis (cat1, cat2, cat5)
• Bash: Gradle / git / API-dump roundtrip (cat2, cat3)
  • Long tasks: timeout 8m <cmd> to protect the 60-min budget
  • Always redirect to .local/tmp/exploratory-nightly-<date>/tmp/<ts>-<cmd>.log — don't truncate with grep / head on the live stream
• Bash + scripts/check-upstream.sh: cat4 version-drift table (deterministic; no manual WebFetch of version pages)
• WebFetch: release notes of versions that check-upstream.sh reports as drifted (cat4 breaking-change judgment only)
• WebSearch: cat5 leftover-angle exploration

The .local/tmp/exploratory-nightly-<date>/tmp/ log files are not artifact-preserved by default — only the issue Markdown files travel out of the job. Whatever matters must be transcribed into the issue file before the job ends.

6. Unexpected-failure handling

• A check-upstream.sh entry with "drift": "error" (rate limit / fetch failure) or a WebFetch failure → write that fact itself as a P3 issue, proceed to the next category
• Gradle task failure → write as P1 (or P2 if the failure is clearly CI-environment-specific)
• This SKILL.md fails to load → the wrapping CI step's if: always() should still surface the failure; no recovery in the skill itself

7. Termination

After the exploration loop ends, run "$SKILL_ROOT/scripts/summary.sh" --date <date>. It writes .local/tmp/exploratory-nightly-<date>/SUMMARY.md (shape below) and prints a JSON report with a lint array; fix any lint findings (dedup, body polish) and re-run until "ok": true. Do not write SUMMARY.md by hand.

# Nightly Exploration Summary

- Run timestamp: <ISO 8601>
- Target commit: <git rev-parse HEAD>
- Issue count: N

## Per-category breakdown
- cat1: N
- cat2: N
- cat3: N
- cat4: N
- cat5: N

## Per-severity breakdown
- P0: N
- P1: N
- P2: N
- P3: N

Then stop. Do not re-kick. Do not call /loop. Do not call ScheduleWakeup. The final user message says only "wrote N issues, wrote SUMMARY.md".

8. Notification integration (optional)

If the wrapping CI workflow uploads the issue Markdown files to Slack / Discord / email, the filename ordering and the issue-format spec (references/issue-format.md) are the contract.

A typical pipeline:

1. This skill writes .local/tmp/exploratory-nightly-<date>/issues/<NN>-<slug>.md
2. A separate CI step (a shell script, not this skill) parses the directory in sort order and produces the notification payload

Keep the parsing contract simple — the H1 line for the title, the **Category** / **Severity** / **Detected at commit** lines for metadata, the first 200 chars of ## Detail for the body excerpt. See references/issue-format.md for the canonical spec.

The notification integration is outside this skill's scope. The skill writes Markdown; the notification step parses Markdown.