# Adaptive Harness

> Use this skill when auditing, simplifying, benchmarking, or refactoring an AI/agent project harness, including CLAUDE.md, AGENTS.md, slash commands, skills, subagents, hooks, tool routing, scheduled reviews, AI-review reports, benchmark scaffolds, and rolling harness maintenance. Not for ordinary code review of application code (use code-review) or for running the method-harness-compiler project (use the root harness ladder).

- Skill: `wenyuchiou/adaptive-harness` (Agent Skill)
- Install (CLI): `npx skillmds@latest add wenyuchiou/adaptive-harness`
- Raw SKILL.md: https://api.skillmd.com/api/skills/wenyuchiou/adaptive-harness/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: WenyuChiou (https://skillmd.com/u/wenyuchiou)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/wenyuchiou/adaptive-harness

---


# SKILL: adaptive-harness

**Relationship to the root `SKILL.md`:** the root file is the repo-level
launcher for working ON the method-harness-compiler project (the L0→L3
ladder). THIS adapter is the skill-aware runtime entry for the **adaptive
harness system** — the improvement loop that keeps a harness (this repo's or
the operator's global one) reviewed, simplified, and benchmarkable over time.
AI-review is the local reviewer; adaptive-harness runs the harness-shaped
review modes over its findings. One system, shared schemas, two runners.
Cross-run linkage (which findings repeat, which are closed) is answered
on demand by `scripts/grep_history.py` — the stateful rolling machinery was
retired per REC-20260714-001 after the pre-registered A/B measured no recall
advantage over re-derivation (a B-loses result that shipped to
`docs/evidence.md`).

## The loop you are operating

1. **Observe** — run AI-review (`python scripts/run_ai_review.py --mode <m>`)
   and, for harness-shaped scans, the adaptive runner modes below. For
   cross-run questions: `python scripts/grep_history.py --repeats` / `--open`
   / `--rec REC-YYYYMMDD-NNN` (read-only; append-only history + git log).
2. **Diagnose** — answer the mode's semantic checklist in
   `prompts/ai-review-modes.md` with cited evidence.
3. **Classify** — every finding gets Keep / Simplify / Remove / Replace /
   Merge / Cache / Experiment per `schemas/recommendation.schema.yaml`.
4. **Act safely** — low-risk docs drift may be edited only when the human
   explicitly allowed it THIS session; high-risk (hooks, subagents, prompts,
   permissions, settings, CI) ⇒ `--mode patch_proposal` renders an
   apply/rollback sheet and STOPS; uncertain value ⇒ add a case to
   `benchmarks/harness_cases.yaml` (pre-registered, never self-graded).
5. **Record** — findings enter via `--ingest findings.json` (validated);
   the runner appends JSONL history and renders JSON→MD deterministically.
   Never hand-write report numbers.
6. **Schedule next** — the report's `next_review_trigger` names the cadence;
   scheduled runs stay report-only.

## Modes

| Mode | Runner call | Semantic checklist |
|---|---|---|
| `harness_inventory` | `run_adaptive_harness_review.py --mode harness_inventory` | none (deterministic) |
| `harness_cleanup_review` | `--mode harness_cleanup_review` | ai-review-modes.md §harness_cleanup_review |
| `code_invocation_review` | `--mode code_invocation_review` | §code_invocation_review |
| `ai_review_integration` | `--mode ai_review_integration` | integration doc checklist |
| `skill_fit_review` | `--mode skill_fit_review` | do skill descriptions still match usage? dead skills? mis-triggering? |
| `diff_only_review` | `--mode diff_only_review --since-ref <ref>` | §diff_review |
| `scheduled_harness_review` | `--mode scheduled_harness_review` | NONE by design (report-only) |
| `experiment_design` | `--mode experiment_design` | §experiment_review |
| `patch_proposal` | `--mode patch_proposal` | render only; human applies |

(`rolling_improvement_review` was retired — applies REC-20260714-001;
linkage queries: `scripts/grep_history.py`.)

## Safety boundaries

Enforced BY CODE (runner invariants, regression-pinned):

- Scheduled runs are report-only: `--ingest` rejected, `source=
  scheduled_runner`, `changes_made` must be empty, zero writes outside
  `--output`.
- `--dry-run` writes nothing.
- Ingest findings are validated (required fields, enums, REC-id pattern,
  human-approval flag on high-risk Remove/Replace) or rejected.
- `grep_history.py` is read-only by construction; a legacy
  `rolling_state.json` (from the retired loop) is still READ by
  `patch_proposal` so pending items never vanish, but nothing writes it.

Operating POLICY (this skill's instructions to the agent — not machine-
enforced; violating them is a harness-governance violation):

- Findings are recommendations; the human decides what merges.
- Deleting prompts / subagents / hooks, changing permissions,
  `.claude/settings.json`, or CI: patch proposal + human approval, always.
- No self-modifying loop: never edit this SKILL.md, the runners, or the
  schemas without a human-reviewed commit.
- Loop-closure convention: a commit that applies a recommendation says
  `applies REC-YYYYMMDD-NNN` (or `resolves ...`) in its message; a revert
  says `reverts REC-...`; anything else must not bare-cite REC ids — the
  applies/resolves verb is what `grep_history.py` treats as closure.

## Model tiers

Deterministic runners + `grep_history.py`: any tier (code — Haiku or cron
can invoke).
`skill_fit_review` / `diff_only_review` semantic passes: Sonnet-capable.
`harness_cleanup_review` interpretation and
`patch_proposal` judgment: Opus/Fable-class. Codex: mechanical scoped work
only, never final authority (docs/codex-delegation-policy.md).
Executed tier evidence: `benchmarks/model_compatibility_cases.yaml` (Haiku
4/4, Sonnet 2/2 real runs; Codex live compliance UNVERIFIED — plan in
`docs/model_compatibility_test_plan.md`).

## Runtime portability (this skill is NOT Claude-Code-only)

The system's contracts are plain files + stdlib-Python CLIs; the skill
format is just one discovery wrapper. Entry per runtime:

| Runtime | Entry | Notes |
|---|---|---|
| Claude Code | this SKILL.md via the skill system | full loop incl. semantic checklists |
| Codex / Cursor / AGENTS.md-convention agents | `AGENTS.md` §Harness-maintenance path → this file as markdown | may RUN runners, draft findings JSON, execute scoped mechanical fixes under docs/codex-delegation-policy.md; NEVER final authority, never applies proposals |
| Hermes (or any router surface) | `prompts/hermes-router.md` routing row "harness maintenance" | runs deterministic scans directly; routes semantic checklist work to a strong reasoning surface |
| Human / cron / any shell | the CLIs directly | `python scripts/run_ai_review.py --mode … `; scheduled = report-only by code |

What travels with any runtime: the runners (stdlib, offline), the schemas
(`--ingest` validation is runtime-blind), the checklists (markdown), the
safety invariants (coded in the runners, not in this wrapper).

## Scenario matrix (適用不同場景)

| Scenario | Invocation | What degrades gracefully |
|---|---|---|
| This repo self-audit (default) | runners with no flags | nothing — full collector set |
| Any OTHER repo / project harness | `--target <repo-path>` | INDEX/ROUTES/benchmark collectors mark `skipped` when those files don't exist; report stays schema-valid |
| Operator's global harness (~/.claude) | default `--home` telemetry collectors; target stays a repo | missing ~/.claude scripts mark `unavailable`, never fabricated |
| Headless / scheduled | `--mode scheduled_review` / `scheduled_harness_review` | report-only BY CODE: --ingest rejected, no ledger writes, no proposals |
| Hermetic / CI | `--no-home --dry-run` | zero machine-specific dependencies, zero writes |